Anyone who has fed web pages to an agent knows where the pain sits: the target site fights back, the content only appears after scripts run, and the article body is tangled up with navigation and advertising. This product cuts exactly at that layer. It calls itself a web data interface — send a URL, receive clean markdown or structured JSON — and the dirty work happens on the server. The homepage slogan is blunt about the purpose, and the project is also open source, which explains how the repository collected a little over 180,000 stars.

Three jobs: search, scrape, interact

The core splits into three. Search finds web pages and brings back the full text of the result pages with them, which suits the pattern of locating something and then reading it. Scrape turns any URL into markdown, JSON or a screenshot, and for structured output you can hand over a JSON template and get your fields back in that shape. Interact is the newest of the three and the least obvious: the session does not end when the page is fetched, so natural language or code can keep clicking, filling forms, pulling dynamic content and going deeper. The official example fetches a shop's homepage, then searches for a product inside it, opens the result and reads the price. The value of tying the three together is that in many collection jobs the real difficulty is the third step — a static copy of the HTML is often not enough.

Beyond one page: whole sites, batches, local files

Around the core endpoints sits a ring of supporting abilities. Map discovers every URL under a site, Crawl takes the site recursively, and batch scraping handles hundreds of thousands of URLs asynchronously. Parsing is not limited to live pages either: local PDF, DOCX, XLSX and HTML files all convert to markdown or structured JSON. There is also an autonomous agent mode, where you describe what you want and it goes collecting; webhooks that deliver results asynchronously; and a browser sandbox that supplies hosted browser sessions for interaction flows. Actions before a scrape get their own hooks too — clicking, scrolling, typing, waiting and key presses can all be scripted into the flow.

Ways to connect are thorough. It runs as a model-context-protocol server, so one command attaches it to a coding agent. It also ships a command-line tool that installs itself and injects a skill into whatever coding agent it detects. SDKs cover Python, Node and the command line, and the integration list includes the two most common retrieval frameworks and several model providers. The documentation even recommends starting without a key: the first requests need no API key at all, and adding one only relaxes the rate limit.

Free allowance and how billing is counted

The quota rules are unusually legible. Every month brings 1,000 credits, roughly a thousand pages, with no credit card required. Above that sit four tiers, with an enterprise tier negotiated separately and an annual subscription adding two months free. The unit shifts with the ability: scraping, whole-site crawling, URL discovery and monitoring each cost one credit per page; search costs two credits per ten results; page interaction costs two credits per browser minute. Carry-over differs too — unused credits on the top two tiers roll into the next month, on the lower tiers they do not. The official FAQ adds two boundaries: a scrape that returns nothing costs nothing, and pages that answer with an error status are still delivered to you. A five-dollar usage top-up converts to 1,000, 2,000, 2,500 or 5,000 credits depending on the tier, so the same money buys more the higher you sit.

The licence shape deserves attention. The project is mainly AGPL-3.0, which means running a modified version as a public service obliges you to publish your changes; the SDKs and some interface components are MIT, so calling the hosted interface is far less constrained. The hosted version also carries abilities the self-hosted one does not, and the feature list in the repository and documentation lets you compare. On compliance, the vendor states that robots.txt is respected by default and puts responsibility for each target site's policy on the user — a sentence worth reading word by word before commercial collection.

Who it fits

Three situations map cleanly. Teams building the retrieval and tool layer for agents, who need web pages turned into a controllable format reliably and do not want to maintain scraping and anti-bot infrastructure of their own. Teams running data pipelines and lead enrichment, where batch scraping plus a structured template maps a messy page straight onto database fields. And individuals or small teams validating an idea, since the free allowance is enough to run a flow end to end and either the protocol server or the command-line tool attaches within minutes.

Two trade-offs are worth naming. The licence first: AGPL-3.0 constrains selling a modified version as a service, which does not affect calling the hosted interface but does mean thinking through obligations before offering a self-built one publicly. Second, self-hosting against the cloud version: self-hosting keeps data and cost under control, at the price of carrying anti-bot strength and page coverage yourself; the cloud version hosts all of that, at the price of metered billing and handing requests to a third party. Repository figures read about 181.6k stars, 9.8k forks, 35 releases, 173 contributors and version v2.11.0, mostly TypeScript with Python and Rust alongside, covering 96 percent of pages at a P95 latency of 3.4 seconds — figures the vendor supplies, which your own target sites still deserve to test.

The worth of a scraping tool is not whether it can fetch. It is whether what it fetches can go straight into your downstream. Clean that layer and the labour actually disappears.