Web Scraper
Lookups
Web Scraper
Scrape any URL and get clean, LLM-ready markdown content with metadata. Handles JavaScript rendering, proxies, and anti-bot bypass automatically.
POST
Web Scraper
Authentication
Requires an API key in theAuthorization header.
Request
The URL to scrape. Must be a valid
http:// or https:// URL.Render the page in a headless browser before extracting content. Use for JavaScript-heavy sites. Default:
false.Response
Returns the page content as clean markdown along with HTTP status and page metadata.Credits
2 credits per scrape. Failed scrapes (blocked, captcha, rate-limited) are automatically refunded.Related Endpoints
POST /api/agent/extract— Extract structured data from a page using a JSON schema (3 credits)POST /api/agent/screenshot— Capture a full-page screenshot (5 credits)