Quickstart
.await it (or call .send().await). Options you leave out are not sent, so the API’s own defaults apply. A builder owns a cheap clone of the client, so a request can be handed straight to tokio::spawn.
Get Markdown for LLMs
get_markdown() returns the page as clean Markdown for LLMs, agents and RAG pipelines. By default you get the main content only. It takes the same options as get_page_source(), and is billed the same.
Get parsed JSON instead of HTML
get_parsed() returns structured data extracted using Schema.org, __NEXT_DATA__, or AI-generated rules - no per-site parsers to maintain.
get_parsed() does not fail: data_extracted is false, detail says why, and html carries the rendered page. The call is billed like a page-source call. See when nothing can be extracted.
Scrape a Google SERP
.full_urls(true) waits until every result has its real URL (up to about 40 seconds longer, billed the same). See full_urls.
Capture cookies and the proxy used
Fetch an image
Drive the browser with steps
Pass actions to run in a real browser after the page loads. You get back the HTML of the resulting state.list_elements(), which returns the page’s interactive elements as JSON:
selector drops straight into a Step. Full action reference in the browser steps guide.
Capture the page’s own API calls (XHR/fetch)
Many sites render from a JSON API the page calls itself, and that API often refuses direct requests.get_page_with_xhr renders the page in a real browser and
returns every XHR/fetch response whose URL contains the fragment you pass, in
the order the page received them. Add steps to make the page fire the call
(a search, “load more”).
url, method, status, content_type, body (text) and
body_omitted (why there is no body). Up to 20 responses and 10 MB of bodies;
page.xhr_truncated is true when more matched. When the site answers every matching
call with 403 or 429, the request fails like a block and is not billed. Rules
and the response shape: capturing the page’s API calls.
Plugins
Every plugin has a method that returns its JSON as aserde_json::Value. The plugin’s required input goes in positionally; every other parameter is a setter. Each plugin page documents the parameters; the Rust method is the Python method’s name (target_search, google_trends), and Skyscanner’s are skyscanner_flights, skyscanner_hotels and skyscanner_carhire.
Concurrency
Plan limits are concurrency: how many requests run at the same time on a key. Cap your own in-flight requests to match, for example with atokio::sync::Semaphore:
Blocking client
Enable theblocking feature for code without an async runtime. The methods are the same; finish a request with .send():
reqwest::blocking, it runs its own small runtime, so do not use it inside async code.
Error handling
Every call returnsscrapeunblocker::Result<T>. An error response from the API is Error::Api, and its kind says what went wrong:
ApiErrorKind covers every status the API sends: InvalidRequest (400), Authentication and NoSubscription (401), QuotaExceeded, CreditLimitExceeded, BudgetExceeded, PaymentFailed and PaymentRequired (402), Blocked (403), NotFound and TargetNotFound (404/410), BrowserTimeout (408), UnsupportedContent (415), Validation and StepFailed (422, a failed step carries step_failed with step_index, action, reason, selector and html), RateLimit (429), UpstreamOutage (503) and Server (other 5xx). See errors for what each means.
HTTP 429, 502, 503, 504 and network errors are retried automatically with exponential backoff (Client::builder().max_retries(n), default 2). A 401 or 402 is never retried, and neither is billed.

