ScrapeUnblocker Documentation

ScrapeUnblocker uses standard HTTP status codes. 2xx means the page was delivered, 4xx points at the request or at the target site, and 5xx means a problem on our side or a request that ran out of time. Only a few responses are billed - they are marked in the table; everything else is free.

Status codes

Code	Meaning	Billed	Where to look
`200`	The page was delivered. Also used for a dead origin’s own `5xx` error page and for the target’s own `401`, `451` or empty `500` - then the `X-Origin-Status` header carries the target’s status. A target that answers `204` (no content) comes back as an empty `200`. See Dead URLs	Yes	Response body; `X-Origin-Status` if present
`400`	Bad request: the `x-scrapeunblocker-key` header is missing, the URL is malformed or not `http`/`https`, an unsupported `proxy_country`, or `Hostname cannot be resolved` (the domain does not exist)	Only `Hostname cannot be resolved`	Check the URL and the key header
`401`	Authentication problem - the key is not recognised, or the account behind it has no valid subscription	No	See 401 unauthorized below and Authentication
`402`	Billing problem - quota exceeded, credit limit exceeded, repeated payment failures, or your own monthly budget limit reached	No	See 402 payment required below
`403`	Either the target blocked us on every bypass path, or the URL is a login / authentication / payment page we do not fetch (`X-SU-Error-Code: restricted_target`), or a government site (`X-SU-Error-Code: government_target`), or a `steps` action tried to fill a password or payment-card field (`X-SU-Error-Code: restricted_action`)	No	`X-SU-Error-Code` header; restricted targets or handling failures
`404`	With `X-Origin-Status: 404`: the target site itself says the page does not exist; the body is the target’s page. See Dead URLs. Without the header: a plugin found no such item, or the API path is wrong	Only with `X-Origin-Status`	`X-Origin-Status` header, then the body
`410`	The target site itself says the page is gone (`X-Origin-Status: 410`); the body is the target’s page. See Dead URLs	Yes	Drop the URL
`413`	`/getImage`: the file is larger than 50 MB	No	-
`415`	`/getPageSource`: the URL is not an HTML document (an image, PDF or file download) - use `/getImage` for images. `/getImage`: the URL is not an image (for example a page); `X-Resource-Content-Type` names what it is	No	Fix the URL or the endpoint
`422`	Validation error - missing required field or wrong type. With `steps`, also `step_failed` when an action could not run. (With `parsed_data=true`, a page that rendered but held no structured data is not an error: it is a `200` with `data_extracted: false` and the page’s `html` - see parsed data)	Only `step_failed` (the page was rendered)	The response body pinpoints the problem field or step
`429`	Concurrent requests limit exceeded (body `Concurrency limit exceeded`, headers `X-SU-Deny-Reason: concurrency` and `Retry-After: 1`)	No	Wait `Retry-After` seconds for a running request to finish, then retry. See concurrency limits
`500`	Internal error on our side	No	Retry with backoff
`502`	Our gateway lost the connection to the scraping backend (HTML body). Dedicated plugin endpoints also return `502` with a plain-text message when the target returned no usable results or the call did not finish in time (`... timed out.`)	No	Retry with backoff
`503`	No API capacity right now (`No server is available to handle this request`, or `Render unavailable.`) - a problem on our side, not the target’s	No	Retry with backoff
`504`	The request reached its time cap: `Request timed out.` (60 s on `/getPageSource`, 90 s with `steps`, 75 s on `/getImage`) or a SERP timeout. Usually a slow target	No	Retry. If persistent for one domain, contact support
`523`	`Origin unreachable` - the target answered nobody over any of our exits (down, or refusing connections in bursts). Not a bot block. A target that accepts the connection but is slow to answer is not a `523`: we keep waiting for it within the time cap	No	Retry after a pause

Dead URLs: `X-Origin-Status`

When the target site itself answers that the page does not exist (404 or 410), that answer is the result for your URL - we reached the site, the URL is just dead. ScrapeUnblocker returns the target’s own status (404 or 410) with the page exactly as the target served it (the body can be empty; with get_cookies=true it is the usual JSON, and with parsed_data=true the body is {"data": {"page_type": "not_found", "data": {}}} - there is nothing to extract from a not-found page) and adds:

Header	Value
`X-Origin-Status`	The target’s own status, e.g. `404`, `410`, `502`
`X-ScrapeUnblocker-Note`	A short human-readable explanation

These calls are billed like any delivered page, and retrying will not change the answer. X-Origin-Status is what tells the target’s 404 apart from a 404 of our own (a wrong API path) - branch on it. On /getImage a dead image comes back the same way, with an empty body:

r = requests.post(
    "https://api.scrapeunblocker.com/getPageSource",
    params={"url": url},
    headers={"x-scrapeunblocker-key": "YOUR_API_KEY"},
    timeout=125,
)
if r.status_code in (404, 410) and r.headers.get("X-Origin-Status"):
    mark_as_dead(url)  # the page no longer exists on the target site

When the target’s origin server returns its own 5xx error page (for example nginx 502 Bad Gateway), you get a 200 with that page, X-Origin-Status carrying the origin’s status and X-Upstream-Unavailable: true, also billed. It stays a 200 on purpose: most HTTP clients retry 5xx automatically, and every retry would be billed. The target’s own 401 (the page needs a login), 451 (unavailable for legal reasons) and a 500 with no body at all come back the same way, once a second exit has seen the same answer: a 200 with the body exactly as the target served it (often empty) and X-Origin-Status: 401, 451 or 500, billed. They are never passed through as the status itself - our own 401 always means a problem with your API key, and our own 500 an error on our side.

Until 2026-09-30 a target’s 404 / 410 also came back as a 200 with X-Origin-Status. Code that checks r.status_code == 200 first should now handle 404 / 410 with the header as a dead URL, not as an error to retry.

A URL whose domain does not exist gets 400 Hostname cannot be resolved instead, also billed.

Is it the target, or is ScrapeUnblocker down?

If you run failover to another provider, only these mean a problem on our side: connection errors, read timeouts, 500, 502, 503, and 504 when it hits many different domains at the same time. All other codes above describe the target site or the request.

A call never hangs indefinitely: if our gateway is unreachable you get a connection error, and if no scraping capacity is available you get an immediate 503. A healthy call takes at most its time cap plus up to 60 s of queueing when all capacity is busy. Use a connect timeout of about 10 s and a read timeout of about 125 s.
Decide on a rate, not on a single response - one failed call is often one difficult domain. For example, switch over when at least half of the calls in the last minute (at least 20 calls, across at least 3 domains) end with one of those signals, and switch back once probe calls succeed.
Responses our gateway generates itself (502, 503, 504) are never billed.

401 unauthorized

A 401 means the request was rejected at our edge, before it ever reached the scraping engine. Because nothing was scraped, a 401 does not count against your quota and is never billed. Unlike validation errors, 401 responses have a plain-text body (Content-Type: text/plain), not JSON. There are two distinct messages, and they mean different things:

Response body	Meaning
`Unauthorized`	The key you sent is not recognised
`No valid subscription`	The key is recognised, but the account behind it has no active subscription

`Unauthorized`

HTTP/1.1 401 Unauthorized
Content-Type: text/plain

Unauthorized

The value of your x-scrapeunblocker-key header does not match any known key. Common causes:

A typo, or a truncated copy-paste. Keys are long; make sure the whole value was copied.
Trailing whitespace or a newline in the header value - especially when the key is read from a file rather than an environment variable.
An empty header value. Sending x-scrapeunblocker-key with nothing after it counts as an unknown key, not as a missing header.
A rotated or revoked key. After you generate a new key in the dashboard, the old one stops working once its short grace period ends.
Environment mismatch. Production keys only work against api.scrapeunblocker.com. A key issued for one environment sent to another is an unknown key there.

`No valid subscription`

HTTP/1.1 401 Unauthorized
Content-Type: text/plain

No valid subscription

The key itself is valid, but the account it belongs to currently has no subscription period covering today - for example the free trial has ended and no plan was chosen, or a plan lapsed and was not renewed. Pick a plan in the dashboard and access resumes within about a minute; no key change is needed.

A billing problem on an active subscription returns 402, not 401 - quota exceeded, credit limit exceeded, or a card that failed repeatedly. 401 is strictly about who you are, 402 about what you owe.

Missing header returns `400`, not `401`

If you omit the x-scrapeunblocker-key header entirely, the response is 400 Bad Request with the body Missing x-scrapeunblocker-key. This is deliberate: it separates “you forgot to authenticate” from “you authenticated, and it was rejected”, so client code can tell a wiring bug from a credential problem.

HTTP/1.1 400 Bad Request
Content-Type: text/plain

Missing x-scrapeunblocker-key

402 payment required

A 402 means your key and account are recognised and in good standing as credentials - the request was stopped for a billing reason. Like 401, it is refused at our edge before anything is scraped, so a 402 consumes no quota and is never billed. The body is plain text (Content-Type: text/plain), not JSON. There are four messages:

Response body	Meaning	Fix
`Quota exceeded`	You have used every request your plan allows this billing period	Upgrade your plan, or wait for the period to reset
`Credit limit exceeded`	Your unpaid balance has grown past your account’s credit limit	Pay the outstanding invoice
`User set budget exceeded`	This billing period’s spend has reached the monthly budget limit you set	Raise or remove the limit in your profile, or wait for the next billing period
`Payment failed - update payment method`	A card payment for an open invoice has failed three times in a row	Update your card, then pay the invoice

If more than one applies, the most serious wins: payment failure outranks credit limit, which outranks quota, which outranks your own budget limit.

All four clear themselves. Our gateway refreshes key statuses about once a minute, so once you upgrade, the invoice is paid or you raise your budget limit, access comes back within roughly a minute - no key change, no support ticket, no redeploy.

`Quota exceeded`

HTTP/1.1 402 Payment Required
Content-Type: text/plain

Quota exceeded

Your usage for the current billing period has passed your plan’s quota. If your plan allows overages, this only fires once you are past quota plus the overage allowance - inside that band requests still succeed and the extra usage is invoiced. Any active coupon credit is spent before plan quota, so a key with remaining credit is never quota-blocked. The counter resets at the start of your next billing period, which starts on your subscription’s anniversary day, not on the first of the month. To get moving sooner, upgrade in the dashboard - the new quota applies on the next status refresh. Current usage against quota is visible in the dashboard, so this is the one 402 you can see coming.

`Credit limit exceeded`

HTTP/1.1 402 Payment Required
Content-Type: text/plain

Credit limit exceeded

This applies to accounts that accrue usage-based charges. We add up what you currently owe - the amount remaining on your open invoices, plus metered usage already consumed on active subscriptions but not yet invoiced - and compare it against your account’s credit limit. Past the limit, the key is paused. When this triggers we also finalise and attempt payment on the outstanding invoices automatically, so in the common case where your card is good it settles itself and access returns within about a minute. If payment does not go through, pay the invoice from the dashboard. A higher credit limit can be arranged through support.

`User set budget exceeded`

HTTP/1.1 402 User set budget exceeded
Content-Type: text/plain

User set budget exceeded

You set a monthly budget limit in your profile, and this billing period’s spend has reached it. Spend is counted the way your invoice is: your plan’s fixed monthly fee, if it has one, plus the requests billed on top of it, excluding VAT. Requests paid from coupon credit do not count. We also email you once when the limit is reached. On a plan with a fixed monthly fee the limit cannot be lower than that fee, and the requests the fee already includes are never blocked - a limit equal to the fee simply stops the key where overage billing would begin. Leave the field empty or set it to 0 for no limit. The key works again at the start of your next billing period, or within about a minute after you raise or remove the limit. Usage is counted with a delay of a few minutes, so the period’s spend can end up slightly above the limit.

`Payment failed - update payment method`

HTTP/1.1 402 Payment Required
Content-Type: text/plain

Payment failed - update payment method

An invoice on your account is open and its payment has been attempted and declined three times. Those attempts are our payment provider’s automatic retries spread over several days, so reaching this state means a card has been failing for a while - typically expired, cancelled, or short of funds. Update your payment method in the dashboard and settle the open invoice. As soon as the invoice is paid the block lifts on the next status refresh, within about a minute.

Subscribing to a new plan does not clear this on its own. The old unpaid invoice stays open, so the block stays in place until that specific invoice is paid, even if the new subscription is active and paid for.

`402` versus its neighbours

Code	It means
`401`	We do not accept your credentials - unknown key, or no subscription at all. See 401 unauthorized
`402`	We accept your credentials; the account owes money, is out of quota, or hit the budget limit you set
`429`	Nothing is wrong with the account - you have more requests in flight at once than your plan’s concurrent requests limit. See concurrency limits

422 validation error shape

When you send an invalid request body, /getPageSource, /serpApi, and /getImage all return a structured validation error:

{
  "detail": [
    {
      "loc": ["query", "url"],
      "msg": "field required",
      "type": "value_error.missing"
    }
  ]
}

loc is the path to the problem field. msg is human-readable. type is a stable machine-readable identifier.

Restricted targets

ScrapeUnblocker does not fetch sign-in or payment surfaces or government sites, and it does not fill credential or payment forms. Three refusals enforce this. All are returned at our edge (or before any action runs), are never billed, and carry an X-SU-Error-Code header so you can tell them apart from an ordinary bot block.

`X-SU-Error-Code`	When	Endpoints
`restricted_target`	The URL is a login, authentication or payment / checkout page - a known identity or payment host (for example `accounts.google.com`, `checkout.stripe.com`), or a path whose segment is one of `login`, `sign-in`, `oauth`, `checkout`, `payment`, and similar. The request is refused before a browser is launched	`/getPageSource`, `/getImage`
`government_target`	The URL is a government, military or intergovernmental site (for example `.gov`, `.mil`, `.int`, `gov.uk`, `gouv.fr`, `go.jp`), or a URL that only wraps one: a redirect link whose query carries the government URL, a `.translate.goog` proxy page, or an archive copy such as `web.archive.org/web/.../https://www.irs.gov/`. The request is refused before anything is fetched	`/getPageSource`, `/getImage`
`restricted_action`	A `steps` `type` action is about to fill a field the page marks as a password or a payment-card field. The run stops on that step	`/getPageSource` with `steps`

A restricted_target refusal is a plain-text 403:

HTTP/1.1 403 Forbidden
Content-Type: text/plain
X-SU-Error-Code: restricted_target

This target is a login, authentication or payment page. ScrapeUnblocker does not fetch sign-in or payment surfaces...

A restricted_action refusal is a JSON 403 naming the step:

{
  "error": "restricted_action",
  "detail": "This step would enter a password or payment-card field. ScrapeUnblocker does not fill credential or payment forms.",
  "step_index": 1,
  "field": "password"
}

Matching is deliberately narrow: a whole path segment must match, so an article at /payment-policies or /how-to-login-safely is fetched normally - only the login or payment surface itself is refused. Ordinary steps into search boxes, filters and forms are unaffected. If you have a legitimate need for such a target, contact support.

403 - blocked vs. invalid key

A 403 from ScrapeUnblocker never means your API key is wrong. Invalid keys return 401. Otherwise a 403 is one of two things, which the X-SU-Error-Code header tells apart: a restricted target (the header is set), or the target site blocked us on every bypass route we tried (no such header). When you see a 403 with no X-SU-Error-Code (a bot block):

Try a different proxy_country. Some sites geo-fence or geo-rotate their bot protection. A US site may be unreachable from EU IPs and vice versa.
Wait and retry. Rate-based blocks expire after a few minutes.
Contact support if the same URL repeatedly fails - we may need to add a custom plugin for that domain.

A 403 with an X-SU-Error-Code is a policy refusal, not a bot block - retrying or changing proxy_country will not help. See restricted targets. More detail in the handling failures guide.

Retries and idempotency

All three endpoints are safe to retry. Requests are idempotent in the sense that retrying with the same parameters does not double-charge or create duplicate state on your account. We recommend exponential backoff for transient 5xx errors and 523 (failed calls are free, so a retry costs nothing extra):

import time
import requests

def fetch_with_retry(url, max_attempts=3):
    for attempt in range(max_attempts):
        r = requests.post(
            "https://api.scrapeunblocker.com/getPageSource",
            params={"url": url},
            headers={"x-scrapeunblocker-key": "YOUR_API_KEY"},
            timeout=125,
        )
        if r.status_code == 200:
            return r
        if r.status_code in (500, 502, 503, 504, 523) and attempt < max_attempts - 1:
            time.sleep(2 ** attempt)
            continue
        r.raise_for_status()
    return r

Get Started

No-code area

Spider Cloud

Plugins

AI Assistants

Guides

Code Examples

Errors

Status codes

Dead URLs: `X-Origin-Status`

Is it the target, or is ScrapeUnblocker down?

401 unauthorized

`Unauthorized`

`No valid subscription`

Missing header returns `400`, not `401`

402 payment required

`Quota exceeded`

`Credit limit exceeded`

`User set budget exceeded`

`Payment failed - update payment method`

`402` versus its neighbours

422 validation error shape

Restricted targets

403 - blocked vs. invalid key

Retries and idempotency

​Status codes

​Dead URLs: X-Origin-Status

​Is it the target, or is ScrapeUnblocker down?

​401 unauthorized

​Unauthorized

​No valid subscription

​Missing header returns 400, not 401

​402 payment required

​Quota exceeded

​Credit limit exceeded

​User set budget exceeded

​Payment failed - update payment method

​402 versus its neighbours

​422 validation error shape

​Restricted targets

​403 - blocked vs. invalid key

​Retries and idempotency

Status codes

Dead URLs: `X-Origin-Status`

Is it the target, or is ScrapeUnblocker down?

401 unauthorized

`Unauthorized`

`No valid subscription`

Missing header returns `400`, not `401`

402 payment required

`Quota exceeded`

`Credit limit exceeded`

`User set budget exceeded`

`Payment failed - update payment method`

`402` versus its neighbours

422 validation error shape

Restricted targets

403 - blocked vs. invalid key

Retries and idempotency