ScrapeUnblocker Documentation

The official Python client wraps the ScrapeUnblocker API in a small, fully typed package - sync and async, with typed errors and automatic retries.

pip install scrapeunblocker

Requires Python 3.8+. The only dependency is httpx.

Prefer no dependencies? The API is plain HTTP, so you can skip the package and call it with requests / httpx directly - see Raw HTTP at the bottom.

Store your API key in an environment variable so the client picks it up automatically:

export SCRAPEUNBLOCKER_KEY="YOUR_API_KEY"

Quickstart

from scrapeunblocker import Client

su = Client()   # reads SCRAPEUNBLOCKER_KEY, or pass Client(api_key="YOUR_API_KEY")

html = su.get_page_source("https://example.com")

Get parsed JSON instead of HTML

The killer feature: get_parsed() returns structured data extracted using Schema.org, __NEXT_DATA__, or AI-generated rules - no per-site parsers to maintain.

result = su.get_parsed("https://www.amazon.com/dp/B08N5WRWNW")

print(result.page_type)         # "product"
print(result.data["title"])
print(result.data["price"])

If a parse ever comes back wrong, force a fresh set of rules:

result = su.get_parsed(url, refresh_rules=True, rules_hint="price is missing")

When the page loads but nothing can be extracted from it, get_parsed() does not raise: it returns a result with page type unknown and empty data, together with the rendered HTML (the API’s data_extracted: false and html fields). The call is billed like a page-source call. See when nothing can be extracted. See the parsed data guide for response shapes.

Scrape a Google SERP

serp = su.serp("best running shoes", pages_to_check=2)

for result in serp["organic"]:
    print(result["position"], result["title"], result["url"])

Force a country

proxy_country works on every method.

result = su.get_parsed(
    "https://www.amazon.de/dp/B08N5WRWNW",
    proxy_country="de",
)

Capture cookies and the proxy used

page = su.get_page_with_cookies("https://example.com")

print(page.html)
print(page.cookies)
print(page.proxy)     # e.g. "us"

Fetch an image

data = su.get_image("https://example.com/photo.jpg")

with open("photo.jpg", "wb") as f:
    f.write(data)

Drive the browser with steps

Pass a native list of actions to run in a real browser after the page loads. You get back the HTML of the resulting state.

html = su.get_page_source(
    "https://example.com",
    steps=[
        {"action": "type", "selector": "input[name=q]", "value": "wireless headphones"},
        {"action": "press_key", "value": "Enter"},
        {"action": "wait_for", "selector": ".search-results", "timeout_ms": 10000},
    ],
)

Discover the selectors first with list_elements(), which returns the page’s interactive elements as parsed JSON:

page = su.list_elements("https://example.com")

for el in page["elements"]:
    print(el["tag"], el["selector"], el["text"])

Each element’s selector drops straight into a steps action. Full action reference in the browser steps guide.

Amazon

Product and search data as JSON, priced in the marketplace’s own currency:

# One product by ASIN (or url="https://www.amazon.de/dp/B0BSHF7WHW")
product = su.amazon_product(asin="B0BSHF7WHW", marketplace="amazon.com")
print(product["title"], product["price"], product["currency"], product["rating"])

# Keyword search
results = su.amazon_search("wireless headphones", sort="price_asc")
for item in results["results"]:
    print(item["title"], item["price"], item["currency"], item["asin"])

proxy_country defaults to the marketplace’s home country (amazon.com -> US, amazon.de -> DE), so prices come back in the right currency with no configuration. Full field list on the Amazon plugin page.

eBay search

Listings from any of the 19 regional eBay marketplaces as JSON:

items = su.ebay_search(
    "iphone 13",
    marketplace="ebay.com",
    condition="used",
    sort="newly_listed",
)

if items["exactMatches"]:
    for item in items["results"]:
        print(item["title"], item["price"], item["currency"], item["condition"])

exactMatches is False when eBay found nothing for the keyword and answered with its own loosely-related suggestions instead. Full field list on the eBay Search plugin page.

Skyscanner plugins

Flights, hotels and car hire as JSON. Resolve a place name to entity IDs, then search:

locations = su.skyscanner.flight_locations("London")

flights = su.skyscanner.flights(
    origin="London", dest="New York",
    depart_date="2026-09-01", adults=1, currency="USD",
)

hotels = su.skyscanner.hotels(
    destination="Madrid", checkin="2026-09-01", checkout="2026-09-03",
)

cars = su.skyscanner.carhire(
    pickup="Madrid",
    pickup_datetime="2026-09-01T10:00",
    dropoff_datetime="2026-09-03T10:00",
)

Every other plugin

The SDK has a method for every plugin (0.10.0 and later) - the plugin’s main input goes in positionally, everything else as keyword options, and options you leave out are not sent, so the API’s defaults apply. Each plugin page shows its method next to cURL.

products = su.target_search("coffee maker", sort="price_low", max_results=48)
milk = su.woolworths_product("888137")                     # one product, with nutrition
trends = su.google_trends(["coffee", "tea"], geo="US")     # a list is sent comma-separated
reviews = su.trustpilot_reviews("www.nike.com", stars=[4, 5])

# Any plugin endpoint by path, parameters named as the API names them
data = su.plugin("/marketplace/target-search", query="tv", max_results=24)

Async

Every method has an async twin on AsyncClient. For high-throughput crawls, cap concurrency to your plan’s limit with a semaphore.

import asyncio
from scrapeunblocker import AsyncClient

CONCURRENCY = 10

async def scrape(su, sem, url):
    async with sem:
        return await su.get_parsed(url)

async def main(urls):
    sem = asyncio.Semaphore(CONCURRENCY)
    async with AsyncClient() as su:
        return await asyncio.gather(*(scrape(su, sem, u) for u in urls))

results = asyncio.run(main([
    "https://www.amazon.com/dp/B08N5WRWNW",
    "https://www.amazon.com/dp/B07FZ8S74R",
]))

Error handling

Non-2xx responses raise typed exceptions, all subclasses of ScrapeUnblockerError. Transient failures (429, 502, 503, 504 and network errors) are retried automatically with exponential backoff.

from scrapeunblocker import (
    Client,
    BlockedError,
    PaymentRequiredError,
    RateLimitError,
    UpstreamOutageError,
)

su = Client()
try:
    html = su.get_page_source("https://example.com")
except BlockedError:
    ...   # 403: the target blocked every bypass path (not billed)
except PaymentRequiredError:
    ...   # 402: quota, credit limit, or a failed payment - fix billing
except RateLimitError:
    ...   # 429: slow down
except UpstreamOutageError:
    ...   # 503: the target site itself is down - retry later

Exception	Status	Meaning
`InvalidRequestError`	400	Bad URL, unsupported scheme, or the API key header was not sent
`AuthenticationError`	401	Key not recognised - typo, stray whitespace, or a rotated key
`NoSubscriptionError`	401	Key is fine, but the account has no active plan
`PaymentRequiredError`	402	Billing block - base class for the four below
`QuotaExceededError`	402	The plan’s requests for this period are used up
`CreditLimitExceededError`	402	Unpaid balance is past the account’s credit limit
`BudgetExceededError`	402	This period’s spend reached the monthly budget limit you set in your profile (from SDK 0.9.0)
`PaymentFailedError`	402	A card payment was declined three times
`BlockedError`	403	Blocked by bot protection on every path
`NotFoundError`	404	A plugin lookup found nothing
`TargetNotFoundError`	404 / 410	The target site says the page does not exist (`X-Origin-Status` set). Billed; carries `origin_status`, `html` and `destination_url`. Subclass of `NotFoundError`, never retried
`BrowserTimeoutError`	408	Our browser run timed out before the page was ready
`UnsupportedContentError`	415	The URL serves something other than HTML (page methods) or other than an image (`get_image`)
`ValidationError`	422	Missing or wrong-typed parameter; `body` holds the `detail` array
`RateLimitError`	429	Too many requests
`UpstreamOutageError`	503	The target origin is down
`ServerError`	5xx	Unexpected server error, including a 504 upstream timeout
`ScrapeTimeoutError`	-	This client gave up locally before the API answered
`ConnectionError`	-	Could not reach the API

The 402 and 404-422 exceptions were added in 0.1.6; before that they arrived as a bare APIError. Nothing was removed, so except APIError handlers written against earlier versions keep working. See all status codes in Errors.

Tune the retry count and timeout on the client:

Client(timeout=180.0, max_retries=2)

See handling failures for what each status code means.

Raw HTTP (without the package)

The API is plain HTTP, so you can call it with requests or httpx directly - no SDK required.

import os
import requests

response = requests.post(
    "https://api.scrapeunblocker.com/getPageSource",
    params={"url": "https://example.com", "parsed_data": True},
    headers={"x-scrapeunblocker-key": os.environ["SCRAPEUNBLOCKER_KEY"]},
)
payload = response.json()
print(payload["data"]["page_type"])

The endpoints are getPageSource, serpApi, and getImage; every parameter shown above maps to a query parameter of the same name.

Get Started

No-code area

Spider Cloud

Plugins

AI Assistants

Guides

Code Examples

Python

Quickstart

Get parsed JSON instead of HTML

Scrape a Google SERP

Force a country

Capture cookies and the proxy used

Fetch an image

Drive the browser with steps

Amazon

eBay search

Skyscanner plugins

Every other plugin

Async

Error handling

Raw HTTP (without the package)

​Quickstart

​Get parsed JSON instead of HTML

​Scrape a Google SERP

​Force a country

​Capture cookies and the proxy used

​Fetch an image

​Drive the browser with steps

​Amazon

​eBay search

​Skyscanner plugins

​Every other plugin

​Async

​Error handling

​Raw HTTP (without the package)

Quickstart

Get parsed JSON instead of HTML

Scrape a Google SERP

Force a country

Capture cookies and the proxy used

Fetch an image

Drive the browser with steps

Amazon

eBay search

Skyscanner plugins

Every other plugin

Async

Error handling

Raw HTTP (without the package)