Skip to main content
​
Documentation moved to docs.scrapeunblocker.com
This site now lives at docs.scrapeunblocker.com. Every page keeps its path: links to developers.scrapeunblocker.com redirect permanently (301) to the same page here, so old bookmarks, SDK READMEs and search results keep working. The API Reference is part of this site, and the raw OpenAPI 3.1 file is served at /api-reference/openapi.json and, as before on this hostname, at /openapi.json.
​
ChatGPT setup guide for the MCP server
ChatGPT can now use the hosted MCP server as a custom connector - there’s a dedicated, screenshot-by-screenshot ChatGPT setup guide. It connects with your API key in the URL (No Auth) from chatgpt.com (web, not the desktop app), under Settings → Plugins → Developer mode.The MCP server exposes four tools: fetch_html, fetch_parsed, google_search and list_elements.
​
New Meta Ad Library plugin - a competitor's live Facebook & Instagram ads as JSON
Meta Ad Library plugin. A new endpoint, POST /ads/meta-ad-library, returns a Facebook/Instagram advertiser’s ads from Meta’s public Ad Library as structured JSON: ad_text, title, cta_text/cta_type, display_format (image/video/DCO/carousel), creatives (image and video URLs), link_url (the landing page), platforms, and started_running/ended dates.Give it an advertiser - a Page name or a numeric Page ID - and a country; filter with active_status and media_type, and bound the result with max_ads. Unlike Meta’s official Ad Library API - which returns full data only for political and social-issue ads - this reads the same public library anyone can open, so it covers ordinary commercial advertisers. A competitor-ads tool.Available in the Python, Node.js, PHP and Ruby clients.
​
Browser steps and element discovery on getPageSource
Drive a real browser after the page loads. getPageSource takes two new parameters, documented in browser steps and element discovery:
  • steps - a JSON array of actions the API runs in a real browser once the page has loaded, before returning the resulting HTML: wait_for, wait_for_text, wait, click, type (human-like keystrokes), select, press_key and scroll. Fill a search form and read the results, click a “load more” button, scroll an infinite list. Bounded to ~10 steps and a ~30-second total budget; a failed step returns HTTP 422 with step_index, action, reason, selector and the page html at the point of failure. Steps have side effects and are not idempotent - treat the call like a POST.
  • list_elements=true - the discovery half. Returns the page’s interactive elements as JSON ({ url, count, elements }) instead of HTML, each element carrying tag, text, id, name, type, placeholder, aria_label, href and a ready-to-use selector you can drop straight into a steps action. List the elements, build the steps, run them - a clean loop for AI agents. Read-only and idempotent.
Both land in the official Python, Node.js, PHP and Ruby clients: pass a native array/list for steps and a boolean for list_elements.
​
New Amazon plugin - product + search, prices in the right currency
Amazon plugin. Two new endpoints: POST /marketplace/amazon-product returns one product by ASIN or URL (title, brand, price + currency, list price and savings, availability, rating, review count, seller, feature bullets, categories and images), and POST /marketplace/amazon-search returns a keyword search’s result cards (ASIN, title, price, rating, review count, clean product URL, sponsored / Prime flags) on any of 20 regional marketplaces.Prices come back in the right currency automatically: proxy_country defaults to the marketplace’s home country (amazon.com -> US, amazon.de -> DE, …) and the request is routed through our Oxylabs ISP + iProyal ISP pool in that country, so amazon.com returns USD and amazon.de returns EUR with no configuration. No Selling Partner / Product Advertising API account needed.
​
API v3 - Google SERP on su_browser, proxy_country fallback, getPageSource redirects Google search
API v3 - Google SERP overhaul. /serpApi is now served by our own su_browser engine over the ISP proxy pool. Callers use it exactly as before, with two user-facing changes:
  • proxy_country falls back instead of erroring. It is matched against the countries we can serve (BE, CA, CH, CN, DE, DK, ES, FR, GB, HK, HR, IE, IT, JP, KR, LT, NL, NO, PL, RO, SE, SG, TW, US). Ask for a country we do not have an exit for and the request still succeeds from an available exit, with two extra response fields: requestedCountry (the ISO2 you asked for) and note (a human-readable explanation of the fallback).
  • getPageSource now redirects Google search URLs. A google.<tld>/search path, or a google.<tld> homepage with a q= query, returns HTTP 400 pointing you to POST /serpApi?keyword=<term>. Other Google URLs (Maps, etc.) are unaffected.
​
Temu search sort - relevance, sales, recent, price
Temu search takes a sort. POST /goods/temu-search?keyword=...&sort=... now orders results using Temu’s own sort: relevance (default), best_selling, recent, price_asc (price low to high) or price_desc (price high to low). It is server-side, so the whole result set comes back sorted, not just a re-order of the page. The chosen sort is echoed in the response.
​
Temu product returns reviews, video and variant group
Temu product returns more fields - all from the same document, at no extra request. POST /goods/temu-product now also includes:
  • reviews - per-review rating and date. Temu masks the review author and text in the crawler document (they come back as ****); the rating and date are real.
  • video - { contentUrl, thumbnailUrl, uploadDate } when the product ships a video, else null.
  • variantGroupId - the Temu product-group ID that ties variants of the same item together.
  • itemCondition - schema.org condition URL, e.g. .../NewCondition.
All additive - existing fields are unchanged. The image list still carries the 1-2 images Temu puts in the crawler document, not the full gallery.
​
Spider Cloud SDK and su-cloud CLI
Spider Cloud has a Python client and a command line. Everything the dashboard does - deploy a project, run and watch jobs, schedule them, pull the scraped items - now works from a script.
  • su-cloud CLI: deploy, run, jobs, logs, items, download, stats, schedule-add, destination-add and more.
  • SpiderCloudClient library: projects, deploys, jobs, schedules and destinations, plus job.items() to stream data by cursor and job.download() for a full .jsonl.gz export.
  • Authenticates with a per-account API token, created in the dashboard under Spider Cloud → API tokens. Pure standard library, no runtime dependencies. See SDK & CLI.
​
Temu search returns ~40 product URLs, ~2s
Temu search is faster and wider. POST /goods/temu-search now returns about 40 product URLs per keyword in ~2s (up from ~14, and no third-party search in the loop). Search is discovery only
  • fetch each URL’s data through POST /goods/temu-product. The details parameter has been removed.
​
New Temu plugin - product data and keyword search
New plugin: Temu. Two endpoints for Temu, which never serves product data to a browser (the page hydrates through a call that refuses automation). The plugin reads the ld+json document Temu serves to search crawlers instead - the only view that carries the real fields.
  • POST /goods/temu-product?url=... returns a single product as JSON: name, description, brand, price, priceCurrency, availability, rating, reviewCount and images.
  • POST /goods/temu-search?keyword=... returns product URLs taken straight from Temu’s own search page (no third-party search in the loop). With details=true it also resolves the top results to full product objects.
Temu meters the crawler view, so the plugin is built for targeted use (price checks, monitoring, comparison), enforces its own fleet-wide rate limit, and answers 429 when the budget is spent. Price currency follows the exit region. See the Temu plugin page.
​
Claude Code plugin
ScrapeUnblocker installs into Claude Code with two commands.
You get the three MCP tools (fetch_html, fetch_parsed, google_search), a /scrape-url command, and reference skills Claude pulls in while writing scraping code for your project. Your API key is prompted for once when the plugin is enabled and stored in your operating system’s keychain - no environment variable to set, no config file to edit. See Claude Code Plugin.
​
TikTok and Instagram parsed data, Google Local contacts, Skyscanner markets
Five fixes, all from one round of outside testing.parsed_data=true now works on TikTok and Instagram. Both sites build their pages in the browser, so there was almost nothing in the markup to read and the response came back as little more than a page title - even though the HTML we handed you carried every number. Dedicated extractors now read the embedded payload, which also means exact integers instead of the rounded prose the page paints: a TikTok video returns plays, likes, comments, shares and saves; an Instagram post returns exact likes and comments, the real author.username and a real posted_at. See Parsed data.Google Local returns phone and website. The phone used to arrive glued to the end of hours; it is now its own field and hours is clean. website is new. Paid listings are flagged with sponsored and no longer report “Sponsored” as the business name. See Google Local.Skyscanner flights honour market, locale and currency. A request for market=TR&locale=tr-TR&currency=TRY came back priced in GBP with en-GB deep links; it now returns Turkish lira and Turkish place names./serpApi no longer fails a query that has an answer. A search whose results page did not finish rendering was returned as a successful but empty fetch, which surfaced as 502 No SERP results collected on a query that worked on the next call. That page is now discarded and retried. Separately, a search where Google genuinely reports no matches - normal for a narrow site: query - returns 200 with an empty result set instead of a 502.
​
New plugin - eBay Search
POST /marketplace/ebay-search turns any eBay search into structured JSON on 19 regional marketplaces, from ebay.com to ebay.com.au. Each listing comes back with title, numeric price and currency, condition (plus a normalised conditionCode), seller username and feedback, shipping cost, sold / watcher / bid counts, image and a clean item URL:
Filter with condition, listing_type, min_price/max_price, free_shipping, seller or category; sort with sort; page with page/page_size (60, 120 or 240 per page).When eBay finds no exact match it still serves a page of loosely-related suggestions. Those are never passed off as results: the response sets exactMatches: false and explains it in notice.The plugin is live in the dashboard, in all four SDKs on 0.1.8, and documented at eBay Search.
​
SERP: every ad, accurate organic counts, AI Overview
/serpApi now returns every paid unit on the page. Google had started serving part of its ad inventory from a wrapper inside the organic column rather than the two containers we read, so those units were missing: a German insurance query carried six ads and came back with four. Ads are now found wherever they sit and sorted into topAds / bottomAds by their position relative to the organic results.organicResultsCount was double the real number. Two of the paths used to collect organic results reached the same entries at different depths, so every result was returned twice - positions 1-9 followed by an identical 10-18. The list is now deduplicated, which means the count you get for an unchanged query will be about half of what it was, and correct. Paid blocks that Google places inside the organic column are excluded from organic as well.New field: aiOverview. Google’s AI-written summary above the results, with the hostnames it cites, or null when the search has none:
It is read from the page we already fetched, never waited for, so it adds no latency. See SERP scraping.Faster, too. The extraction step used to wait a flat 1.2 s for ads that measurement showed arrive within ~0.4 s, and Google’s “unusual traffic” challenge went undetected whenever it was served at the search URL instead of redirecting - so a burnt exit was retried instead of replaced. Both fixed.wait_after_load works again. The engine now serving SERP was ignoring it entirely. It is also no longer something you should need: extraction waits for the ads on its own, so use wait_after_load only when you deliberately want a longer settle.
​
Spider Cloud documented
Spider Cloud now has its own section: host your own Scrapy project with us, deploy it straight from a git repository, run and schedule spiders, and read the items back - with ScrapeUnblocker routing built in, so a spider that gets blocked elsewhere needs no proxy or fingerprint code here.Eight pages, starting with Getting started, which walks one repository from an empty dashboard to a finished run with data in it. The one to read before your first run is routing modes - the choice between off, marked and all decides what a job costs:
all sends every request through the API, so a spider walking a hundred thousand detail pages makes a hundred thousand paid calls. marked exists so a producer can be unblocked without dragging its consumer onto the paid path.Also covered: projects and settings (including why some projects cannot see environment variables at all), deploys and refs, jobs, schedules and delivery destinations.
​
All four SDKs on 0.1.6 - typed billing errors
The Node.js, Ruby and PHP SDKs now carry the same typed billing errors released for Python earlier today, and all four are on 0.1.6 so their version numbers line up from here on:
Each one gains PaymentRequired* for 402 (with the quota, credit-limit and failed-payment subclasses), the no-subscription flavour of 401, and typed errors for 404, 408, 415 and 422. Nothing was removed, so existing APIError / ApiException handlers keep working. See the Node.js, Ruby and PHP guides.
​
Python SDK 0.1.6 - typed billing errors
scrapeunblocker 0.1.6 for Python raises a typed exception for every documented status code. 402 previously arrived as a bare APIError; it now raises PaymentRequiredError, with a subclass per billing block - QuotaExceededError, CreditLimitExceededError and PaymentFailedError - so each can be handled separately:
Also new: NoSubscriptionError (the 401 that means the key is fine but the account has no plan), plus NotFoundError, BrowserTimeoutError, UnsupportedContentError and ValidationError. Nothing was removed and every new class derives from APIError, so existing handlers keep working. See the Python SDK guide.
​
402 and 401 responses documented
The Errors page now documents HTTP 402, which was missing from the status table entirely. A 402 has three distinct plain-text bodies, each with its own fix:
  • Quota exceeded - the plan’s requests for this billing period are used up
  • Credit limit exceeded - the unpaid balance has grown past the account’s credit limit
  • Payment failed - update payment method - a card payment for an open invoice has been declined three times
All three clear on their own once the underlying billing state changes - access returns within about a minute, with no key change needed. Note that subscribing to a new plan does not clear a failed payment: the old unpaid invoice has to be settled. See 402 payment required.HTTP 401 is now documented properly too, with its two bodies - Unauthorized (key not recognised) and No valid subscription (key is fine, account has no active plan) - plus the neighbouring cases: omitting the x-scrapeunblocker-key header entirely returns 400, not 401, and neither 401 nor 402 counts against your quota. See 401 unauthorized.
​
New oopbuy product-search plugin
A new plugin returns oopbuy.com product-search listings as structured JSON - title (English and Chinese), USD and CNY price, monthly sales count, image and a product-detail link, in about a second:
Pick the source marketplace with channel (1688, taobao, official), order with sort, and page with page / page_size. Note that oopbuy trademark-blocks brand-name keywords (nike, adidas) at its own backend - those come back as a successful 200 with keywordRejected: true and an empty results array (oopbuy’s own genuine response), not an error. Available in the dashboard, the API reference, and all four SDKs. See oopbuy Search plugin.
​
New Google Local (Maps) plugin
A new plugin returns Google Local (Maps) business listings as structured JSON - name, rating, review count, price level, category, address, opening hours and a top review snippet:
It runs on our warm, anti-bot-cleared Google sessions, so you get the local pack without a Google Places API key or billing project. Available in the dashboard, the API reference, and all four SDKs. See Google Local plugin.
​
Skyscanner plugins in the API reference
The three Skyscanner plugin endpoints are now part of the public OpenAPI spec, so they show up in the API Reference and in any OpenAPI-aware client (Postman, Insomnia, code generators):
Nothing changed about how they work - they were already documented and available in all four SDKs. See Skyscanner plugins.
​
Clearer errors for non-HTML URLs and Skyscanner searches
getPageSource no longer reports two cases as 403 Blocked by bot protection when no block was involved:
  • A URL that serves an image or other non-HTML resource now returns 415 naming the content type. For images, use getImage.
  • A Skyscanner flights / hotels / car-hire search URL now returns 400 pointing at the plugin that serves it, with the endpoint in the X-Recommended-Endpoint header. Other Skyscanner pages are unaffected.
Also fixed: getImage returned 404 No image found for every URL. It now returns the PNG.
​
Hosted MCP connector
Added a hosted MCP connector so you can use ScrapeUnblocker in claude.ai (web and mobile) with no local install - just add a custom connector with your key:
See the MCP server guide.
​
MCP server for Claude
Released the official MCP server so Claude (Desktop, Code, and the web app) and any other Model Context Protocol client can fetch any web page through ScrapeUnblocker, using your own API key:
Three tools: fetch_html, fetch_parsed, and google_search. See the MCP server guide.
​
Official SDKs
Released official client libraries for four languages, each wrapping getPageSource (HTML / parsed JSON / cookies), serpApi, getImage, and the Skyscanner plugins - with typed errors and automatic retries:
See the Python, Node.js, PHP and Ruby guides.
​
Plugins
Introduced Plugins - ready-made scrapers for specific sites that return clean, structured JSON. Available in the dashboard (no code) and via the API for volume.First release covers three Skyscanner plugins:
  • Flights - POST /flights/skyscanner-quotes
  • Hotels - POST /hotels/skyscanner-quotes
  • Car Hire - POST /carhire/skyscanner-quotes
All three share your existing API key and plan quota. See the Plugins overview.
​
v1.0.0
Public documentation launched at developers.scrapeunblocker.com.Three endpoints documented:
  • POST /getPageSource
  • POST /serpApi
  • POST /getImage
All other paths remain internal and are not part of the public API contract.