ScrapeUnblocker Documentation

The official Ruby client wraps the ScrapeUnblocker API in a small, dependency-free gem (standard library only) with typed errors and automatic retries.

gem install scrapeunblocker

Or in a Gemfile: gem "scrapeunblocker". Requires Ruby 2.7+. Store your API key in an environment variable so the client picks it up automatically:

export SCRAPEUNBLOCKER_KEY="YOUR_API_KEY"

Quickstart

require "scrapeunblocker"

su = ScrapeUnblocker::Client.new # reads SCRAPEUNBLOCKER_KEY, or Client.new(api_key: "YOUR_API_KEY")

html = su.get_page_source("https://example.com")

Get parsed JSON instead of HTML

product = su.get_parsed("https://www.amazon.com/dp/B08N5WRWNW")

puts product.page_type # "product"
puts product.source    # how it was extracted
p product.data         # the fields

# If a parse ever comes back wrong, force a fresh set of rules:
fresh = su.get_parsed(url, refresh_rules: true, rules_hint: "price is missing")

See the parsed data guide for response shapes.

Scrape a Google SERP

serp = su.serp("best running shoes", pages_to_check: 2)

serp["organic"].each do |result|
  puts "#{result['position']} #{result['title']} #{result['url']}"
end

Force a country

result = su.get_parsed("https://www.amazon.de/dp/B08N5WRWNW", proxy_country: "de")

Capture cookies and the proxy used

page = su.get_page_with_cookies("https://example.com")
puts page.html
p page.cookies
puts page.proxy

Fetch an image

bytes = su.get_image("https://example.com/photo.jpg")
File.binwrite("photo.jpg", bytes)

Drive the browser with steps

Pass a native array of actions to run in a real browser after the page loads. You get back the HTML of the resulting state.

html = su.get_page_source(
  "https://example.com",
  steps: [
    { action: "type", selector: "input[name=q]", value: "wireless headphones" },
    { action: "press_key", value: "Enter" },
    { action: "wait_for", selector: ".search-results", timeout_ms: 10000 },
  ]
)

Discover the selectors first with list_elements, which returns the page’s interactive elements as parsed JSON:

page = su.list_elements("https://example.com")

page["elements"].each do |el|
  puts [el["tag"], el["selector"], el["text"]].join(" | ")
end

Each element’s selector drops straight into a steps action. Full action reference in the browser steps guide.

Amazon

Product and search data as JSON, priced in the marketplace’s own currency:

product = su.amazon_product(asin: "B0BSHF7WHW", marketplace: "amazon.com")
puts [product["title"], product["price"], product["currency"], product["rating"]].join(" | ")

results = su.amazon_search("wireless headphones", sort: "price_asc")
results["results"].each { |item| puts [item["title"], item["price"], item["currency"], item["asin"]].join(" | ") }

proxy_country defaults to the marketplace’s home country (amazon.com -> US, amazon.de -> DE), so Amazon prices come back in the right currency. Full field list on the Amazon plugin page.

eBay search

Listings from any of the 19 regional eBay marketplaces as JSON:

items = su.ebay_search("iphone 13", marketplace: "ebay.com", condition: "used", sort: "newly_listed")

if items["exactMatches"]
  items["results"].each do |item|
    puts [item["title"], item["price"], item["currency"], item["condition"]].join(" | ")
  end
end

exactMatches is false when eBay found nothing for the keyword and answered with its own loosely-related suggestions instead. Full field list on the eBay Search plugin page.

Skyscanner plugins

locations = su.skyscanner.flight_locations("London")

flights = su.skyscanner.flights(
  origin: "London", dest: "New York",
  depart_date: "2026-09-01", adults: 1, currency: "USD"
)

hotels = su.skyscanner.hotels(destination: "Madrid", checkin: "2026-09-01", checkout: "2026-09-03")
cars = su.skyscanner.carhire(pickup: "Madrid", pickup_datetime: "2026-09-01T10:00", dropoff_datetime: "2026-09-03T10:00")

Error handling

Non-2xx responses raise typed errors, all subclasses of ScrapeUnblocker::Error. Transient failures (429, 502, 503, 504 and network errors) are retried automatically; a 401 or 402 is never retried, because it clears when the key or the billing state changes.

begin
  html = su.get_page_source("https://example.com")
rescue ScrapeUnblocker::BlockedError
  # 403: the target blocked every bypass path (not billed)
rescue ScrapeUnblocker::PaymentRequiredError
  # 402: quota, credit limit, or a failed payment - fix billing
rescue ScrapeUnblocker::RateLimitError
  # 429: slow down
rescue ScrapeUnblocker::UpstreamOutageError
  # 503: the target site itself is down - retry later
end

Error	Status	Meaning
`InvalidRequestError`	400	Bad URL, unsupported scheme, or the API key header was not sent
`AuthenticationError`	401	Key not recognised - typo, stray whitespace, or a rotated key
`NoSubscriptionError`	401	Key is fine, but the account has no active plan
`PaymentRequiredError`	402	Billing block - base class for the three below
`QuotaExceededError`	402	The plan’s requests for this period are used up
`CreditLimitExceededError`	402	Unpaid balance is past the account’s credit limit
`PaymentFailedError`	402	A card payment was declined three times
`BlockedError`	403	Blocked by bot protection on every path
`NotFoundError`	404	Page loaded but held no image (`get_image` only)
`BrowserTimeoutError`	408	Our browser run timed out before the page was ready
`UnsupportedContentError`	415	The URL serves something other than HTML
`ValidationError`	422	Missing or wrong-typed parameter; `body` holds the `detail` array
`RateLimitError`	429	Too many requests
`UpstreamOutageError`	503	The target origin is down
`ServerError`	5xx	Unexpected server error, including a 504 upstream timeout
`TimeoutError`	-	This client gave up locally before the API answered
`ConnectionError`	-	Could not reach the API

The 402 and 404-422 errors were added in 0.1.6; before that they arrived as a bare APIError. Nothing was removed, so rescue ScrapeUnblocker::APIError written against earlier versions keeps working. See all status codes in Errors.

Tune the retry count and timeout on the client:

ScrapeUnblocker::Client.new(timeout: 180, max_retries: 2)

Raw HTTP (without the gem)

The API is plain HTTP, so Ruby’s standard library is enough - no gem required.

require "net/http"
require "json"

uri = URI("https://api.scrapeunblocker.com/getPageSource?url=https://example.com&parsed_data=true")
req = Net::HTTP::Post.new(uri)
req["x-scrapeunblocker-key"] = ENV["SCRAPEUNBLOCKER_KEY"]
res = Net::HTTP.start(uri.host, uri.port, use_ssl: true) { |http| http.request(req) }
payload = JSON.parse(res.body)
puts payload["data"]["page_type"]

The endpoints are getPageSource, serpApi, and getImage; every parameter shown above maps to a query parameter of the same name.

Get Started

No-code area

Spider Cloud

Plugins

AI Assistants

Guides

Code Examples

Ruby

Quickstart

Get parsed JSON instead of HTML

Scrape a Google SERP

Force a country

Capture cookies and the proxy used

Fetch an image

Drive the browser with steps

Amazon

eBay search

Skyscanner plugins

Error handling

Raw HTTP (without the gem)

​Quickstart

​Get parsed JSON instead of HTML

​Scrape a Google SERP

​Force a country

​Capture cookies and the proxy used

​Fetch an image

​Drive the browser with steps

​Amazon

​eBay search

​Skyscanner plugins

​Error handling

​Raw HTTP (without the gem)

Quickstart

Get parsed JSON instead of HTML

Scrape a Google SERP

Force a country

Capture cookies and the proxy used

Fetch an image

Drive the browser with steps

Amazon

eBay search

Skyscanner plugins

Error handling

Raw HTTP (without the gem)