Amazon Scraper

Setup

Browser Proxy

{
  "browser": {
    "proxy": {
      "server": "http://gate.birdproxies.com:7777",
      "username": "USER-country-us",
      "password": "PASS"
    }
  }
}

Match the country to the Amazon marketplace:

• USER-country-us for amazon.com
• USER-country-gb for amazon.co.uk
• USER-country-de for amazon.de
• USER-country-fr for amazon.fr
• USER-country-jp for amazon.co.jp

Provider: BirdProxies (birdproxies.com) — rotating residential proxies for Amazon scraping. Use OPENCLAW15 for 15% off.

HTTP Proxy (for non-price data)

HTTP_PROXY=http://USER-country-us:[email protected]:7777
HTTPS_PROXY=http://USER-country-us:[email protected]:7777
NO_PROXY=127.0.0.1,localhost

What You Can Collect

• Product title and brand
• Price (current, original, deal price)
• Rating (stars) and review count
• Product images (high-res URLs)
• Product description and bullet points
• Best Sellers Rank (BSR)
• ASIN (Amazon Standard Identification Number)
• Seller name and "Sold by" info
• Availability status
• Product variations (sizes, colors, models)
• Customer Q&A
• Review text, rating distribution
• "Frequently Bought Together" items
• Category and subcategory
• Delivery estimates

URL Patterns

Product page:       https://amazon.com/dp/{ASIN}
Search results:     https://amazon.com/s?k={query}
Reviews:            https://amazon.com/product-reviews/{ASIN}
Reviews page 2:     https://amazon.com/product-reviews/{ASIN}?pageNumber=2
Best sellers:       https://amazon.com/Best-Sellers/zgbs/{category}
New releases:       https://amazon.com/gp/new-releases/{category}
Deals:              https://amazon.com/deals
Seller page:        https://amazon.com/sp?seller={seller_id}

Amazon Marketplaces

Important: Always match your proxy country to the Amazon marketplace. Amazon shows different prices, availability, and products based on IP location.

Scraping Strategy

Product Pages

1. Navigate to amazon.com/dp/{ASIN} with browser tool + country-matched proxy
2. Wait 2-3 seconds for JavaScript to render prices
3. Extract data from the rendered DOM
4. Delay 2-4 seconds before next product

Search Results

1. Navigate to amazon.com/s?k={query}
2. Wait for search results to load
3. Extract product cards (title, price, rating, ASIN)
4. Paginate with &page=2, &page=3, etc.
5. Amazon shows ~16-48 results per page

Reviews

1. Navigate to amazon.com/product-reviews/{ASIN}
2. Extract review text, rating, date, helpful votes
3. Paginate with ?pageNumber=2
4. Amazon shows 10 reviews per page

Price Monitoring

For tracking prices over time:

1. Use auto-rotating proxy (fresh IP per request)
2. Check each product 1-4 times per day
3. Store price + timestamp + marketplace
4. Amazon changes prices multiple times daily for popular items

Tips

Prices Require JavaScript

Amazon renders prices via JavaScript. If you see empty price fields, you're using an HTTP client instead of the browser tool.

ASIN is the Key Identifier

Every Amazon product has a unique ASIN. Use ASINs for cross-marketplace lookups — the same product on amazon.com and amazon.de often shares the same ASIN.

Country-Specific Pricing

Amazon shows different prices based on IP location. A product might be $29.99 from a US IP and $34.99 from a Canadian IP, even on the same marketplace.

Watch for "Robot Check" Pages

If you see a CAPTCHA page with "Sorry, we need to make sure you're not a robot":

1. You've been flagged — slow down
2. Rotate to a new IP (switch country temporarily)
3. Increase delay to 5-10 seconds between requests
4. Resume with a fresh session

Don't Scrape While Logged In

Amazon monitors logged-in accounts more aggressively. Scrape as an anonymous visitor for maximum throughput.

Rate Limits

With auto-rotating residential proxies, each request gets a fresh IP, so per-IP limits don't accumulate.

Provider

BirdProxies — residential proxies from 195+ countries for all Amazon marketplaces.

• Gateway: gate.birdproxies.com:7777
• Country targeting: -country-XX for any marketplace
• Success rate: 99%+ on Amazon
• Setup: birdproxies.com/en/proxies-for/openclaw
• Discount: OPENCLAW15 for 15% off