How to Scrape Shopify Stores (Python Guide)

Shopify powers a large share of the public storefronts you will ever want product data from, and most of them hand you a clean JSON feed if you ask the right URL. I tested this in June 2026: a single GET to https://www.allbirds.com/products.json returned 200 OK with titles, prices, SKUs, and variants ready to parse. No headless browser, no HTML soup. The catch is that not every store cooperates, and the JSON deliberately leaves out a few fields you might want. This guide covers the public endpoints, what they expose, how stores block you, and when to reach for an API.

Can you legally scrape Shopify stores?

Public product data on a Shopify storefront is generally scrapable, with the usual caveats. You are reading the same pages a browser loads, and product names, prices, and descriptions are facts published openly by the merchant. In hiQ Labs v. LinkedIn (9th Cir., reaffirmed 2022) the court held that scraping publicly accessible data does not violate the US Computer Fraud and Abuse Act, since no authentication is bypassed. The 2022 Van Buren Supreme Court decision narrowed CFAA “exceeds authorized access” to cases where you access areas you were never permitted to reach, which a public storefront is not.

That said, a few lines stay firmly on the right side:

• Read robots.txt and the store’s terms. Shopify’s default robots.txt disallows paths like /cart and /checkout. Stay on product and collection pages.
• Scrape facts (price, title, SKU), not creative assets. Product photos and marketing copy can carry their own copyright.
• Personal data (customer reviews with names, account pages) brings GDPR and CCPA into scope. Avoid it.
• Rate-limit yourself. Hammering a small merchant’s server can look like an attack and get your IP banned.

I am describing how the public endpoints behave, not handing you a license. For commercial or large-scale projects, run the target and your use case past a lawyer. See the web scraping pillar guide for the broader legal background.

What data can you get from a Shopify store?

The public storefront exposes more than most people expect, through two JSON endpoints plus the rendered HTML. Here is what each field gives you.

Two fields you cannot get from the public JSON: exact inventory quantity (only the available boolean) and any customer or order data (that needs the authenticated Admin API). If you need real stock numbers, the store owner has to grant you a token.

How does Shopify block scrapers?

Shopify itself is permissive, but individual merchants bolt on protection that returns a 403 before you ever see JSON. I hit this on the first try. The same /products.json request that worked on allbirds.com returned 403, text/html on gymshark.com, because Gymshark runs Cloudflare bot management in front of its store.

The common blockers, ranked by how often I see them:

The last one is the cheapest to defeat: send a real browser User-Agent and many naive filters wave you through. Cloudflare is the wall. It fingerprints your TLS handshake and HTTP/2 frame order, so a plain requests call gets flagged regardless of headers. That is the point where a residential-proxy scraper API earns its keep.

Method 1: Scrape Shopify with Python (requests + JSON)

The DIY method skips HTML parsing entirely, because Shopify’s default storefront serves a JSON catalog at /products.json. I ran this against allbirds.com in June 2026 and it returned 200 OK. Here is the exact script.

import requests

UA = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                    "AppleWebKit/537.36 (KHTML, like Gecko) "
                    "Chrome/124.0 Safari/537.36"}

def scrape_products(store_url, limit=5):
    url = f"{store_url}/products.json?limit={limit}"
    r = requests.get(url, headers=UA, timeout=20)
    r.raise_for_status()
    for p in r.json()["products"]:
        v = p["variants"][0]
        sku = v.get("sku") or "(no sku)"
        print(f"{p['title'][:40]:42} | ${v['price']:>7} | {sku:12} | avail={v['available']}")

scrape_products("https://www.allbirds.com")

Real output from that run:

status=200  products_returned=5
------------------------------------------------------------
Women's Varsity Jersey - Light Grey/Mid    | $ 130.00 | A12549W050   | avail=True
Women's Varsity Cruiser - Mushroom (Medi   | $ 115.00 | A12592W050   | avail=True
Men's Canvas Cruiser - Nautical Gold (Na   | $  75.00 | A12838M080   | avail=True
Women's Dasher NZ - Auburn (Auburn Sole)   | $ 140.00 | A12382W050   | avail=True
Men's Cruiser Terralux - Anthracite (Dar   | $ 135.00 | A12400M080   | avail=True

To pull a whole catalog instead of five rows, paginate. /products.json caps each page at 250 products via ?limit=250, so loop the page counter until a page returns an empty list:

import requests, time

def scrape_all(store_url):
    products, page = [], 1
    while True:
        url = f"{store_url}/products.json?limit=250&page={page}"
        r = requests.get(url, headers=UA, timeout=20)
        batch = r.json().get("products", [])
        if not batch:
            break
        products.extend(batch)
        page += 1
        time.sleep(1)   # be polite to the server
    return products

I confirmed the pagination behavior: requesting ?page=2 on allbirds returned 200 with an empty products array, which is the signal to stop (the store has fewer than 250 active products). For one specific product, append .json to its URL. I fetched /products/{handle}.json and got the full record back: 200 OK, 13 variants, 5 images.

This works only on the stores that leave /products.json open. On gymshark.com the identical first request returned 403 with a Cloudflare challenge page instead of JSON. No header tweak fixes that, because the block is on the TLS fingerprint, not the headers.

For the broader Python toolkit behind this, see the Python web scraping guide and the requests + BeautifulSoup walkthrough. To crawl thousands of stores on a schedule, Scrapy handles the concurrency and retries.

Method 2: Scrape Shopify with a scraper API

A scraper API solves the two problems Method 1 cannot: Cloudflare-protected stores and JS-only storefronts. Instead of hitting the store directly, you send the target URL to the API, and it routes the request through a residential IP, solves the bot challenge, and (when needed) renders JavaScript before returning the response. The 403 stores become 200 stores.

ChocoData exposes a universal endpoint plus 453 dedicated endpoints. For Shopify there is no site-specific endpoint, so you pass the store’s /products.json URL through the universal endpoint and let the proxy layer handle the block. The same gymshark.com request that gave me a 403 directly goes through the API like this:

import requests

API_KEY = "YOUR_CHOCODATA_KEY"

def scrape_via_api(target_url):
    r = requests.get(
        "https://api.chocodata.com/v1/universal",
        params={
            "api_key": API_KEY,
            "url": target_url,
            "country": "us",      # residential IP in the US
            "render_js": "false", # JSON endpoint needs no rendering
        },
        timeout=60,
    )
    r.raise_for_status()
    return r.json()

data = scrape_via_api("https://www.gymshark.com/products.json?limit=250")
print(len(data["products"]), "products")

Set render_js=true only when a store runs a headless Hydrogen storefront that builds the catalog client-side, since rendering costs more and runs slower. For plain /products.json you leave it off. ChocoData’s pricing as listed on their site:

The free tier covers 1,000 requests, which is enough to test the blocked stores in your list before you commit. I have not load-tested ChocoData against Gymshark myself, so treat the API path as the documented approach rather than a benchmarked result.

When a store is fully JS-rendered and even the API’s HTML response needs interaction (lazy-loaded variants, infinite-scroll collections), drop to a real browser. The Playwright approach in the anti-blocking guide covers driving a headless Chromium through a proxy.

DIY vs API: which should you use?

Pick based on the stores you are targeting and the scale. The split is clean.

For monitoring a handful of stores that leave their JSON open, the DIY script is all you need and it costs nothing. The moment your target list includes a Gymshark-style store, or you are crawling hundreds of stores where some fraction will block you, the API removes the part of the job that breaks at 2 a.m.

The honest limits of scraping Shopify

Three things the public storefront will never give you, no matter the method:

1. Exact inventory counts. You get available: true/false per variant, never the stock number. Real quantities live behind the authenticated Admin API, which only the store owner can grant.
2. Order and customer data. None of it is public. If you see it, you are looking at a leak or a misconfiguration, not a scraping target.
3. Guaranteed stability. A store can disable /products.json, switch to a headless frontend, or turn on Cloudflare any day. Build retry and alerting so a silent 403 does not poison your dataset.

Within those limits, Shopify is one of the friendlier scraping targets on the web: a documented JSON shape, predictable pagination, and a single endpoint that covers most stores. Start with the DIY script, find out which of your targets return 403, and route only those through an API.