How to Scrape Food Delivery Sites (Python Guide)

Food sites split into two groups that behave nothing alike when you point a scraper at them. Recipe content sites (AllRecipes, BBC Good Food, Serious Eats) publish their data as structured JSON for Google, so a plain Python script reads ingredients and ratings on the first request. Delivery apps (DoorDash, Uber Eats) hide the menu behind a JavaScript front end and a Cloudflare anti-bot wall. I built one requests scraper and ran it against both kinds in June 2026, and I pasted the real output below, including the page that blocked me. If the tooling is new to you, start with my Python web scraping guide and the BeautifulSoup tutorial, then come back.

Can you scrape food delivery and recipe sites?

You can scrape recipe content sites with a plain script today, and delivery-app menus need more machinery. I sent the same requests GET with a real browser User-Agent to a live page on each in June 2026. Here is what came back:

So the answer divides cleanly. Recipe pages served the real HTML, JSON-LD and all. The DoorDash store page returned a 403 with a Cloudflare block body, not a menu, before the app ran. The DIY method below works on recipe sites and stops at the door on delivery menus. I cover the fix for the blocked case in Method 2.

Is it legal to scrape recipes and restaurant menus?

Scraping public recipe and menu pages is generally lawful in the US, with contract and copyright as the live risks. Two layers matter here, and the copyright one is specific to food.

On US computer-access law, reading pages anyone can load without logging in sits on the safer side. In hiQ Labs v. LinkedIn, the Ninth Circuit held that scraping public data does not violate the Computer Fraud and Abuse Act, and the Supreme Court’s Van Buren decision narrowed the CFAA toward traditional hacking. Querying a public recipe or restaurant page with a bot is not, by itself, a federal computer crime.

Copyright is the layer food scrapers underestimate. US copyright law (and the long-standing reading in cases like Publications International v. Meredith) treats a bare list of ingredients as a statement of facts, which is not protected. The written method, the headnote, the styling, and the photos are creative expression, which is protected. So extracting the ingredient list and structured fields (rating, time, calories) is far safer than copying the step-by-step prose or images and republishing them.

Contract law is the third risk. Site terms of service prohibit automated access, and hiQ is the cautionary tale: it won the CFAA question and still paid a $500,000 judgment for breaching LinkedIn’s user agreement. Three rules keep you on the lower-risk path: collect facts (ingredients, ratings, prices) rather than copyrighted prose or photos, never script a logged-in account or a checkout flow, and respect each site’s robots.txt (next section). I am a tester, not a lawyer, so get advice before any commercial scrape. My is web scraping legal pillar has the full picture.

What does robots.txt allow on these sites?

Each site publishes a robots.txt that returns HTTP 200, and the recipe sites leave their recipe pages crawlable for generic clients while blocking named AI bots. I fetched each file in June 2026 and read the rules. The patterns differ in a way that matters.

The AllRecipes line is the one to read twice. Under User-agent: *, the recipe page itself is fair game, which is exactly the page my test parsed. The same file lists ClaudeBot, GPTBot, anthropic-ai, CCBot, PerplexityBot, and Google-Extended in their own groups with Disallow: /, which targets AI training and answer-engine crawlers by name. A generic requests client falls under User-agent: * and is allowed on recipe URLs, so identify your scraper honestly and stay within that group. robots.txt is a site’s stated crawling policy, not a law; treating it as a hard boundary keeps you on the defensible side of the contract question above.

What data can you extract from a food site?

A recipe page carries almost every useful field inside a single schema.org/Recipe JSON-LD block, and a delivery store page carries its menu in an internal API you cannot reach with a cold request. The split decides your method. These are the fields and where each one lives:

The pattern to internalize: a recipe site does the structuring work for you, because it wants Google to show a rich result, so the data ships as clean JSON in the page. A delivery app renders its menu client-side from a private API and guards both the page and the API, so the price you want is the hardest field to reach.

Method 1: scrape recipes with requests and BeautifulSoup

A requests GET with a real browser User-Agent plus BeautifulSoup to pull the Recipe JSON-LD is the whole scraper, and on recipe sites it works. The trick is to skip the visible HTML and read the structured block the page publishes for search engines. One detail makes the parser robust: many sites wrap the recipe inside a @graph array or nest it under other schema types, so the function below walks the JSON recursively and returns the first object whose @type is Recipe. Here is the exact code I ran:

import requests, json
from bs4 import BeautifulSoup

HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/126.0.0.0 Safari/537.36"
    ),
    "Accept-Language": "en-US,en;q=0.9",
}

def find_recipe(html):
    """Walk every JSON-LD block and return the first @type=Recipe object."""
    soup = BeautifulSoup(html, "html.parser")
    def walk(node):
        if isinstance(node, dict):
            t = node.get("@type", "")
            types = t if isinstance(t, list) else [t]
            if "Recipe" in types:
                return node
            for v in node.values():
                hit = walk(v)
                if hit:
                    return hit
        elif isinstance(node, list):
            for v in node:
                hit = walk(v)
                if hit:
                    return hit
        return None
    for block in soup.find_all("script", type="application/ld+json"):
        try:
            data = json.loads(block.string or "")
        except json.JSONDecodeError:
            continue
        hit = walk(data)
        if hit:
            return hit
    return None

def scrape(url):
    r = requests.get(url, headers=HEADERS, timeout=30)
    print("STATUS:", r.status_code, "| bytes:", len(r.content))
    if r.status_code != 200:
        print("  blocked, no recipe parsed")
        return
    rec = find_recipe(r.text)
    if not rec:
        print("  no Recipe JSON-LD on page")
        return
    agg = rec.get("aggregateRating") or {}
    nutr = rec.get("nutrition") or {}
    print("name:       ", rec.get("name"))
    print("rating:     ", agg.get("ratingValue"), "from", agg.get("ratingCount"), "ratings")
    print("totalTime:  ", rec.get("totalTime") or rec.get("cookTime"))
    print("yield:      ", rec.get("recipeYield"))
    print("calories:   ", nutr.get("calories"))
    print("ingredients:", len(rec.get("recipeIngredient", [])))
    for ing in rec.get("recipeIngredient", [])[:3]:
        print("   -", ing)

scrape("https://www.allrecipes.com/recipe/20144/banana-banana-bread/")
print()
scrape("https://www.bbcgoodfood.com/recipes/easy-chocolate-cake")

The real output from that run, on June 12, 2026:

STATUS: 200 | bytes: 525372
name:        Banana Banana Bread
rating:      4.7 from 17341 ratings
totalTime:   PT75M
yield:       ['12', '1 (9x5-inch) loaf']
calories:    231 kcal
ingredients: 7
   - 2 cups all-purpose flour
   - 1 teaspoon baking soda
   - 0.25 teaspoon salt

STATUS: 200 | bytes: 567177
name:        Easy chocolate cake
rating:      None from None ratings
totalTime:   PT55M
yield:       14
calories:    523 calories
ingredients: 16
   - 200g golden caster sugar
   - 200g unsalted butter softened plus extra for the tins
   - 4 large eggs

That is the honest result. Both requests succeeded (200), and the recipe fields parsed from the JSON-LD on the first, cookie-less request: no headless browser, no proxy, no API key. AllRecipes returned a ratingValue of 4.7 across 17,341 ratings; BBC Good Food returned 16 ingredients and 523 calories. Reading the embedded JSON-LD beats HTML selectors because the site maintains that block for Google, so it changes far less often than the visual markup.

Three honest limits. First, the totalTime is ISO 8601 (PT75M means 75 minutes), so convert it before display. Second, the field set varies by site: BBC Good Food’s “Easy chocolate cake” returned None for rating because that page’s JSON-LD omits aggregateRating, so always guard for missing keys (the code above does, with or {}). Third, the recipeInstructions text and the photos are the copyrighted parts; pulling them to republish is the copyright risk from the legality section, so I keep this scraper on the factual fields. For a recipe dataset, this method is genuinely enough and free.

How do delivery apps block scrapers?

Delivery apps block the store page on the first request, and even past that the menu loads from a private API rather than the HTML. My recipe fetches returned 200; the DoorDash store fetch did not. These are the layers, and I hit the first one live:

Here is the exact result the same scraper returned on the DoorDash store page, on June 12, 2026:

STATUS: 403 | bytes: 5892
  blocked, no recipe parsed

The 403 with a ~5.9 KB body is a Cloudflare block, not the restaurant. The menu you would want never renders for a plain client, and even in a real browser the item prices arrive through an internal GraphQL call that the front end makes after load, not in the initial HTML. A scraper that trusts a 2xx status would still need to find and replay that private API, which changes without notice. The general playbook for surviving these defenses is in my scraping without getting blocked guide; the rest of this article is the food-specific fix.

Method 2: scrape delivery menus with a scraper API

A scraper API solves the failure I hit by running a real browser and rotating residential IPs, so the Cloudflare check clears and the menu page comes back rendered. You hand it the URL and get HTML or parsed data instead of fighting the wall yourself. This is the practical route for DoorDash and Uber Eats, and for recipe sites the moment you need volume or a JS-rendered field.

ChocoData is the service I checked against its own docs for this category. It exposes a universal endpoint (one URL for any site) plus 453 dedicated endpoints for common targets. Here is the request shape, swapping the raw requests.get for the API and pointing it at the DoorDash store page that gave me the flat 403:

import requests

API_KEY = "YOUR_API_KEY"  # free tier: 1,000 requests/month, no card

target = "https://www.doordash.com/store/chipotle-mexican-grill-san-francisco-3145/"

resp = requests.get(
    "https://chocodata.com/api/v1/universal",
    params={
        "url": target,
        "render_js": "true",   # run the page in a real browser to clear Cloudflare
        "country": "us",        # US residential IP
        "api_key": API_KEY,
    },
    timeout=90,
)
print("STATUS:", resp.status_code)
html = resp.text  # rendered HTML, past the 403
# Parse the rendered menu here (selectors or the internal JSON the page exposes).

The difference from Method 1 is what happens behind that one call. Per ChocoData’s site, the service fetches through a country-matched residential IP that rotates automatically, retries through a fresh IP when a block appears, and runs the page in a real browser so the Cloudflare challenge clears before the HTML is returned. Billing applies only to successful responses, so the blank-403 problem does not quietly drain your budget. The endpoint pattern is GET /api/v1/{site}/{resource}?api_key=... for dedicated targets plus the universal endpoint above for any URL; authentication is the api_key query parameter.

I did not paste live API output here, because that requires a paid key and I will not fake numbers. The endpoint path, parameters, and free-tier limit above come from the ChocoData docs, so treat the exact parameter names as illustrative and verify them on the free tier before you build on them. The honest claim is narrow: the universal endpoint takes a URL and returns the rendered page, which removes the specific failure (DoorDash 403) I reproduced with plain requests. ChocoData’s published pricing: a free tier of 1,000 requests/month with no card, paid plans from $19/month (27,000 requests) and $49/month (82,000 requests), and pay-as-you-go at $0.90 per 1,000 successful requests.

Should you build it yourself or use an API?

Use Method 1 for recipe data at small scale, and a scraper API for delivery menus, live prices, or any volume. The decision comes down to which target, which fields, and how many pages.

My honest take from the test: food scraping is feasible DIY only on recipe content sites, and best kept to the factual fields (ingredients, rating, time, nutrition). Delivery apps block a cold request, so DoorDash and Uber Eats need a rendering scraper API before you parse anything, and their live prices sit behind an internal API on top of that. If you want a recipe dataset, Method 1 is enough and free. For delivery menus, prices, or scale, the scraper API is the route that actually returns the data.

To go deeper on the building blocks, see the Python web scraping guide for the language fundamentals, BeautifulSoup for parsing the JSON-LD, and Scrapy if you are crawling many pages and want queueing and retries. The web scraping pillar ties the whole workflow together.