How to Scrape Yelp (Python Guide)
- I fetched a live Yelp business and search page on June 12, 2026 with requests: both returned HTTP 403 from DataDome, not the listing HTML.
- Yelp's
robots.txtbans automated access except for named search bots, so a DIYrequestsscraper is blocked at the door. - The useful data (name, rating, review count, address, phone, price) ships as JSON-LD in the page, so once you get past the wall, parsing is easy.
- Lead with a scraper API (ChocoData example) that rotates IPs and solves the DataDome challenge, or run a headed browser with residential proxies.
Yelp holds review and local-business data that powers lead lists, competitor research, and sentiment analysis, so “how do I scrape it” is a fair question. I built a Python scraper and pointed it at live Yelp on June 12, 2026. The honest headline first: a plain requests call does not get the page. Yelp answers with an HTTP 403 from DataDome, an anti-bot wall, before you see a single listing. This guide shows the real block, the parser that works once you are past it, and the two routes that actually return data. For fundamentals, see my web scraping guide and the Python walkthrough.
Can you scrape Yelp, and is it legal?
You can extract public Yelp data, but a naive script is blocked, and the legality depends on method and what you keep. The technical wall comes first, covered in the next section. The legal picture has two layers worth separating.
On US computer-access law, reading pages anyone can load without logging in sits on the safer side. In hiQ Labs v. LinkedIn, the Ninth Circuit held that scraping public data does not violate the Computer Fraud and Abuse Act, and the Supreme Court’s Van Buren decision narrowed the CFAA toward traditional hacking. Querying a public Yelp page with a bot is not, by itself, a federal computer crime.
Contract law and privacy law are the live risks. Yelp’s Terms of Service prohibit automated access, and Yelp’s robots.txt states that “use of any robot, spider, service search/retrieval application, or other automated device… to access, retrieve, copy, scrape, or index any portion of the service or any content is prohibited, except as expressly permitted by Yelp.” hiQ is the cautionary tale: it won the CFAA question and still paid a $500,000 judgment for breaching LinkedIn’s user agreement. Review text and reviewer names are also personal data under the GDPR. Three rules keep you on the lower-risk path: stay on public business data, never script a logged-in account, and treat reviewer profiles as out of scope. I am a tester, not a lawyer, so get advice before any commercial scrape. My is web scraping legal pillar has the full picture.
What data can you extract from Yelp?
A Yelp business page carries most of the commercially useful fields as structured JSON-LD, plus review and search data that load differently. Knowing where each field lives tells you what to target.
| Field | Where it lives | Easy to parse? |
|---|---|---|
| Business name | JSON-LD (name) | Yes |
| Average rating | JSON-LD (aggregateRating.ratingValue) | Yes |
| Review count | JSON-LD (aggregateRating.reviewCount) | Yes |
| Full address | JSON-LD (address) | Yes |
| Phone | JSON-LD (telephone) | Yes |
| Price range | JSON-LD (priceRange) | Yes |
| Categories | Embedded page JSON | Inconsistent |
| Hours | Embedded page JSON | Inconsistent |
| Review text | Paginated reviews endpoint | Separate call |
| Search result list | /search page (also gated) | Separate call |
The pattern: the headline business fields ship as a clean JSON-LD <script> block, so once you have the HTML, you parse a dict, not brittle CSS selectors. The deeper data (full reviews, the search result list) lives on separate, equally gated requests.
How does Yelp block scrapers?
Yelp fronts its pages with DataDome, and I confirmed the block on live URLs. Here is exactly what I ran and got back on June 12, 2026:
import requests
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/126.0.0.0 Safari/537.36",
"Accept-Language": "en-US,en;q=0.9",
}
url = "https://www.yelp.com/biz/molinari-delicatessen-san-francisco"
r = requests.get(url, headers=headers, timeout=30)
print("STATUS", r.status_code)
print("SERVER", r.headers.get("server"))
print("SNIPPET", r.text[:160])The real response, not a guess:
STATUS 403
SERVER DataDome
SNIPPET <html lang="en"><head><title>yelp.com</title>...<p id="cmsg">Please enable JS and disable any ad blocker</p><script data-cfasync="false">var dd={'rt':'c','cid':'AHrlqAAAA...The /search results URL returned the same 403. Four things define the wall:
- DataDome edge block. The
Serverheader readsDataDomeand the response sets adatadomecookie. The body is a JavaScript challenge (var dd={...}plus acaptcha-deliveryreference), not the listing HTML. The 403 fires before Yelp’s app even runs. - TLS and header fingerprinting. DataDome inspects the TLS handshake (JA3/JA4) and header order, so a Python
requestsclient is flagged even with a browser User-Agent. I swapped the UA and still got 403. - robots.txt allowlist. Yelp’s robots.txt grants crawl access only to named bots (Googlebot, Bingbot, social preview bots) and blocks everyone else. A generic client is outside that list.
- JS-rendered challenge. Clearing the block needs a real JavaScript engine to execute the DataDome script and return a valid token.
requestscannot do that.
Because the live fetch returned 403, I did not get HTML to parse, so this guide carries no “tested” stamp and no fake listing output. The parser below is real, working code, but it runs against output you get past the wall (a scraper API’s HTML or a saved page), not against a raw requests call to Yelp.
Method 1: the scraper API route (recommended)
A scraper API is the route that returns Yelp data today, because it solves the DataDome challenge, rotates IPs, and renders JavaScript, then hands you HTML. You make one HTTP call; the vendor owns the wall.
ChocoData (chocodata.com) is one example I checked against its own docs. It exposes a universal endpoint (“one endpoint, any site”) plus 453 dedicated endpoints. The documented request shape, fetching a Yelp business page through the universal endpoint:
import requests
from bs4 import BeautifulSoup
import json
API_KEY = "YOUR_API_KEY" # free tier: 1,000 requests/month, no card
resp = requests.get(
"https://chocodata.com/api/v1/universal",
params={
"url": "https://www.yelp.com/biz/molinari-delicatessen-san-francisco",
"render_js": "true", # execute the DataDome challenge
"api_key": API_KEY,
},
timeout=60,
)
resp.raise_for_status()
html = resp.text # rendered HTML, past the 403
# Yelp ships business data as JSON-LD, so parse the dict, not selectors.
soup = BeautifulSoup(html, "html.parser")
biz = {}
for tag in soup.find_all("script", type="application/ld+json"):
try:
data = json.loads(tag.string or "")
except json.JSONDecodeError:
continue
if isinstance(data, dict) and "aggregateRating" in data:
rating = data["aggregateRating"]
addr = data.get("address", {})
biz = {
"name": data.get("name"),
"rating": rating.get("ratingValue"),
"reviews": rating.get("reviewCount"),
"phone": data.get("telephone"),
"price": data.get("priceRange"),
"address": f'{addr.get("streetAddress")}, '
f'{addr.get("addressLocality")}, '
f'{addr.get("addressRegion")}',
}
break
print(json.dumps(biz, indent=2))The endpoint pattern is GET /api/v1/{site}/{resource}?api_key=... for dedicated targets, plus the universal endpoint above for any URL. Per ChocoData’s site, the service handles residential proxy rotation, CAPTCHA and DataDome solving, retries, and JavaScript rendering, and returns the rendered page. I have not independently benchmarked their Yelp success rate, so treat the parameter names (render_js, the endpoint path) as illustrative and verify against their docs on the free tier before you build on them.
I verified the JSON-LD parsing block above against a Yelp-shaped JSON-LD sample locally, so the parsing logic is correct working code. What I did not do is run it end to end against live Yelp, because the raw fetch is walled. That is the honest line: the parser is real, the live data depends on getting past DataDome, which is what the API does.
ChocoData’s published pricing: a free tier of 1,000 requests/month with no card, paid plans from $19/month (27,000 requests), and pay-as-you-go top-ups at $0.90 per 1,000 successful requests.
Method 2: DIY with a headed browser and proxies
If you want to scrape Yelp yourself, you need a real browser engine plus residential proxies, because DataDome blocks plain HTTP clients. Playwright drives Chromium, executes the challenge script, and renders the page; residential IPs keep you off the datacenter blocklists DataDome leans on.
# pip install playwright && playwright install chromium
from playwright.sync_api import sync_playwright
import json, re
URL = "https://www.yelp.com/biz/molinari-delicatessen-san-francisco"
def scrape_yelp(url):
with sync_playwright() as p:
browser = p.chromium.launch(
headless=False, # headless is detected faster; headed clears more challenges
proxy={"server": "http://USER:PASS@residential-endpoint:PORT"},
)
page = browser.new_page()
page.goto(url, wait_until="networkidle", timeout=60000)
html = page.content()
browser.close()
# Same JSON-LD parse as Method 1.
blocks = re.findall(
r'<script type="application/ld\+json">(.*?)</script>', html, re.S
)
for raw in blocks:
try:
data = json.loads(raw)
except json.JSONDecodeError:
continue
if isinstance(data, dict) and "aggregateRating" in data:
return data
return None
print(scrape_yelp(URL))I did not run this end to end here, because it needs a paid residential proxy endpoint and a headed display, and faking the output would break the one rule that matters. Three honest caveats from how DataDome behaves:
- Headless gets flagged faster. DataDome scores headless Chromium harder than a headed window, so
headless=False(or a stealth plugin) clears more challenges, at the cost of needing a display or virtual framebuffer. - Datacenter IPs fail. A cloud server’s IP is on DataDome’s blocklist within a few requests. Residential or mobile proxies are effectively required, and they are the real cost line.
- One browser per page is slow. Playwright spends seconds per page launching and rendering. For more than a few hundred pages, the API route is cheaper than the proxy plus compute bill.
For the browser-automation fundamentals this builds on, see scraping without getting blocked. For the HTML-parsing side, BeautifulSoup covers the JSON-LD extraction pattern used above, and if you outgrow a single script, Scrapy handles queueing and retries.
Which method should you choose?
Pick based on volume, your tolerance for maintaining a proxy stack, and risk. Here is my decision rule.
| Factor | Yelp Fusion API | DIY (Playwright + proxies) | Scraper API |
|---|---|---|---|
| Gets past DataDome | Not applicable (official) | You build it | Included |
| Returns data today | Yes (capped) | Yes (with residential IPs) | Yes (vendor-handled) |
| Full review text | No (3 excerpts) | Yes | Yes |
| Proxies / CAPTCHA | Not needed | You own it | Included |
| Volume ceiling | ~300 calls/day free | Your proxy budget | Per request |
| ToS risk | Lowest (sanctioned) | Higher (unofficial) | Medium (still scraping) |
- Need a few businesses, official, with light review data: use the Yelp Fusion API. It is the sanctioned path, returns clean JSON, and caps you at roughly 300 calls/day on the free tier with only 3 review excerpts per business.
- High volume or full review text, and you do not want to run infrastructure: use a scraper API. It removes the DataDome, proxy, and CAPTCHA line items for one per-request price.
- You specifically want to own the stack and have residential proxies already: drive Playwright headed. Accept the per-page latency and the proxy bill.
The deciding fact from my run: a plain requests call to Yelp returns 403 from DataDome, not data. Closing the gap to real listings means either staying inside Yelp’s official API limits or letting something (an API or your own browser-plus-proxy rig) execute the JavaScript challenge and rotate IPs. For most readers past a handful of lookups, that is work a scraper API already did.
FAQ
Yelp Fusion is the official API. The free tier allows 300 calls per day (5,000 historically, now lower) and returns business search, details, and up to 3 review excerpts as JSON. It does not return full review text or unlimited volume, so heavy review pulls still need scraping or a paid data agreement.
Yelp sits behind DataDome. A plain request gets a 403 with a JS challenge body ('Please enable JS and disable any ad blocker') and a datadome cookie, not the page. DataDome fingerprints the TLS handshake and headers, so swapping the User-Agent alone does not clear it.
Reading public pages is on the safer side of US computer-access law after hiQ v. LinkedIn, but Yelp's Terms of Service prohibit scraping, and review text plus reviewer names are personal data under the GDPR. Stay on public business data, never script a logged-in account, and get legal advice before any commercial scrape.