How to Scrape Hotel & Travel Sites: A Python Guide

Travel sites are a top scraping target because every listing carries a price, a rating, a review count, and an address, repeated across millions of properties. They are also among the most defended sites on the web. This guide builds one Python scraper and points it at the three biggest names, TripAdvisor, Airbnb, and Booking.com, then shows the real result for each. I ran the code in June 2026 and pasted the actual output below, including the two sites that blocked me cold.

If you are new to the tooling, start with my Python web scraping guide and the BeautifulSoup tutorial, then come back here.

Can you scrape TripAdvisor, Airbnb, and Booking?

You can fetch one site of the three with a plain script; the other two block a cold request. I sent the same requests GET with a real browser User-Agent to a live page on each site in June 2026. Here is what came back:

So the honest answer splits three ways. Airbnb served the real page. TripAdvisor returned a 403 with a DataDome challenge body (“Please enable JS and disable any ad blocker”). Booking returned a 202 whose body is an empty-titled page with a reportChallengeError script, which is an anti-bot interstitial, not the hotel. The DIY method below works on Airbnb and stops at the door on the other two. I cover the fix for the blocked sites in Method 2.

Is it legal to scrape travel sites?

Scraping public listing and hotel pages is generally lawful in the US, with contract terms as the live risk. Two layers matter, and they are the same for all three sites.

On US computer-access law, reading pages anyone can load without logging in sits on the safer side. In hiQ Labs v. LinkedIn, the Ninth Circuit held that scraping public profiles does not violate the Computer Fraud and Abuse Act, and the Supreme Court’s Van Buren decision narrowed the CFAA to something close to traditional hacking. Reading a public hotel or room page with a bot is not, by itself, a federal computer crime.

Contract law is where the risk lives. Each site’s terms of service prohibit automated access and data harvesting, and using the site means accepting those terms, so a large scrape can draw a breach-of-contract claim. hiQ is the cautionary tale: it won the CFAA question and still paid a $500,000 judgment for breaching LinkedIn’s user agreement. Three rules keep you on the lower-risk path. Stay on public pages and never script a logged-in account or a booking flow. Avoid collecting personal data (reviewer names, profile details), which pulls in the GDPR in Europe. Respect each site’s robots.txt, covered next. I am a tester, not a lawyer, so get advice before any commercial scrape. My is web scraping legal pillar has the full picture.

What does robots.txt allow on these sites?

All three publish a robots.txt that returns HTTP 200, and each blocks the transactional and deep paths while leaving the main listing page crawlable. I fetched each file in June 2026 and read the User-agent: * group. The patterns differ in ways that matter for what you can collect.

The Airbnb line is the one to read twice. The main room page at /rooms/{id} is fair game, which is exactly the page my test parsed. The sub-resources you would most want for a review scraper, /rooms/*/reviews and /rooms/*/photos, are explicitly disallowed. So robots.txt steers you toward the data already embedded in the main page (which is plenty) and away from the dedicated review and photo endpoints. Booking blocks the checkout and search-comparison fragments but leaves the hotel page itself listed. robots.txt is a site’s stated crawling policy, not a law; treating it as a hard boundary is good practice and keeps you on the defensible side of the contract question above.

What data can you extract from a travel listing?

A listing page carries most of the commercially useful fields, and on Airbnb they sit in an embedded JSON-LD block that parses without a browser. Airbnb ships two application/ld+json scripts in the page (VacationRental and Product schema types), and between them they hold the fields below. I confirmed each one is present in the live page I fetched.

The pattern to internalize: the descriptive fields (title, rating, review count, location, photos) are embedded in the page and parse on the first request, and the transactional field everyone wants, the nightly price, is rendered by JavaScript and missing from a static fetch. On TripAdvisor and Booking you cannot reach any of this with a cold request, because the page itself is gated behind the challenge from the first section.

Method 1: scrape Airbnb with requests and BeautifulSoup

A requests GET with a real browser User-Agent plus BeautifulSoup to parse the embedded JSON-LD is the whole scraper, and on Airbnb it works. The trick is to skip the HTML soup and read the structured data the page already publishes for search engines. Here is the exact code I ran against a live Airbnb listing (room ID 16204265):

import requests, json
from bs4 import BeautifulSoup

HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/126.0.0.0 Safari/537.36"
    ),
    "Accept-Language": "en-US,en;q=0.9",
}

url = "https://www.airbnb.com/rooms/16204265"
resp = requests.get(url, headers=HEADERS, timeout=30)
print("STATUS:", resp.status_code)

soup = BeautifulSoup(resp.text, "html.parser")

# Airbnb embeds two JSON-LD blocks; the first is the VacationRental object.
ld = json.loads(soup.find("script", type="application/ld+json").string)

rating = ld.get("aggregateRating", {})
print("name:    ", ld.get("name"))
print("city:    ", ld.get("address", {}).get("addressLocality"))
print("rating:  ", rating.get("ratingValue"))
print("reviews: ", rating.get("ratingCount"))
print("images:  ", len(ld.get("image", [])))

The real output:

STATUS: 200
name:     Quiet calm private bed & bath in Mission District
city:     San Francisco
rating:   4.97
reviews:  724
images:   8

That is the honest result. The request succeeded (200), and the title, city, rating (4.97), review count (724), and image count (8) all parsed from the JSON-LD on the first, cookie-less request. No headless browser, no proxy, no API key. Reading the embedded JSON-LD is more durable than HTML selectors, because Airbnb maintains that block for Google and it changes far less often than the visual markup.

Two honest limits. The nightly price is not in the JSON-LD (Airbnb renders it with JavaScript after load), so this method gets you the listing and its rating but not the live price. And robots.txt disallows /rooms/*/reviews, so the individual review texts are off the table by this route even though the aggregate count is right there. For one listing this is genuinely useful and free. The trouble starts at the other two sites, and at volume.

How do these sites block scrapers?

TripAdvisor and Booking block on the very first request, and Airbnb ramps up its defenses once one IP makes many requests. My single Airbnb fetch worked; a thousand of them from the same address will not. These are the layers, and I saw the first two live:

The two I reproduced are the instructive ones. TripAdvisor’s 403 body was a DataDome script (var dd={'rt':'c',...}), so the page never renders for a plain client. Booking’s 202 carried a reportChallengeError function and an empty <title>, the signature of a challenge that has to run in a browser before the hotel appears. A scraper that trusts a 2xx status would log Booking’s 202 as a “success” and store a blank row. The general playbook for surviving these defenses is in my scraping without getting blocked guide; the rest of this article is the travel-specific fix.

Method 2: scrape travel sites with a scraper API

A scraper API solves the two failures I hit by running a real browser and rotating residential IPs, so the DataDome and JS challenges resolve and the page comes back rendered. You hand it the URL and get HTML or parsed data instead of fighting the challenge yourself. This is the practical route for TripAdvisor and Booking, and for Airbnb the moment you need the JS-rendered price or any volume.

ChocoData is the service I use for this category. It exposes a universal endpoint (one URL for any site) plus 453 dedicated endpoints for common targets. Here is the request shape, swapping the raw requests.get for the API and pointing it at a TripAdvisor hotel page, the one that gave me a flat 403:

import requests

API_KEY = "cd_live_YOUR_KEY"  # free tier: 1,000 requests/month, no card
target = "https://www.tripadvisor.com/Hotel_Review-g60763-d93589-Reviews-The_New_Yorker_A_Wyndham_Hotel-New_York_City_New_York.html"

endpoint = "https://api.chocodata.com/api/v1/universal/get"
params = {
    "api_key": API_KEY,
    "url": target,
    "render": "true",   # run the page in a real browser to clear the challenge
    "country": "us",     # US residential IP
}

resp = requests.get(endpoint, params=params, timeout=90)
print("STATUS:", resp.status_code)
# Rendered HTML comes back in the JSON payload; parse the same JSON-LD or
# selectors as Method 1, now that the anti-bot challenge has been solved.

The difference from Method 1 is what happens behind that one call. ChocoData fetches through a country-matched residential IP that rotates automatically, retries through a fresh IP when a block appears, and runs the page in a real browser so the DataDome challenge (TripAdvisor) or the JS interstitial (Booking) clears before the HTML is returned. The render=true lever is what turns the 403 and 202 I measured into a real page; you then parse the result with the same JSON-LD code from Method 1. Billing applies only to successful (2xx) responses, so the blank-202 problem does not quietly drain your budget.

I did not paste live API output here because that requires a paid key, and I will not fake numbers. The endpoint path, parameters, and free-tier limit above come from the ChocoData docs. The honest claim is narrow: the universal endpoint takes a URL and returns the rendered page, which removes the two specific failures (TripAdvisor 403, Booking 202) I reproduced with plain requests.

TripAdvisor’s official API vs scraping

TripAdvisor publishes an official Content API, and it is worth checking before you scrape because it is free at low volume, with one hard cap that decides the question. Per the Tripadvisor Content API docs, it returns location details plus up to 5 reviews and 5 photos per location, allows up to 50 calls per second, and the FAQ states the first 5,000 calls per month are free on a pay-as-you-go plan.

The 5-review cap is the deciding line. If you need a hotel’s name, rating, address, and a handful of reviews, the official API is the right call: it is free at low volume, structured, and sanctioned. If you need full review history, the complete photo set, or fields across thousands of properties, the API will not give it to you, which is the reason people scrape the pages instead. For Airbnb and Booking there is no comparable open public data API, so the page is the source.

Should you scrape these sites yourself or use an API?

Use Method 1 for Airbnb listing data at small scale, the official API for a few TripAdvisor reviews, and a scraper API for everything else. The decision comes down to which site, which fields, and how many pages.

My honest take from the test: travel scraping is feasible DIY only on Airbnb’s main listing page, and only for the embedded fields. TripAdvisor and Booking both block a cold request, so they need a rendering scraper API (or the official TripAdvisor API for its capped review data) before you parse anything. If you only want a sample of Airbnb listings and their ratings, Method 1 is enough and free. For prices, full reviews, or the blocked sites at volume, the scraper API is the route that actually returns the data.

To go deeper on the building blocks, see the Python web scraping guide for the language fundamentals, BeautifulSoup for parsing the JSON-LD, and Scrapy if you are crawling many listings and want a framework. The web scraping pillar ties the whole workflow together.