How to Scrape YouTube (Python Guide, Tested)

YouTube is the second-most-visited site on the web, and the data devs want is right there on public pages: video titles, view counts, durations, upload dates, channel stats. The catch is that none of it sits in the HTML. YouTube renders everything client-side from a giant JSON object embedded in a <script> tag. So scraping YouTube is less about parsing markup and more about extracting and walking that JSON. I tested the approach below against live pages in June 2026, and every number you see was produced by code I actually ran.

Can you legally scrape YouTube?

You can scrape public YouTube data, with real limits. Two things govern this: YouTube’s Terms of Service and its robots.txt. The Terms of Service prohibit accessing the service through automated means except as permitted by the API, which is a contract term, not a copyright law. US courts (hiQ v. LinkedIn, and the 2022 Ninth Circuit ruling) have repeatedly held that scraping public data is not a Computer Fraud and Abuse Act violation, so the exposure here is breach-of-contract and account termination, not a criminal matter. Scraping public, non-personal facts like view counts for research sits on far safer ground than republishing video content or harvesting personal data.

robots.txt tells you which paths Google itself asks bots to avoid. I fetched it:

import requests
r = requests.get("https://www.youtube.com/robots.txt", timeout=20)
print(r.text)

The disallow list is short and specific:

User-agent: *
Disallow: /api/
Disallow: /comment
Disallow: /get_video
Disallow: /get_video_info
Disallow: /login
Disallow: /results
Disallow: /signup
Disallow: /youtubei/
...

That matters for planning. Channel pages (/@handle/videos) and watch pages (/watch) are not disallowed. Search (/results), the comment system (/comment), and the internal InnerTube API (/youtubei/) are. So a polite scraper sticks to channels and watch pages and treats search and comments as off-limits for the DIY route. For commercial use, read the official policy at youtube.com/t/terms and prefer the API where you can.

For the broader legal picture, see my guide on whether web scraping is legal.

What data can you actually get from a YouTube page?

Quite a lot, and all of it from the embedded JSON rather than the visible HTML. Here is what each public surface exposes and where it lives:

The split is the whole story. Titles, IDs, durations, and view counts are one GET away. Comments and transcripts are not: they load asynchronously through endpoints robots.txt disallows, so they need either reverse-engineered InnerTube calls or a tool that handles them.

Does a plain request to YouTube get blocked?

Yes, on the first try. A bare requests.get with a normal browser User-Agent does not get the page. From an EU IP it gets a 302 to the cookie-consent wall. Here is the exact result I got:

import requests

UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
      "(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36")
r = requests.get("https://www.youtube.com/@TED/videos",
                 headers={"User-Agent": UA}, timeout=30)
print("STATUS:", r.status_code)
print("FINAL URL:", r.url)
print("HAS ytInitialData:", "ytInitialData" in r.text)

STATUS: 200
FINAL URL: https://consent.youtube.com/m?continue=https%3A%2F%2Fwww.youtube.com...
HAS ytInitialData: False

The status is 200, but the final URL is consent.youtube.com and the page has no video data. This is the single most common reason a YouTube scraper “silently” returns nothing. The fix is one cookie: SOCS=CAI signals consent and YouTube serves the real page. With that cookie set, the same request returns 200, lands on the real URL, and ytInitialData is present. That is the version I build on below.

Beyond the consent wall, YouTube also rate-limits by behavior. A handful of requests at human speed work fine. Push hundreds of rapid requests from one IP and you get throttling, CAPTCHAs, or 429s. My guide on scraping without getting blocked covers the proxy and pacing tactics that matter at volume.

Method 1: How do you scrape a YouTube channel with Python?

Set the consent cookie, fetch the /videos page, extract the ytInitialData JSON, then walk it. The reason you cannot use a normal BeautifulSoup find_all here is that the video list is not in the HTML elements at all. It is one big JSON object assigned to var ytInitialData = {...} inside a script tag. So the job is: grab that object with a brace-balanced slice (a regex breaks on the nested braces), parse it, and pull the fields.

One important update for 2026: YouTube moved the channel grid from the old videoRenderer schema to a new lockupViewModel schema. If you follow an older tutorial that searches for videoRenderer, you get zero results on a current channel page. The parser below targets lockupViewModel.

import requests, json

UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
      "(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36")
HEADERS = {"User-Agent": UA, "Accept-Language": "en-US,en;q=0.9"}

def find_json(html, key):
    """Slice the brace-balanced JSON object assigned after `key`."""
    start = html.find("{", html.find(key))
    depth = 0; in_str = False; esc = False
    for j in range(start, len(html)):
        c = html[j]
        if in_str:
            esc = (c == "\\" and not esc)
            if c == '"' and not esc:
                in_str = False
        elif c == '"': in_str = True
        elif c == "{": depth += 1
        elif c == "}":
            depth -= 1
            if depth == 0:
                return json.loads(html[start:j + 1])

def iter_key(obj, key):
    """Recursively yield every value stored under `key`."""
    if isinstance(obj, dict):
        if key in obj:
            yield obj[key]
        for v in obj.values():
            yield from iter_key(v, key)
    elif isinstance(obj, list):
        for v in obj:
            yield from iter_key(v, key)

def scrape_channel(handle):
    s = requests.Session()
    s.cookies.set("SOCS", "CAI", domain=".youtube.com")  # skip EU consent wall
    url = f"https://www.youtube.com/@{handle}/videos"
    html = s.get(url, headers=HEADERS, timeout=30).text
    data = find_json(html, "ytInitialData")

    videos = []
    for lv in iter_key(data, "lockupViewModel"):
        meta = lv.get("metadata", {}).get("lockupMetadataViewModel", {})
        cmv = meta.get("metadata", {}).get("contentMetadataViewModel", {})
        parts = [p["text"]["content"]
                 for row in cmv.get("metadataRows", [])
                 for p in row.get("metadataParts", []) if "text" in p]
        dur = next((b["text"] for b in iter_key(lv.get("contentImage", {}),
                    "thumbnailBadgeViewModel") if "text" in b), "")
        videos.append({
            "video_id": lv.get("contentId"),
            "title": meta.get("title", {}).get("content"),
            "duration": dur,
            "views": next((p for p in parts if "view" in p), ""),
            "published": parts[-1] if parts else "",
        })
    return videos

videos = scrape_channel("TED")
print(f"Parsed {len(videos)} videos\n")
for v in videos[:8]:
    print(f"{v['video_id']}  {v['duration']:>7}  {v['views']:>12}  {v['title'][:45]}")

I ran this against the TED channel. Real output:

Parsed 30 videos

Kxw_e4bDxoQ     9:35     668 views  The Case for Flying Robots as First Responder
6bIORK0izg4     9:56    8.9K views  Why the Best Ideas Come from Play | Maxwell P
0ADHiUWjo6c  5:25:49               Can Play Change the World? | Play@TED Full Ev
lBx86QE4k3s    14:54     22K views  The Love of My Life (and Why I Need to Share
xrV2cTj6E_g    15:10     12K views  We're Keeping the Ocean Wild and You Can Jo
6tC4NLu20Cc    14:13     17K views  How to Google Your Symptoms Without Freaking
OAIlucguNzY    12:45     22K views  How to Give Feedback That Lands | Renee St Ja
DH9L7vJ03DE    18:26     77K views  How Screens Stole Childhood and How to Get

A few honest notes on this output. The view counts are the rounded ones YouTube shows in the grid (8.9K, not the exact integer), and the upload age comes through as relative text (1d ago, 2d ago). For exact view counts you need the watch page (next section). The one row with a blank views field is a live-event item whose metadata layout differs slightly; in production you would handle that branch. And this is one page: 30 videos. Older videos sit behind infinite scroll, which loads through the InnerTube continuation API that robots.txt disallows, so the simple GET tops out at the first 30.

If BeautifulSoup and requests are new to you, start with my BeautifulSoup guide and the broader Python web scraping tutorial.

Method 1b: How do you get exact stats for one video?

Fetch the watch page and read ytInitialPlayerResponse instead of ytInitialData. The watch page carries a different JSON object that holds the precise, unrounded numbers plus category, keywords, and the ISO publish date. Same consent cookie, same brace-balanced extractor, different key:

import requests, json

UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
      "(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36")

def find_json(html, key):  # same helper as above
    start = html.find("{", html.find(key))
    depth = 0; in_str = False; esc = False
    for j in range(start, len(html)):
        c = html[j]
        if in_str:
            esc = (c == "\\" and not esc)
            if c == '"' and not esc: in_str = False
        elif c == '"': in_str = True
        elif c == "{": depth += 1
        elif c == "}":
            depth -= 1
            if depth == 0:
                return json.loads(html[start:j + 1])

s = requests.Session()
s.cookies.set("SOCS", "CAI", domain=".youtube.com")
html = s.get("https://www.youtube.com/watch?v=dQw4w9WgXcQ",
             headers={"User-Agent": UA}, timeout=30).text

pr = find_json(html, "ytInitialPlayerResponse")
vd = pr["videoDetails"]
mf = pr["microformat"]["playerMicroformatRenderer"]

print("title       :", vd["title"])
print("author      :", vd["author"])
print("channel_id  :", vd["channelId"])
print("length_sec  :", vd["lengthSeconds"])
print("views       :", vd["viewCount"])
print("publish_date:", mf["publishDate"])
print("category    :", mf["category"])

Real output:

title       : Rick Astley - Never Gonna Give You Up (Official Video) (4K Remaster)
author      : Rick Astley
channel_id  : UCuAXFkgsw1L7xaCfnd5JJOw
length_sec  : 213
views       : 1782101415
publish_date: 2009-10-24T23:57:33-07:00
category    : Music

That viewCount is the exact figure (1,782,101,415), not the rounded “1.8B” the page displays. So a common pattern is: scrape a channel to collect video IDs, then hit each watch page to get precise stats. Be polite about pacing if you loop over many IDs.

Method 2: How do you scrape YouTube at scale with a scraper API?

Route the request through a scraper API and let it handle the consent cookie, proxy rotation, and rendering. The DIY method above is great for a few channels. It falls down on three things: search (/results is in robots.txt), comments and transcripts (the /youtubei/ endpoint is disallowed and needs continuation tokens), and volume (one IP looping over thousands of watch pages gets throttled). A scraper API solves all three by giving you a clean endpoint and a fresh IP per request.

ChocoData is one option that fits this shape. It offers a universal endpoint plus 453 dedicated endpoints, so you can pass a YouTube URL and get parsed JSON back without maintaining the brace-slicing and schema-walking yourself. The request looks like this (check the dashboard for the current parameter names and your key):

import requests

resp = requests.get(
    "https://api.chocodata.com/v1",
    params={
        "api_key": "YOUR_KEY",
        "url": "https://www.youtube.com/@TED/videos",
        "render": "true",          # execute JS, scroll for more videos
    },
    timeout=120,
)
data = resp.json()
print(resp.status_code, len(data.get("videos", [])))

I did not run this one, it needs a paid key, so I am not pasting fake output. Here is how the two methods compare honestly:

The honest rule: if you are pulling a handful of channels for a side project, the free requests method in Method 1 is enough and you learn how YouTube actually works. The moment you need search, comments, transcripts, or thousands of pages without babysitting IPs and schema changes, the economics flip toward an API.

What are the limits of scraping YouTube?

Four real ones, worth knowing before you build. First, the first 30 videos only on a channel page from a plain GET; deeper history is behind the InnerTube continuation API. Second, rounded view counts in the channel grid; exact figures require a per-video watch-page fetch. Third, no comments or transcripts without reverse-engineering disallowed endpoints. Fourth, schema drift: the videoRenderer-to-lockupViewModel change is proof that YouTube reshapes this JSON without warning, so any DIY parser is something you will revisit.

For projects that outgrow these limits, the official YouTube Data API v3 is the cleanest answer for structured fields within its 10,000-unit daily quota, and a scraper API like ChocoData covers what the official API gates or omits. For everything else, the two scripts above, both run live in June 2026, will get you real YouTube data today.