Web Scraping Tutorial for Beginners (Step by Step)

Web scraping is reading data off a web page with code instead of copying it by hand. This tutorial walks you through your first scraper from an empty file to a saved spreadsheet, using Python with Requests and BeautifulSoup. I ran every command against quotes.toscrape.com, a sandbox built for exactly this, in June 2026, and pasted the real output so you can check your results against mine.

What do you need to start web scraping?

You need Python and two libraries: Requests to download pages and BeautifulSoup to read the HTML. That is the whole toolkit for a static site like the one in this tutorial.

Check that Python is installed by running python --version in a terminal. I am on Python 3.13 here, and anything from 3.8 up works fine. If you want the wider Python picture before you dive in, the web scraping guide covers where this fits.

How do you install the libraries?

Install both libraries with a single pip command:

pip install requests beautifulsoup4

beautifulsoup4 is the package name you install, and bs4 is the name you import in code. You do not need a separate HTML parser because html.parser ships with Python. If pip is not found, try python -m pip install requests beautifulsoup4, which calls pip through Python directly.

How do you fetch a web page in Python?

Use Requests to download the page, then check the status code to confirm it worked. A status of 200 means the server returned the page:

import requests

url = "https://quotes.toscrape.com/page/1/"
response = requests.get(url)
print("status code:", response.status_code)
print("page length:", len(response.text))

The response.text attribute holds the page’s HTML as a string. Printing its length is a quick sanity check that you actually received content and not an empty body. At this point you have the raw HTML in memory, and the next step is finding the data inside it.

How do you inspect a page to find the data?

Open the page in your browser, right-click the data you want, and choose Inspect. This opens the developer tools and highlights the exact HTML tag holding that value.

On quotes.toscrape.com, each quote sits in a block like this:

<div class="quote">
  <span class="text">"The world as we have created it..."</span>
  <small class="author">Albert Einstein</small>
  <a class="tag">change</a>
</div>

That structure is the map for your selectors. The quote text lives in span.text, the author in small.author, the tags in a.tag, and the whole thing is wrapped in div.quote. Reading these three or four lines is the core skill of scraping, and every site rewards you with a similar layout once you look.

How do you select and extract elements?

Pass the HTML to BeautifulSoup, then use select with a CSS selector to pull the elements you mapped. This script fetches page one, counts the quotes, and prints the first one:

import requests
from bs4 import BeautifulSoup

url = "https://quotes.toscrape.com/page/1/"
response = requests.get(url)

soup = BeautifulSoup(response.text, "html.parser")
quotes = soup.select("div.quote")
print("quotes found:", len(quotes))

first = quotes[0]
print("text:", first.select_one("span.text").get_text())
print("author:", first.select_one("small.author").get_text())

When I ran it, BeautifulSoup found all ten quotes on the page and read the first one cleanly:

status code: 200
quotes found: 10
text: “The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”
author: Albert Einstein

Two methods do the work here. select returns a list of every matching element, and select_one returns the first match inside a block. get_text() strips the HTML tags and gives you the plain string. With those three, you can read almost anything off a static page.

How do you scrape multiple pages?

Follow the “Next” link at the bottom of each page until it disappears. quotes.toscrape.com splits its quotes across ten pages, so a scraper that only reads page one misses 90 percent of the data. The pattern is to scrape a page, look for the next link, and repeat:

import requests
from bs4 import BeautifulSoup

base = "https://quotes.toscrape.com"
url = "/page/1/"
all_quotes = []

while url:
    response = requests.get(base + url)
    soup = BeautifulSoup(response.text, "html.parser")
    all_quotes += soup.select("div.quote")

    next_link = soup.select_one("li.next a")
    url = next_link["href"] if next_link else None

print("total quotes:", len(all_quotes))

The li.next a selector targets the Next button’s link. Reading its href attribute gives the path to the following page, and when there is no Next button left, select_one returns None, the loop ends, and you have every quote. Add time.sleep(1) inside the loop on real sites so you space out your requests and stay polite to the server.

How do you save scraped data to a CSV?

Use Python’s built-in csv module with DictWriter, which turns each dictionary into a spreadsheet row. This is the full scraper: it loops all pages, collects text, author, and tags, then writes a file:

import csv
import time
import requests
from bs4 import BeautifulSoup

base = "https://quotes.toscrape.com"
url = "/page/1/"
all_quotes = []

while url:
    response = requests.get(base + url)
    soup = BeautifulSoup(response.text, "html.parser")

    for quote in soup.select("div.quote"):
        all_quotes.append({
            "text": quote.select_one("span.text").get_text(),
            "author": quote.select_one("small.author").get_text(),
            "tags": ", ".join(t.get_text() for t in quote.select("a.tag")),
        })

    next_link = soup.select_one("li.next a")
    url = next_link["href"] if next_link else None
    time.sleep(1)

with open("quotes.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["text", "author", "tags"])
    writer.writeheader()
    writer.writerows(all_quotes)

print("total quotes:", len(all_quotes))
print("saved to quotes.csv")

Running it walked all ten pages and wrote the file:

total quotes: 100
saved to quotes.csv

Open quotes.csv and the first rows look like this, tags joined into one column:

text,author,tags
“The world as we have created it is a process of our thinking...”,Albert Einstein,"change, deep-thoughts, thinking, world"
“It is our choices, Harry, that show what we truly are...”,J.K. Rowling,"abilities, choices"

Two arguments keep the file clean. encoding="utf-8" preserves the curly quotes and any accented characters, and newline="" stops Windows from adding a blank line between every row. That is a complete scraper: it fetches, parses, paginates, and saves.

Where do you go after your first scraper?

Point this script at a site you care about and adjust the selectors to match its HTML. The loop, the pagination, and the CSV writing stay the same, and the only part that changes per site is the select strings, which you find with the inspect step from earlier.

Two walls show up on real targets. Some pages build their content with JavaScript after the initial load, so the data is missing from the HTML that Requests downloads, and you render the page first with a browser tool like Playwright. Other sites block repeated requests from the same address, and at that point a scraper API like ChocoData handles the fetch and returns the page so your parsing code keeps running unchanged. To go deeper on the parsing side, the BeautifulSoup tutorial expands every selector technique used here. You wrote a working scraper today, and the same 25 lines scale to thousands of pages.