Screen Scraping Explained (and How It Differs From Web Scraping)

I started in web scraping, but the first scrapers I ever read about did something stranger: they captured the green-on-black screen of a mainframe terminal and fed it to newer software. That is screen scraping, and it is older than the web. This piece defines it, walks through where it came from, and lays out how it differs from web scraping and from a clean API. There is one tiny command you can run to feel the idea.

What is screen scraping?

Screen scraping is extracting data from a program’s display output, the same output a human would read on screen. A program reads the rendered result of another program (the text on a terminal, the controls in a desktop window, or the pixels in a screenshot) and pulls structured values out of it. The source was built for a person to look at, and the scraper reads it anyway.

The defining trait is the layer it reads from. A normal integration reads a program’s data, through a file, a database, or an API. A screen scraper reads a program’s presentation, the finished surface meant for eyes. That choice is what makes it both powerful and fragile: it can reach data that has no other exit, and it breaks the moment the layout shifts.

Here is the idea in one command. Treat fixed-column terminal output as the “screen” and pull two fields out of it by position:

printf 'NAME   STATUS\nweb01  UP\nweb02  DOWN\n' | awk 'NR>1 {print $1, $2}'

That captured the rendered rows and extracted the host and status columns:

web01 UP
web02 DOWN

No API and no data file were involved. The script read text laid out for a human and recovered the values by their place on the line. That positional, presentation-layer reading is the heart of screen scraping.

Where did screen scraping come from?

Screen scraping started in the 1980s and 1990s as a way to bridge mainframe terminals to newer software. Banks, airlines, and insurers ran applications on IBM mainframes that users reached through 3270 terminals: fixed grids, often 24 rows by 80 columns, of monochrome text. These systems had no API. When a company wanted a desktop app or, later, a website in front of that mainframe, developers wrote programs that drove the terminal session, read the character grid, and typed input back, all as if a person sat there.

That practice picked up the name “screen scraping.” The same approach spread to other green-screen systems: AS/400 (5250 terminals), VT100 sessions, and early point-of-sale software. The pattern held everywhere: a working system with valuable data, no machine interface, and a screen as the only way in.

When the web arrived, the technique moved with it. Reading an HTML page to extract data is the direct descendant of reading a terminal screen, which is why early HTML extraction was also called screen scraping. The vocabulary later split, and “web scraping” became the term for the page-based version.

How does screen scraping differ from web scraping?

Screen scraping reads a rendered display; web scraping reads a page’s HTML source. They share a goal, automated extraction of data built for humans, and they split on the layer they read and the targets they handle. Web scraping is best understood as the web-native branch of screen scraping, covered in depth in the web scraping guide.

The practical differences:

The line blurs in one case. A modern web scraper that drives a real browser and reads pixels, or runs OCR on a rendered page, is doing screen scraping on a web target. The label follows what you read, the painted surface or the markup behind it, more than the kind of program you point it at.

How does screen scraping differ from an API?

An API hands you structured data directly; screen scraping recovers data from output meant for display. With an API, the provider publishes an endpoint that returns clean JSON or XML with named fields and a documented contract. With screen scraping, no such channel exists, so you read the surface a human would and reconstruct the fields yourself.

That gap drives every tradeoff between the two:

The summary is short. When an API exists, use it: it is the supported, stable path. Screen scraping is what you reach for when no API exists and the data lives only on a screen, which is exactly the situation that created the technique and keeps it alive.

Where is screen scraping still used in 2026?

Screen scraping survives wherever data lives behind a display and no API has replaced it. Two settings dominate today.

Legacy systems. Mainframes and old terminal applications are still in daily production at banks, insurers, airlines, and government agencies. Many have no modern interface, so screen scraping remains the practical bridge between a 1985 green-screen and a 2026 web portal. The host runs untouched, and a scraper reads its terminal output and relays it onward. Replacing these systems is expensive and risky, so the bridge stays in place for years.

Robotic process automation (RPA). RPA bots automate routine office work by operating the same desktop applications a person would: a billing tool, an ERP screen, a legacy client with no API. The bot reads field values off the window, types into forms, and clicks buttons. That on-screen reading step is screen scraping, and it is the core capability that lets an RPA bot work against software that exposes no other interface.

A third, smaller case is accessibility and testing, where tools read the rendered UI to verify what a user would actually see. Across all of these, the trigger is the same: a useful interface with no clean data channel underneath.

What tools do people use for screen scraping?

The tools split by what kind of screen you read. Each category targets a different surface, from a character grid to raw pixels.

For the terminal world, HLLAPI (High Level Language API) is worth a name. It is a long-standing standard that lets a program read and write a 3270 or 5250 session field by field, the structured way to scrape a mainframe screen rather than guessing at coordinates.

When the screen you care about is a web page, browser automation is where screen scraping and web scraping overlap. Driving Playwright to render a page and read it is screen-scraping mechanics applied to web targets, and for HTML-only pages a plain HTTP client with cURL or a parser is lighter. Once you scrape web pages at volume and start hitting blocks, a scraper API such as ChocoData handles the fetch and the anti-bot layer for you, which is the point where reading raw pixels in a browser stops being worth the cost.