Python visual regression testing with Playwright and Pillow

Install the tools

Install Playwright, its browsers and Pillow into your test environment.

pip install playwright pillow pytest
playwright install chromium

Capture and compare

Capture a full-page screenshot with animations disabled, then compare it with the stored baseline. ImageChops.difference returns a black image where nothing changed; getbbox() gives the bounding box of anything that did.

from pathlib import Path
from PIL import Image, ImageChops
from playwright.sync_api import sync_playwright

BASELINE = Path("baselines/home.png")
CURRENT = Path("output/home.png")


def capture(url: str, path: Path) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    with sync_playwright() as p:
        browser = p.chromium.launch()
        page = browser.new_page(viewport={"width": 1280, "height": 800})
        page.goto(url, wait_until="networkidle")
        page.screenshot(path=str(path), full_page=True, animations="disabled")
        browser.close()


def test_homepage_visual():
    capture("https://example.com/", CURRENT)
    if not BASELINE.exists():
        BASELINE.parent.mkdir(parents=True, exist_ok=True)
        CURRENT.replace(BASELINE)
        return  # first run creates the baseline

    base = Image.open(BASELINE).convert("RGB")
    current = Image.open(CURRENT).convert("RGB")
    assert base.size == current.size, f"page size changed: {base.size} -> {current.size}"

    diff = ImageChops.difference(base, current)
    box = diff.getbbox()
    if box:
        diff.save("output/home-diff.png")
    assert box is None, f"visual change detected in region {box}"

Make it practical

Pixel-exact comparison is strict. Real suites add a tolerance (for example, counting changed pixels and failing above a percentage), mask regions that change by design, and keep baselines per viewport and browser.

• Allow a small changed-pixel ratio
• Paint over dynamic regions before diffing
• Store baselines per viewport
• Run in the same container locally and in CI

Scaling beyond a script

Once you need many pages, languages, scheduled runs and a shared review history, a script turns into a product to maintain. That is the point where teams usually adopt a dedicated tool.

Or skip the test code entirely

Code-based visual tests only cover the pages and states someone wrote a test for, and only run when the pipeline runs. VisualRunner captures production pages from a sitemap, crawl or URL list, across languages and viewports, and monitors them on a schedule — so CMS edits, translations and third-party scripts are caught too. Engineers can still trigger a monitor from CI through the API.