Visual Regression Testing: When and How

Screenshot diffs catch layout bugs that assertions never name: spacing collapses, theme tokens drifting, a button with the right role and wrong padding. They can also become a Monday ritual of "approve all" after a font tweak.

Playwright and Chromatic show up below. I add screenshot coverage when layout bugs are the product risk and the team will actually review baselines in CI.

When visual diffs pay off

I add screenshot coverage when:

1. Layout is the product risk: design-system primitives, marketing hero, dense data tables where CSS bugs ship past role assertions.
2. Tokens/themes multiply states: light/dark, compact density, RTL if you ship it.
3. You have a stable viewport + font contract in CI (same OS image, locked fonts, disabled animations).

I refuse them when:

• The surface is copy-heavy and churns weekly (blog CMS chrome).
• Assertions on accessible name + outcome already catch the bug cheaper.
• Nobody owns baseline review: unowned goldens rot into noise.

ts
// e2e/visual/button.styles.spec.ts, illustrative
import { test, expect } from '@playwright/test';

test('primary button states', async ({ page }) => {
	await page.goto('/_test/design-system/button');
	await expect(page.getByTestId('button-gallery')).toHaveScreenshot(
		'button-gallery.png',
		{ animations: 'disabled' },
	);
});

Prefer a dedicated gallery route or Storybook over screenshotting random app pages that load live data. Data variance is not a visual bug.

How: keep the surface small

Start with primitives and one complex organism (e.g. order summary card), not the entire authenticated shell. Expand only when a real CSS bug escaped.

ts
// playwright.config.ts, illustrative visual project
{
  name: 'visual',
  testMatch: /visual\/.*\.spec\.ts/,
  use: {
    viewport: { width: 1280, height: 720 },
    deviceScaleFactor: 1,
    colorScheme: 'light',
  },
}

Pin viewport and color scheme. Run dark mode as an explicit second shot when tokens matter, not as accidental OS theme drift.

Review workflow (the real product)

Visual suites fail in process more than in pixels:

1. PR shows the diff image, not only a red X.
2. One owner (or rotating reviewer) accepts intentional changes.
3. Baselines commit beside the change that caused them, not in a weekly "sync goldens" dump that hides regressions.
4. Animations off, fonts licensed/locked in the CI image, and flaky regions masked deliberately (ads, avatars, timestamps).

ts
// Mask volatile regions instead of disabling the whole assertion
await expect(page.locator('main')).toHaveScreenshot('checkout.png', {
	animations: 'disabled',
	mask: [page.getByTestId('live-clock'), page.getByTestId('avatar')],
});

If reviewers cannot see diffs easily, they will approve blindly. That is worse than no visual suite.

Pair with non-visual assertions

Visual diffs catch paint. They do not replace:

• Role/name checks for the same gallery (did we remove the button entirely?)
• Theme token unit tests where pure CSS values matter
• Axe or equivalent on the gallery route for contrast regressions screenshots may or may not make obvious depending on baseline freshness

ts
// illustrative, same gallery, two lenses
test('button gallery a11y + paint', async ({ page }) => {
	await page.goto('/_test/design-system/button');
	await expect(page.getByRole('button', { name: 'Save' })).toBeVisible();
	await expect(page.getByTestId('button-gallery')).toHaveScreenshot(
		'button-gallery.png',
		{ animations: 'disabled' },
	);
});

If the screenshot is the only assertion on a control, a deleted button that leaves empty space can still look "close enough" after a careless baseline update.

Decision criteria before adding a snapshot

1. Would a role/name or outcome assertion catch this cheaper?
2. Is the surface stable for a month, or will copy/layout churn weekly?
3. Who reviews the diff image on the PR, named owner or "whoever merges"?
4. Can CI reproduce the raster (OS image, fonts, animations off)?

If (3) or (4) is fuzzy, fix process before adding goldens. A visual suite without review ownership is how flake becomes culture.

Failure modes

Screenshot everything. CI time explodes; baselines churn; people quit opening the report. Scope is a feature.

Platform drift. Developer laptops produce different AA font rasterization than CI. Decide: baselines only from CI, or use a container locally that matches CI. Do not argue about subpixel ghosts in Slack for a week.

Asserting copy in pixels. Use DOM assertions for text; use screenshots for layout/paint. Mixing them doubles maintenance.

Vendor mythology. "AI will triage diffs" does not remove ownership. Someone still decides intentional vs bug.

Baseline drive-bys. Updating twenty goldens in a dependency bump PR without paging through diffs is how regressions land with a green check.

Continuity

The 2025 testing thread closes here with visual regression, after trophy allocation, layer criteria, Vitest, Playwright page objects, and auth fakes. CI pipeline design for monorepos is the natural sequel: wiring the layers this year configured into repeatable delivery.

Takeaway

Screenshot diffs pay off on stable design-system surfaces with owned baselines - not as a blanket full-page net that trains the team to approve blindness.