Visual AI testing vs DOM assertions

Your checkout test suite is fully green. Every assertion passes: the total shows $249.00, the Pay button exists, the confirmation text is correct. Meanwhile, a customer emails a screenshot in which the Pay button sits underneath the cookie banner and cannot be clicked.

Nothing was wrong with your assertions. They checked the right things. They just could not see the page.

This is the split between the two ways of checking a user interface, and the practical question is not which one is better but which bugs each one is blind to.

Short version

• A DOM assertion checks the page's structure and text: "an element with this label contains $249.00".
• Visual AI testing compares screenshots and uses a model to judge whether a difference matters.
• DOM assertions catch wrong data, wrong state, and wrong behaviour. They cannot see layout.
• Visual testing catches overlap, cut-off text, broken styles, and missing images. It cannot check meaning.
• Both produce false positives, but for opposite reasons: selectors change, and pixels change.
• Use DOM assertions everywhere, visual checks on a small set of high-value pages.

---

What each one actually does

DOM assertions. The DOM is the browser's internal object model of the page — the structure behind what you see. A DOM assertion finds an element and checks something about it, usually text, an attribute, or whether it exists. Playwright, Cypress, and Selenium all work this way.

await expect(page.getByTestId('order-total')).toHaveText('$249.00');
await expect(page.getByRole('button', { name: 'Pay now' })).toBeEnabled();

The test knows the total is correct. It has no idea whether that total is visible, readable, or on the screen at all.

Visual testing. The tool takes a screenshot and compares it against an approved baseline image. Classic tools compare pixel by pixel. Visual AI tools add a model that tries to judge whether a difference is a real change or noise — an anti-aliased edge, a shifted shadow, a one-pixel font rendering difference.

The test knows the page looks different from last week. It has no idea whether $249.00 is the right number.

---

Which catches which bug

This is the table worth keeping.

Read the "Misses" column for DOM assertions and you see a single theme: anything about position, size, colour, or overlap. Read it for visual checks and you see another: anything about meaning, behaviour, or state over time.

---

The false-positive tax

Both approaches cost you time in failures that are not bugs. The costs have different shapes, which matters when you decide how much of each to run.

DOM assertions break on structure. A developer renames a class, wraps an element in a new div, or changes a label from "Pay now" to "Pay securely". Your test fails and nothing is broken.

This cost is controllable. Use stable, purpose-built selectors:

Fragile: page.locator('.btn-primary.mt-4 > span')

Fragile: page.locator('text=Pay now')

Stable: page.getByTestId('checkout-submit')

Stable: page.getByRole('button', { name: /pay/i })

A data-testid attribute added for testing survives redesigns. Role-based selectors survive most text changes and double as an accessibility check.

Visual checks break on pixels. A font renders half a pixel differently on a different machine. A date in the footer changes. An animation is captured mid-frame. A user avatar loads from a random source. None of these are bugs, and all of them turn your run red.

This cost is harder to control, and it is the reason most visual testing programmes die. The failure mode is that someone starts approving diffs without looking, and the moment that becomes routine, the whole suite stops catching anything.

What actually keeps it manageable:

1. Freeze everything dynamic. Fixed dates, fixed seeded data, the same user avatar every run.
2. Disable animations in the test environment with a CSS override.
3. Wait for a stable state before the screenshot — fonts loaded, images loaded, no pending network requests.
4. Mask known-variable regions, such as a "last updated 3 minutes ago" label.
5. Run in one fixed environment, ideally a container, so rendering is identical every time.
6. Keep the set small. Ten important screens reviewed properly beats 300 approved blindly.

That last point is where visual AI genuinely helps compared with plain pixel diffing. A model that classifies a two-pixel shadow shift as "no meaningful change" and a button overlapping a banner as "layout broken" reduces the review load a lot. It does not eliminate it, and it will occasionally judge wrongly in both directions.

---

Where each one belongs

A workable split for a typical web application.

Use DOM assertions for:

• Every functional flow: sign-up, login, checkout, settings, permissions
• All data correctness: totals, dates, sort order, counts, currency
• All state: loading, empty, error, disabled, success
• Anything involving a sequence of actions
• Accessibility structure: roles, labels, headings, tab order

Use visual checks for:

• The five to fifteen screens your business depends on, at two or three viewport widths
• Shared components in a component library, where one change affects everything
• Email templates, where you cannot rely on the DOM
• Marketing and pricing pages, where appearance is the product
• After any change to global CSS, design tokens, or a UI dependency

Use both together for: the checkout page, the dashboard, and the sign-up form. Assert the numbers with the DOM, assert the appearance with a screenshot. This is the combination that would have caught the cookie banner bug at the start of this article.

---

A practical starting setup

If you have DOM tests and no visual checks, here is a two-week path.

1. Pick five screens: home, sign-up, the main dashboard, checkout, and one settings page.
2. Make their data deterministic. Fixed seed, fixed dates, fixed avatars, no live third-party content.
3. Add a CSS override in the test environment that disables animations and transitions.
4. Capture baselines at 375px, 768px, and 1440px. Review every baseline by hand before approving it — a wrong baseline poisons everything after it.
5. Run the visual suite on pull requests that touch CSS or shared components, and nightly on everything else.
6. Track two numbers for a month: real bugs caught, and diffs reviewed. If the ratio is worse than one real bug per fifty reviews, cut screens rather than accept the noise.

Step 6 is the one teams skip, and it is the one that tells you whether the programme is working.

---

What neither approach does

Worth being clear about the shared blind spots.

Neither one knows what the page should look like. Both compare against something you approved. If the design was wrong when the baseline was taken, both will happily confirm it forever.

Neither judges usability. A form can pass every assertion and every pixel comparison while being genuinely confusing to use. That is a job for a person.

And neither catches what a real user hits on their own machine — the ad blocker that removes an element, the 4G connection that reorders loading, the browser extension that injects CSS. Those still arrive as bug reports from people, which is why the reports themselves need to carry the environment: browser, viewport, console output, and network activity at the moment things broke.

---

Frequently asked questions

Do I need visual testing if I have a design system? Yes, and it is often more valuable there. A component library means one CSS change touches every screen at once. Visual checks on the components themselves catch that blast radius in one run.

Is visual AI better than plain pixel comparison? For review workload, usually yes: it filters out rendering noise that would otherwise fail your run. For correctness, it introduces its own judgement calls, so you still review the failures. Start with strict pixel comparison on a stable environment, and add the model layer when noise, not bugs, is your main cost.

How many visual snapshots is too many? When nobody reads the diffs, you have too many. In practice that threshold is around 30 to 50 comparisons per run for most teams. Beyond that, approvals become reflexive.

Can visual testing replace my end-to-end tests? No. It cannot click through a flow, check that data saved, or verify that a permission was denied. It only checks how one moment looks.

What about testing dark mode and responsive layouts? This is exactly where visual checks earn their cost. Capture each key screen in both themes and at three widths. DOM assertions almost never catch a theme or breakpoint bug, because the structure is identical and only the appearance is wrong.