Skip to main content

Spurious diffs

An important factor when constructing a good screenshot testing setup is to keep the number of spurious diffs to a minimum. A spurious diff (or a false positive) is when Happo finds a difference that isn't caused by a change in the code. These involve (but are not limited to):

  • image loading
  • font loading
  • asynchronous behavior (e.g. components fetching data)
  • animations
  • random data, counters, etc
  • dates, timestamps, etc
  • stack traces

Happo tries to take care of as many of these as possible, automatically. For instance, the following tasks are performed before taking the screenshot:

  • wait for images (including background images, srcset)
  • wait for custom fonts
  • wait for asynchronous data fetching (XHR, window.fetch)
  • disable CSS animations/transitions
  • stop SVG animations

Tips & tricks​

In some cases however, Happo can't automatically detect things that cause spuriousness. Here are some tips & tricks that you might find useful when dealing with spurious diffs.

Dates and timestamps​

If you have dates/timestamps, either injecting a fixed new Date('2019-05-23T08:28:02.446Z') into your component or freezing time via something like mockdate or Sinon.js can help.

External data​

If a component depends on external data (via some API), consider splitting out the data-fetching from the component and test the component without data fetching, injecting the data needed to render it.

Animations​

If you have animations controlled from JavaScript, find a way to disable them for the Happo test suite.

Stack traces​

If you have components that throw errors, consider catching them and rendering a normalized error message. This is especially useful if you have components that render stack traces, which are likely to change between test runs.

One way to normalize these stack traces in a React component is to use an error boundary. Here's an example:

import { ErrorBoundary } from 'react-error-boundary';

// https://github.com/bvaughn/react-error-boundary?tab=readme-ov-file#errorboundary-with-fallbackrender-prop
const fallbackRender = ({ error }) => {
  // We need to sanitize ports, asset hashes, and line/col numbers from
  // the stack trace to make the Happo diffs stabilized.
  error.stack = error.stack.replace(
    /http:\/\/localhost:\d{4}.*?:\d+:\d+/g,
    'http://localhost:1234/path-to-file.1234abcd.bundle.js:1234:56',
  );

  throw error;
};

export const MiscFailing = () => (
  <ErrorBoundary fallbackRender={fallbackRender}>
    <ComponentThatThrows />
  </ErrorBoundary>
);

Hiding content with data-happo-hide​

If individual elements are known to cause spuriousness, consider adding the data-happo-hide attribute. This will render the element invisible in the screenshot. E.g. <div data-happo-hide>{Math.random()}</div>.