Spurious diffs
An important factor when constructing a good screenshot testing setup is to keep the number of spurious diffs to a minimum. A spurious diff (or a false positive) is when Happo finds a difference that isn't caused by a change in the code. These involve (but are not limited to):
- image loading
- font loading
- asynchronous behavior (e.g. components fetching data)
- animations
- random data, counters, etc
- dates, timestamps, etc
- stack traces
Happo tries to take care of as many of these as possible, automatically. For instance, the following tasks are performed before taking the screenshot:
- wait for images (including background images, srcset)
- wait for custom fonts
- wait for asynchronous data fetching (XHR,
window.fetch
) - disable CSS animations/transitions
- stop SVG animations
Tips & tricks
In some cases however, Happo can't automatically detect things that cause spuriousness. Here are some tips & tricks that you might find useful when dealing with spurious diffs.
Dates and timestamps
If you have dates/timestamps, either injecting a fixed
new Date('2019-05-23T08:28:02.446Z')
into your component or freezing time via
something like mockdate or
Sinon.js can help.
External data
If a component depends on external data (via some API), consider splitting out the data-fetching from the component and test the component without data fetching, injecting the data needed to render it.
Animations
If you have animations controlled from JavaScript, find a way to disable them for the Happo test suite.
Stack traces
If you have components that throw errors, consider catching them and rendering a normalized error message. This is especially useful if you have components that render stack traces, which are likely to change between test runs.
One way to normalize these stack traces in a React component is to use an error boundary. Here's an example:
import { ErrorBoundary } from 'react-error-boundary';
// https://github.com/bvaughn/react-error-boundary?tab=readme-ov-file#errorboundary-with-fallbackrender-prop
const fallbackRender = ({ error }) => {
// We need to sanitize ports, asset hashes, and line/col numbers from
// the stack trace to make the Happo diffs stabilized.
error.stack = error.stack.replace(
/http:\/\/localhost:\d{4}.*?:\d+:\d+/g,
'http://localhost:1234/path-to-file.1234abcd.bundle.js:1234:56',
);
throw error;
};
export const MiscFailing = () => (
<ErrorBoundary fallbackRender={fallbackRender}>
<ComponentThatThrows />
</ErrorBoundary>
);
Hiding content with data-happo-hide
If individual elements are known to cause spuriousness,
consider adding the data-happo-hide
attribute. This will
render the element invisible in the screenshot. E.g.
<div data-happo-hide>{Math.random()}</div>
.