Skip to main content

Performance

Short feedback cycles help you ship faster. Waiting for CI to finish is neither fun nor productive. We work hard to keep Happo as fast as we possibly can without sacrificing robustness.

Conventional screenshot testing setups are typically slow to run. Happo tries to do better by running things in parallel, caching previous runs and optimizing how screenshot images are stored. The time it takes to run Happo will mostly be decided by a few factors:

  • The complexity of your test suite.
  • What browsers you are targeting.
  • The dimensions of your components.

Keep complexity down by reducing the number of images and other assets in your tests, avoid testing the same elements multiple times, and keep components isolated.

Browsers perform differently. The fastest ones we have are Chrome and Firefox, capable of producing more than 10 screenshots per second. Other browsers are slightly slower and will max out at 5-10 screenshots per second. Choose your targets wisely as the performance of your test suite is mostly going to be decided by your slowest target. The (global) average time it takes to produce a screenshot on a Happo worker is 200 ms.

Examples in the wild​

Mineral UI​

Mineral UI is a UI toolkit. They have ~350 snapshots in their test suite. Happo runs take about 4 mins in CI. Out of that, ~3 mins is spent npm installing and building files, then the actual Happo run takes just over a minute. They use a "generated examples" approach where they use an already-existing style guide to generate happo examples. The .happo.js config and a call to the happo-ci-travis script provided by the happo.io library is what's required to get things working in CI.

Airbnb's Lunar project​

Airbnb's Lunar project has a slightly more complex setup. They have ~600 snapshots. Happo tests are split up in two projects — one with a light theme and one with a dark theme. Happo CI runs take 10 mins in total (5 mins installing and building, 5 mins taking screenshots). They use the happo-plugin-storybook plugin where happo examples are taken straight from a Storybook setup. The config required to get happo running in CI involves a .happo.js file and a call to the happo-ci-travis script provided by the happo.io library.

Execution time​

To calculate the execution time for a test suite, you can use this rough formula:

npm install time +
precompilation (optional) +
prerendering (optional) +
(number of examples × 200ms) +
between 2 and 20 seconds network time/time in queue
= total happo time

The 200 milliseconds per screenshot is a global average for all Happo.io projects. For smaller components, this number is a lot smaller, so don't worry if your test suite consists of a lot of examples.