Skip to main content
Version: Current

Continuous Integration

Adding Happo to your Continuous Integration setup is the best way to catch visual regressions early. Happo will compare your PRs with the base branch and let you know exactly what has changed in your UI.

Happo status on a GitHub pull request Example of a Happo status posted to a GitHub pull request.

Setup​

The instructions on this page apply to all integrations except the Cypress, Playwright and Vitest integrations. Refer to the Cypress integration page, the Playwright integration page and the Vitest integration page for instructions on how to integrate with CI there.

To simplify using Happo in a pull-request/merge-request model, Happo provides a unified CLI command that automatically detects CI environments. The main happo CLI command will:

  1. Figure out the right baseline report to compare with, starting at the merge base for the pull request branch
  2. Create a Happo report for the current HEAD commit
  3. Compare the baseline report with the new report
  4. If allowed to, post back a status to the commit/PR

The CLI auto-detects the following CI environments:

  • GitHub Actions
  • Circle CI
  • Travis CI
  • Azure DevOps
  • GitLab CI
  • Bitbucket Pipelines

If your CI environment is not listed here or if you run into any issues, please reach out to support@happo.io and we will help you get things set up.

GitHub Actions​

This script knows about the GitHub Actions build environment, assuming a PR based model. To run it, configure your workflow file to run the happo command. The workflow can authenticate with Happo in one of two ways:

Either way, make sure that the workflow is configured to run on pushes to your default branch. This will ensure that baselines exist for PR builds to compare against.

Authenticating with an OIDC token​

Available since happo v6.25.0.

GitHub Actions can give a job an OpenID Connect (OIDC) token that proves which repository it runs in. When neither apiKey and apiSecret nor HAPPO_API_KEY and HAPPO_API_SECRET are set, the happo CLI asks GitHub for that token and signs in to the Happo account the repository is connected to.

This needs two things:

  1. The repository connected to your Happo account through the Happo GitHub app (Step 1 and Step 2 below).
  2. The id-token: write permission on the job, so that it may request the token. Once a workflow sets permissions, every permission it leaves out is turned off, so add contents: read too for actions/checkout.
.github/workflows/happo.yml
name: Happo CI

on:
  push:
    branches: [main]
  pull_request:

jobs:
  happo:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.pull_request.head.sha || github.ref }}
          fetch-depth: 100
      - name: Fetch main branch
        if: github.ref != 'refs/heads/main'
        run: git fetch origin main:main
      - uses: actions/setup-node@v4
      - run: npm ci
      - run: npx happo

Nothing changes in the Happo GitHub app's settings: GitHub issues the token to the workflow, not to the app.

A few things to know:

  • GitHub doesn't give a token to a workflow run by a pull request from a fork, so such runs need another way to authenticate.
  • The token works only for github.com repositories, and only for a repository connected to exactly one Happo account. If several accounts are connected to it, or you use GitHub Enterprise Server, authenticate with an API key and secret.
  • If HAPPO_API_KEY and HAPPO_API_SECRET are set, the CLI uses them and never asks for a token. Remove them from the workflow when you switch.

Authenticating with an API key and secret​

Find your API key and secret on your account's API tokens page and add them as repository secrets named HAPPO_API_KEY and HAPPO_API_SECRET. Then pass them to the happo command:

.github/workflows/happo.yml
name: Happo CI

on:
  push:
    branches: [main]
  pull_request:

jobs:
  happo:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.pull_request.head.sha || github.ref }}
          fetch-depth: 100
      - name: Fetch main branch
        if: github.ref != 'refs/heads/main'
        run: git fetch origin main:main
      - uses: actions/setup-node@v4
      - run: npm ci
      - run: npx happo
        env:
          HAPPO_API_KEY: ${{ secrets.HAPPO_API_KEY }}
          HAPPO_API_SECRET: ${{ secrets.HAPPO_API_SECRET }}

Circle CI​

The happo script knows about the CircleCI build environment, assuming a PR based model. To run it, configure .circleci/config.yml to run the happo command. Something like this:

.circleci/config.yml
jobs:
  build:
    docker:
      - image: cimg/node:lts
    steps:
      - checkout:
          method: full

      - run:
          name: happo
          command: npx happo

The happo command assumes your PRs are based off of the main branch. If you're using a different default branch, you can set the --baseBranch argument.

.circleci/config.yml
jobs:
  build:
    docker:
      - image: cimg/node:lts

    steps:
      - checkout:
          method: full

      - run:
          name: happo
          command: npx happo --baseBranch origin/main

Azure DevOps​

The happo script knows how to resolve variables from Azure Pipelines. It can be used with pull requests and regular pushes. To run it, configure azure-pipelines.yml to run the happo command.

In the example below the HAPPO_API_KEY and HAPPO_API_SECRET environment variables are populated from two user-defined secret variables.

azure-pipelines.yml
trigger:
  - main
pool:
  vmImage: ubuntu-latest
steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '22.x'
    displayName: 'Install Node.js'
  - script: |
      npm ci
      npx happo
    displayName: 'Install dependencies and run Happo'
    env:
      HAPPO_API_KEY: $(happoApiKey)
      HAPPO_API_SECRET: $(happoApiSecret)

The trigger is set to run for pushes to the main branch. You'll have to replace this if you are using a different default branch. To trigger builds for pull requests, you can use a branch policy for the main branch.

Travis CI​

This script knows about the Travis build environment, assuming a PR based model. To run it, configure .travis.yml to run the happo command:

.travis.yml
language: node_js
script:
  - npx happo

The happo command assumes that your PRs are based off of the main branch. If you're using a different default branch, you can set the --baseBranch argument.

.travis.yml
language: node_js
script:
  - npx happo --baseBranch origin/master

GitLab CI​

The happo command knows about GitLab CI's predefined variables. Run it in merge request pipelines, and in pipelines for your default branch so that baselines exist for merge requests to compare against:

.gitlab-ci.yml
happo:
  image: node:lts
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  script:
    - npm ci
    - npx happo

In a merge request pipeline, Happo compares with the merge request's diff base (CI_MERGE_REQUEST_DIFF_BASE_SHA): the commit where the merge request's branch forked off its target branch, whichever branch that is (a release branch, or another merge request's branch). Merged results and merge train pipelines run on a merge of the merge request into its target branch, but Happo files the screenshots under the merge request's head (CI_MERGE_REQUEST_SOURCE_BRANCH_SHA) and compares with the diff base too, so check the head out before running happo (CI merge commits). Given --afterSha "$CI_COMMIT_SHA" to file them under the merge instead, Happo compares with the target branch commit that merge was made on (CI_MERGE_REQUEST_TARGET_BRANCH_SHA). Without a diff base, a merge request pipeline compares with the merge base of its target branch (CI_MERGE_REQUEST_TARGET_BRANCH_NAME). Other pipelines compare with the merge base of your default branch (CI_DEFAULT_BRANCH), or of --baseBranch when it's given. A merge request's own base and target branch take precedence over --baseBranch, as a GitHub pull request's do; --beforeSha takes precedence over all of them.

When Happo compares with the diff base, it fetches the base's history if the clone stops short of it, for the earlier commits Happo falls back to when the base has no report.

GitLab clones the last 20 commits of a new project's pipelines. When the base branch or the merge base is missing from the clone, Happo fetches more of their history. If it still can't find the merge base, it warns and compares the commit with itself. To clone the whole history instead, set GIT_DEPTH to 0:

.gitlab-ci.yml
variables:
  GIT_DEPTH: 0

Bitbucket Pipelines​

The happo command knows about Bitbucket Pipelines' variables. Run it for pull requests, and for pushes to your default branch so that baselines exist for pull requests to compare against:

bitbucket-pipelines.yml
image: node:lts
pipelines:
  pull-requests:
    '**':
      - step:
          name: Happo
          script:
            - npm ci
            - npx happo
  branches:
    main:
      - step:
          name: Happo
          script:
            - npm ci
            - npx happo

In a pull request pipeline, Happo compares with the merge base of the pull request's destination branch (BITBUCKET_PR_DESTINATION_BRANCH), whichever branch that is. Pull request pipelines merge the destination branch into the pull request before their steps run, but Happo files the screenshots under the pull request's head (BITBUCKET_COMMIT), so check the head out before running happo (CI merge commits). Other pipelines compare with the merge base of origin/main, or of --baseBranch when it's given. A pull request's destination branch takes precedence over --baseBranch; --beforeSha takes precedence over both.

Bitbucket clones the last 50 commits by default. When the destination branch or the merge base is missing from the clone, Happo fetches more of their history. If it still can't find the merge base, it warns and compares the commit with itself. To clone the whole history instead, set the clone depth to full:

bitbucket-pipelines.yml
clone:
  depth: full

Generic CI​

If you are using a different CI service, you'll have to set a few different CLI arguments when invoking the happo command:

  • --beforeSha - the SHA of the baseline commit
  • --afterSha - the SHA of the current HEAD
  • --link - a URL that links back to the change (further instructions)

Posting build statuses​

Your Happo account can be configured to post build statuses to your PRs/commits. Happo currently integrates with GitHub, GitLab, Bitbucket, and Azure DevOps. See specific instructions for the different providers below.

GitHub​

The instructions in this section only work if you are using github.com or the on-premise version of happo.io. If you're using a self-hosted GitHub Enterprise Server, follow the GitHub Enterprise (On-premise) guide instead. If you can't install a GitHub App at all, there is also an alternative solution described in the next section.

Step 1: Install Happo GitHub app​

First you need to install the Happo GitHub App in the repository/repositories you want to run Happo in.

Installing the Happo GitHub App Installing the Happo app at https://github.com/apps/happo

Step 2: Connect with repository​

Once you have the Happo GitHub app installed, you need to connect/activate the right repository with your Happo account on the GitHub integration page on happo.io. Once you're done with that, you're all set to have Happo automatically post statuses on your PRs/commits.

Activating the GitHub repository at https://happo.io/github-integration

Happo build statuses​

Here's what it looks like when Happo posts a status on a pull request:

Happo status posted on a commit on github

If there is a diff, the status will be set to failure. To manually flip this to a success status, just go to the Happo comparison page (linked to by the "Details" link next to the Happo status) and click the Accept button in the Reject/Accept pair in the left sidebar.

The status over on github.com will then change to success (green) for the PR/commit.

Happo status manually accepted cross-posted to github

If there are no diffs, the status is automatically set to success.

Posting statuses without installing the Happo GitHub App​

Removed in happo v7. The --githubToken flag and the githubApiUrl option are gone, and the CLI no longer posts comments. See Upgrading to happo v7.

In happo v6, the CLI could post the Happo status as a pull request comment when passed a --githubToken, for repositories that couldn't install the Happo GitHub App. happo.io now posts statuses itself: install the GitHub App, or see GitHub Enterprise.

Bitbucket​

To allow Happo to post build statuses to your Bitbucket repo, you need to configure Happo with an API token or a Repository access token.

Step 1: Generate a token​

Alternative A: API Token​

On your API tokens page for your profile, generate a scoped API token with the scope read:repository:bitbucket:

  1. Click Create API token with scopes.
  2. Name the token (for example Happo), pick an expiry date, and click Next.
  3. Select Bitbucket as the app, and click Next.
  4. Search for read, select read:repository:bitbucket, and click Next.
  5. Check the summary, and click Create token.
  6. Copy the token. Bitbucket won't show it again once you close the dialog.
Alternative B: Repository access token​

On your repository settings page, go to Security > Access tokens. Generate a repository access token with the Repositories > Read scope:

  1. Click Create access token.
  2. Name the token (for example Happo), and pick an expiry date.
  3. Under Scopes, select Read under Repositories, and click Create.
  4. Copy the access token. Bitbucket won't show it again once you close the dialog.

Step 2: Fill in form at Happo​

Once you have the API token, you can go to the Bitbucket integration page on happo.io and fill out the form. If you have an API token, set the email field to your bitbucket account email. If you have a repository access token, set the email field to x-token-auth. When you're done with filling out the form and the connection is working, you're all set to have Happo automatically post statuses on your PRs/commits.

Happo build statuses​

Here's what it looks like when Happo posts a status on a pull request:

Happo status posted on a bitbucket commit

If there is a diff, the status will be set to failure. To manually flip this to a success status, just go to the Happo comparison page (linked to from the status) and accept the diffs.

The status over on bitbucket.org will then change to success (green) for the PR/commit. If there are no diffs, the status is automatically set to success.

Azure​

Step 1: Generate a Personal Access Token (PAT)​

To authorize Happo to post statuses to your PRs/commits, you need to generate an Personal Access Token.

  1. In Azure DevOps, open User settings and choose Personal access tokens.
  2. Click New Token.
  3. Name the token (for example Happo), and choose the organization your repositories are in and an expiration.
  4. Under Scopes, choose Custom defined, then click Show all scopes.
  5. Under Code, select Read and Status, and click Create.
  6. Copy the token. Azure DevOps won't show it again once you close the dialog.

We need the read scope to figure out the right baseline reports to use. The status scope is used when posting build statuses to PRs.

Step 2: Fill in form at Happo​

Once you have the PAT, you can go to the Azure integration page on happo.io and fill out the form. Once you're done with that, you're all set to have Happo automatically post statuses on your PRs/commits.

Happo build statuses​

Here's what it looks like when Happo posts a status on a pull request:

Happo status posted on an azure PR

If there is a diff, the status will be set to failure. To manually flip this to a success status, just go to the Happo comparison page (linked to from the status) and accept the diffs.

The status over on Azure DevOps will then change to success (green) for the PR/commit. If there are no diffs, the status is automatically set to success.

GitLab​

This integration is experimental. Statuses, baselines, and merge request events are in place; if something looks wrong, please get in touch.

To allow Happo to post build statuses to your GitLab project, generate a project access token with the api scope.

Step 1: Generate a project access token​

In GitLab, open the project → Settings → Access tokens. Create a token with the api scope. Note the project's numeric ID (shown on the project overview page) — Happo uses the ID rather than the path so the connection survives renaming or moving the project.

Step 2: Fill in the form at Happo​

Once you have the token, go to the GitLab integration page on happo.io and fill out the form. Leave Instance URL blank for gitlab.com; for a self-managed instance, enter the origin it is served from (for example https://gitlab.example.com).

When the connection is working, Happo posts statuses to your commits and merge requests and uses Git history to find the right baseline report.

Projects can override the account-default GitLab project ID from each project's edit page.

Merge request events (optional)​

Without a webhook, Happo still posts statuses and finds baselines. With a webhook, Happo can also show who opened a merge request, cancel superseded runs when you push, and copy reports across on merge.

In GitLab, go to Settings → Webhooks, add a webhook pointing at the URL shown on the Happo GitLab integration page with the Merge request events trigger, and paste the same secret token into the Happo form.

Email notifications​

You can set up the CI integration to send email notifications when comparison reports are ready. Set the --notify CLI argument to one or more (comma-separated) email addresses and emails will be sent from Happo when results are available.

npx happo --notify user@example.com

If you want to send the email to the person responsible for the change/PR that triggered the Happo tests, you can do that via a git show one-liner. Here's an example for GitHub Actions:

.github/workflows/happo.yaml
- run: npx happo --notify $(git show -s --format=%ae HEAD)

Multiple recipients​

Use a comma-separated list of email addresses to send the notification to several recipients:

npx happo --notify user@example.com,service-account@mycompany.com

The --link argument is used to contextualize the report. Happo will link back to the --link URL whenever the report is shown. Some good examples of links to use:

  • A URL that leads to the pull request/merge request that started the build
  • A link to the commit associated with the build

When --link isn't given, the happo command sets it from the CI environment: on GitHub Actions, Circle CI, Travis CI, Azure DevOps, GitLab CI and Bitbucket Pipelines, it links to the pull request or merge request that started the build, or to the commit when the build isn't for one.