Developers are usually concerned with the quality of their code. There are different kinds of tests that help us avoid breaking code when a new feature is added in a project. But what can we do to ensure that components don't look different over time?

In this post, you will learn how to use Cypress to capture parts of pages of a website. After that, you will integrate the testing tool in CI to ensure that in the future no one will make unwanted changes to your project.

My motivation for creating this testing strategy came from work. At Thinkific we have an internal Design System and we added Cypress to avoid surprises when working in CSS/JS files.

By the end of this post we will have PRs with Cypress tests:

Cypress bot

Before we start

I created a sample website to mimic a Component Library. It is a very simple website created with TailwindCSS and hosted in Vercel. It documents 2 components: badge and button.

You can check out the source code in GitHub. The website is static and it is inside the public folder. You can see the website locally by running npm run serve and checking in the browser http://localhost:8000.

Sample website

Adding Cypress and Cypress Image Snapshot

Start by cloning the example repository. Next, create a new branch and install Cypress Image Snapshot, the package responsible for capturing/comparing screenshots.

git checkout -b add-cypress
npm install -D cypress cypress-image-snapshot

After adding the packages, a few extra steps are needed to add Cypress Image Snapshot in Cypress.

Create a cypress/plugins/index.js file with the following content:

const { addMatchImageSnapshotPlugin } = require('cypress-image-snapshot/plugin');

module.exports = (on, config) => {
  addMatchImageSnapshotPlugin(on, config);
};

Next, create a cypress/support/index.js file containing:

import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command';

addMatchImageSnapshotCommand();

Creating the screenshot test

Time to create the screenshot test. Here is the plan:

  1. Cypress will visit each page (badge and button) of the project.
  2. Cypress will take a screenshot of each example in the page. The Badge page has 2 examples (Default and Pill), while the Button page has 3 examples (Default, Pill and Outline). All these examples are inside a <div> element with a cypress-wrapper. This class was added with the only intention being to identify what needs to be tested.

The first step is creating Cypress configuration file (cypress.json):

{
  "baseUrl": "http://localhost:8000/",
  "video": false
}

The baseUrl is the website running locally. As I mentioned before, npm run serve will serve the content of the public folder. The second option, video disables Cypress video recording, which we won't use in this project.

Time to create the test. In cypress/integration/screenshot.spec.js, add:

const routes = ['badge.html', 'button.html'];

describe('Component screenshot', () => {
  routes.forEach((route) => {
    const componentName = route.replace('.html', '');
    const testName = `${componentName} should match previous screenshot`;

    it(testName, () => {
      cy.visit(route);
  
      cy.get('.cypress-wrapper').each((element, index) => {
        const name = `${componentName}-${index}`;
  
        cy.wrap(element).matchImageSnapshot(name);
      });
    });
  });
});

In the code above, I am dynamically creating tests based in the routes array. The test will create one image per .cypress-wrapper element that the page has.

Last, inside the package.json let's create the command to trigger the tests:

{
  "test": "cypress"
}

From here, there are 2 options: run Cypress in headless mode with npm run cypress run or use the Cypress Test Runner with npm run cypress open.

Headless option

Using npm run cypress run, the output should be similar to the next image:

Output of first test

The tests will pass and 5 images will be created under the /snapshots/screenshot.spec.js folder.

Test Runner option

Using npm run cypress open, Cypress Test Runner will be opened and you can follow the tests step by step.

Cypress Test Runner screenshot

Our first milestone is done, so let's merge this branch to master. If you want to see the work done so far, jump in my Pull Request.

Using Cypress inside Docker

If you run the test above alternating between headless and Test Runner, you may notice that the screenshot will vary.

Using the Test Runner with a retina display computer, you may get retina images (2x), while the headless mode doesn't give you high-quality screenshots.

Also, it is important to say that the screenshots may vary according to your Operating System.

Linux and Windows, for instance, have apps with visible scrollbars, while macOS hides the scrollbar.

If the content captured in the screenshot doesn't fit a component, you may or may not have a scrollbar. If your project relies on OS default fonts, screenshots will also be different according to the environment.

In order to avoid these inconsistencies, tests will run inside Docker so the developer's computer won't affect the screenshot captures.

Let's start by creating a new branch:

git checkout -b add-docker

Cypress offers different Docker images - you can check out the details in their documentation and their blog.

For this example, I will use the cypress/included image, which includes Electron and is ready to be used.

We need to make two changes: change the baseUrl in the cypress.json file:

{
  "baseUrl": "http://host.docker.internal:8000/",
}

and the test command in the package.json file:

{
  "test": "docker run -it -e CYPRESS_updateSnapshots=$CYPRESS_updateSnapshots --ipc=host -v $PWD:/e2e -w /e2e cypress/included:4.11.0"
}

Running npm run test will bring us a problem:

Output of test

The images are slightly different but why? Let's see what is inside the __diff_output__ folder:

Button's difference

As I mentioned earlier, typography inconsistencies! The Button component uses the OS default font. Since Docker is running inside Linux, the rendered font won't be the same that I have installed on macOS.

Since now we moved to Docker, these screenshots are outdated. Time to update the snapshots:

CYPRESS_updateSnapshots=true npm run test

Please notice that I am prefixing the test command with the environment variable CYPRESS_updateSnapshots.

The second milestone is done. In case you need help, check out my pull request.

Let's merge this branch and move forward.

Adding CI

Our next step is adding the tests in CI. There are different CI solutions in the market but for this tutorial, I will use Semaphore. I am not affiliated with them and I use their product at work, so it was for me a natural choice.

The configuration is straightforward and it can be adapted to other solutions like CircleCI or Github Actions.

Before we create our Semaphore configuration file, let's prepare our project to run in CI.

The first step is installing start-server-and-test. As the package name says, it will start a server, wait for the URL, and then run a test command:

npm install -D start-server-and-test

Second, edit the package.json file:

{
  "test": "docker run -it -e CYPRESS_baseUrl=$CYPRESS_baseUrl -e CYPRESS_updateSnapshots=$CYPRESS_updateSnapshots --ipc=host -v $PWD:/e2e -w /e2e cypress/included:4.11.0",
  "test:ci": "start-server-and-test serve http://localhost:8000 test"
}

In the test script, we are adding the CYPRESS_baseUrl environment variable. This will allow us to change the base URL used by Cypress dynamically. Also, we are adding the test:ci script, which will run the package we just installed.

We are ready for Semaphore. Create the .semaphore/semaphore.yml file with the following content:

 1 version: v1.0
 2 name: Cypress example
 3 agent:
 4   machine:
 5     type: e1-standard-2
 6     os_image: ubuntu1804
 7 blocks:
 8   - name: Build Dependencies
 9     dependencies: []
10     task:
11       jobs:
12         - name: NPM
13           commands:
14             - sem-version node 12
15             - checkout
16             - npm install
17   - name: Tests
18     dependencies: ['Build Dependencies']
19     task:
20       prologue:
21         commands:
22           - sem-version node 12
23           - checkout
24       jobs:
25         - name: Cypress
26           commands:
27             - export CYPRESS_baseUrl="http://$(ip route | grep -E '(default|docker0)' | grep -Eo '([0-9]+\.){3}[0-9]+' | tail -1):8000"
28             - npm run test:ci

Breaking the configuration down in detail:

  • Lines 1-6 define which kind of instance we will use in their environment
  • Lines 8 and 16 create 2 blocks: the first block, "Build Dependencies" will run npm install, downloading the dependencies we need. The second block, "Tests" will run Cypress, with a few differences.
  • In line 27, we are dynamically setting the CYPRESS_baseUrl environment variable based on the IP Docker is using at the moment. This will replace http://host.docker.internal:8000/, which may not work in all environments.
  • In line 28, we finally run the test using start-server-and-test: once the server is ready for connections, Cypress will run the test suite.

Another milestone is done, time to merge our branch! You can check out the Pull request that contains all the files from this section and check the build inside Semaphore.

Recording the tests in cypress.io

Reading the output of tests in CI is not very friendly. In this step, we will integrate our project with cypress.io.

The following steps are based on the Cypress documentation.

Let's start by getting a project ID and a record key. In the terminal, create a new branch and run:

git checkout -b add-cypress-recording
CYPRESS_baseUrl=http://localhost:8000 ./node_modules/.bin/cypress open

Earlier I mentioned that we would be using Cypress inside Docker. But here we are opening Cypress locally since this is the only way to integrate with the website dashboard.

Inside Cypress, let's go to the Runs tab, click in "Set up project to record", and choose a name and visibility. We will get a projectId that is automatically added in the cypress.json file and a private record key. Here is a video of the steps:

In Semaphore, I added the record key as an environment variable called CYPRESS_recordKey. Next let's update our test script to use the variable:

{
  "test:ci": "start-server-and-test 'serve' 8000 'npm run test -- run --record --key $CYPRESS_recordKey'"
}

That is pretty much all that needs to be done. In the Pull request we can see the cypress.io integration in the comments. There is even a deep link that takes us to their dashboard and shows all the screenshots. Check out the video below:

Time to merge our work, and that is the end of our integration.

Testing in real life

Imagine we are working on a change that affects the padding of the buttons: time to test if Cypress will capture the difference.

In the example website, let's double the horizontal padding from 16px to 32px. This change is quite simple since we are using Tailwind CSS: px-4 gets replaced by px-8. Here is that Pull request.

As we might expect, Cypress captured that the button doesn't match the screenshots. Visiting the page, we can check the screenshot of the broken test:

The diff file shows the original screenshot on the left, the current result on the right, and they are combined in the middle. We also have the option to download the image so we can see the issue better:

Button before and after

If this is not an issue, update the screenshots:

CYPRESS_updateSnapshots=true npm run test

The end

That's it for today. I hope you have learned how Cypress can be useful to ensure no one is adding unexpected changes to a project.

Also posted on my blog. If you like this content, follow me on Twitter and GitHub.