The Quick Guide to Visual Regression Testing With Cypress

‘Visual regression testing is the process of validating visual parts of an application or website to ensure that any code modifications do not introduce any unintentional changes’ - (Getting started with visual regression testing and WebdriverIO, 2021)

Here’s our quick guide on conducting visual regression testing with Cypress and the Cypress Image Snapshot plugin.

What is Cypress?

Cypress is the testing tool that is helping teams around the world provide good quality software.

In the QA world, Cypress appears to be gaining more and more attention in recent years, and its intuitive interface and simple setup often triumph over the long-time open-source testing application, Selenium. (I won’t go into a full-on comparison of the two), but developers often lean towards Cypress because of its modern architectural approach, dependable results and plugin extensibility. Cypress also has built-in snapshot functionality and video recording capabilities, making it a great tool for “proper” UI testing!  

For developers, using Cypress is a no-brainer as it’s the testing platform dedicated to building modern apps and websites with modern JavaScript frameworks, allowing you to easily write end-to-end tests, integration tests or unit tests.

How to conduct visual regression testing with Cypress.

Before we get stuck into setup and test writing, you’ll need Node and npx installed on your machine. If you need to set this up, head to our post on installation and follow the simple steps.

1.  Install Cypress.

Start by installing Cypress with npm:

npm install cypress --save-dev

Following a successful install, runCypress Test Runner from your project root using the npx command:

npx cypress open

This will start Cypress Test Runner and create all the necessary files you need for your project (you should be notified about the file changes in a dialogue window). Some handy example files are also added to the project, so do check these out for inspiration on writing initial tests. 

Cypress Test Runner will also detect and show you a list of browsers available on your device that you can use for testing purposes. 

Cypress has a lot of configuration possibilities that are worth exploring before you start writing any tests. For example, you can change settings for tasks and commands timeout, customize names and paths for test files, plugins and screenshots etc. You can also adjust your browser options, like security checks or user agent options.

Let’s start by adding a base URL and video recording setting (all of this is done in the cypress.json file).

{  "video": false,  "baseUrl": "https://wearecogworks.netlify.app"}

2. Decide on your visual regression testing tool.

With Cypress ready, it’s time to install our visual regression tool. There are plenty of possibilities (check https://docs.cypress.io/plugins/directory#Visual%20Testing). Technically, you can choose whatever suits you best. For this tutorial, we’ve opted for  Cypress Image Snapshot.

3. Install Cypress Image Snapshot.

 npm install --save-dev cypress-image-snapshot

Now you need to modify your /cypress/plugins/index.js file and add this little snippet:

const { addMatchImageSnapshotPlugin } = require('cypress-image-snapshot/plugin')module.exports = (on, config) => {  addMatchImageSnapshotPlugin(on, config)}

Additionally,  add image snapshot command to your /cypress/support/commands.js

import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command'addMatchImageSnapshotCommand()

You can, of course customize some plugin settings here or you can do it later in the test file where you run image snapshot function. By default, the settings are set to:

addMatchImageSnapshotCommand({  failureThreshold: 0.03, // threshold for entire image  failureThresholdType: 'percent', // percent of image or number of pixels  customDiffConfig: { threshold: 0.1 }, // threshold for each pixel  capture: 'viewport', // capture viewport in screenshot})

And that’s all in the basic setup. We are ready to write the first test and take initial screenshots.

4. Write your visual regression test.

If you look at the Cypress folder structure, you might wonder, where do I start writing my tests? The recommended place is the ‘integration’ directory, but you can create your own and set its path in the configuration file using the ‘integrationFolder’ option.

In the ‘integration’ directory, you will see that we already have the examples folder where you can look at a few predefined test examples. It’s worth exploring them to see how you can approach your own scenarios, but I've removed them so you can see a clear folder structure for this tutorial.

Let’s write the first visual test!

Create a new file called homepage.spec.js. Inside the file, specify that the browser should open the homepage, take a full-page snapshot and compare it to the baseline image. Don’t worry if you do not have any baseline images yet; when running the test for the first time, Cypress will create them for you!

Additionally,  create snapshots in multiple screen resolutions, so you should define this scenario at the beginning of the test. You can specify the exact screen width and height in an Array or choose predefined options for devices like iPad 2 or MacBook 13. My code for the test looks like this:

describe('Homepage Test', function () { // Go to Homepage  before(() => {    cy.visit('/')  })  // Make the snapshot and compare it with baseline  // for each of the defined screen sizes  sizes.forEach((size) => {    it(`Should match  ${size} image snapshot`, function () {      cy.get('.header').invoke('css', 'position', 'absolute')      cy.matchImageSnapshot(`Homepage-Test-${size}`, { capture: 'fullPage' })    })  })})

Now, it's time to review the code. At the top of the file, define an array of screen sizes you want to check the page on, then create a test suite with a describe() and a snapshot test case on it() section. (The syntax in Cypress is similar to Mocha BDD, so you can use all the patterns here). Next, wrap the code in a simple forEach() function to iterate over defined screen sizes and make screenshots of the page.

Because the header on the testing page has the position set to ‘fixed’  and Cypress Image Snapshot creates a full-page screenshot by joining multiple images on scroll, we have to change the  CSS property for this HTML element. To change the CSS property,  simply adjust the header style ‘position’ value to ‘absolute’ by running the invoke() method. After this, add the image snapshot function with the customized file name and with the ‘fullPage’ value for the capture option.

The test is now ready, so let’s return to Cypress Test Runner and start it. You can do it two ways:

1) By clicking on the spec file name 

2) When multiple tests are in the integration folder, click the ‘Run (x)  integration spec’ button on the right-hand side of the Test Runner. 

The difference between the above is essentially the image output. Clicking a single test will save a snapshot in the Cypress/snapshots/testFileName.js folder. Clicking the button will create screen capture images in the Cypress/snapshots/All Integration Specs directory.

The initial run should go without any issues because (like I mentioned above) it creates baseline images and saves them to the snapshots folder. The next test runs will refer to those saved screenshots. 

If you ever need to create a new baseline, you can run Cypress with:

npx cypress open --env updateSnapshots=true 

Let’s assume you made changes in the code, and there were some global font adjustments. You ran the visual regression test for the homepage, and it failed. What happened?

As you can see the new screenshot was different from the baseline one. The pictured error message also points out that you can check this difference in a comparison file saved to our some_local_path\cypress\snapshots\homepage.spec.js_diff_output_\Homepage-Test-1920,1080.diff.png.

This way, we can exactly know what happened and review changes to the code.

 

And that’s it. Visual regression test complete.

Learn more about how visual regression can help your projects.

Head over to my other post: how to get started with visual regression testing and Webdriver.IO!

Leave a comment below if you want me to cover anything specific related to this topic, or why not say Hi on LinkedIn: https://www.linkedin.com/in/arkadiusz-koperek-27a35a45/

Arek