The Quick Guide to Visual Regression Testing With Cypress

A quick guide to visual regression testing with the popular end to end testing platform, Cypress.

‘Visual regression testing is the process of validating visual parts of an application or website to ensure that any code modifications do not introduce any unintentional changes’ - (Getting started with visual regression testing and WebdriverIO, 2021)

Here’s our quick guide on how to conduct visual regression testing with Cypress and the Cypress Image Snapshot plugin.

What is Cypress?

Cypress is the testing tool that is helping teams around the world provide good quality software.

In the QA world, Cypress appears to be gaining more and more attention in recent years and its intuitive interface and simple setup often triumphs over long-time open source testing application, Selenium. (I won’t go into a full-on comparison of the two), but developers often lean towards Cypress because of its modern architectural approach, dependable results and plugin extensibility. Cypress also has built-in snapshot functionality and video recording capabilities, making it a great tool for “proper” UI testing!  

For developers, using Cypress is a no-brainer as it’s the testing platform dedicated to building modern apps and websites with modern JavaScript frameworks, allowing you to write end-to-end tests, integration tests or unit tests with ease.

 How to conduct visual regression testing with Cypress.

Before we get stuck into setup and test writing, you’ll need Node and npx installed on your machine. If you need to set this up, head to our post on installation and follow the simple steps.

1.  Install Cypress.

Start by installing Cypress with npm:

npm install cypress --save-dev

Following a successful install, runCypress Test Runner from your project root using the npx command:

npx cypress open

This will start Cypress Test Runner and create all the necessary files you need for your project (you should be notified about the file changes in a dialogue window). Some handy example files are also added to the project, so do check these out for inspiration on writing initial tests. 

Cypress Test Runner will also detect and show you a list of browsers available on your device that you can use for testing purposes. 

Cypress has a lot of configuration possibilities that are worth exploring before you start writing any tests. For example, you can change settings for tasks and commands timeout, customize names and paths for test files, plugins and screenshots etc. You can also adjust your browser options, like security checks or user agent options.

Let’s start by adding a base URL and video recording setting (all of this is done in cypress.json file).

{
  "video": false,
  "baseUrl": "https://wearecogworks.netlify.app"
}


2. Decide on your visual regression testing tool.

With Cypress ready, it’s time to install our visual regression tool. There are plenty of possibilities (check https://docs.cypress.io/plugins/directory#Visual%20Testing). Technically you can choose whatever suits you best. For this tutorial, we’ve opted for  Cypress Image Snapshot.

3. Install Cypress Image Snapshot.

 npm install --save-dev cypress-image-snapshot

Now you need to modify your /cypress/plugins/index.js file and add this little snippet:

const { addMatchImageSnapshotPlugin } = require('cypress-image-snapshot/plugin')

module.exports = (on, config) => {
  addMatchImageSnapshotPlugin(on, config)
}

Additionally,  add image snapshot command to your /cypress/support/commands.js

import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command'

addMatchImageSnapshotCommand()

You can of course customize some plugin settings here or you can do it later in the test file where you run image snapshot function. By default the settings are set to:

addMatchImageSnapshotCommand({
  failureThreshold: 0.03, // threshold for entire image
  failureThresholdType: 'percent', // percent of image or number of pixels
  customDiffConfig: { threshold: 0.1 }, // threshold for each pixel
  capture: 'viewport', // capture viewport in screenshot
})


And that’s all in the basic setup. Now we are ready to write the first test and take initial screenshots.

4. Write your visual regression test.

If you take a look at the Cypress folders structure you might be wondering, where do I start writing my tests? The recommended place is the ‘integration’ directory but you are free to create your own and set its path in the configuration file by using ‘integrationFolder’ option.

In the ‘integration’ directory you will see that we already have the examples folder where you can take a look at a few predefined test examples. It’s worth exploring them to see in action how you can approach your own scenarios, but I've removed them so you can see a clear folder structure for this tutorial.

Let’s write the first visual test!

Create a new file called homepage.spec.js. Inside the file, specify that the browser should open the homepage, take a full-page snapshot and compare it to the baseline image. Don’t worry if you do not have any baseline images yet, when running the test for the first time Cypress will create them for you!

Additionally,  create snapshots in multiple screen resolutions so you should define this scenario at the beginning of the test. You can specify the exact screen width and height in an Array or choose some of the predefined options for devices like iPad 2 or macbook 13. My code for the test looks like this:

describe('Homepage Test', function () {
// Go to Homepage
  before(() => {
    cy.visit('/')
  })

  // Make the snapshot and compare it with baseline
  // for each of the defined screen sizes

  sizes.forEach((size) => {
    it(`Should match  ${size} image snapshot`, function () {
      cy.get('.header').invoke('css', 'position', 'absolute')
      cy.matchImageSnapshot(`Homepage-Test-${size}`, { capture: 'fullPage' })
    })
  })
})


Now it's time to review the code. At the top of the file, define an array of screen sizes you want to check the page on, then create a test suite with describe() and a snapshot test case on it() section. (The syntax in Cypress is similar to Mocha BDD so you can use all the patterns here). Next, wrap the code in a simple forEach() function to iterate over defined screen sizes and make screenshots of the page.

Because the header on the testing page has the position set to ‘fixed’  and Cypress Image Snapshot creates a full-page screenshot by joining multiple images on scroll, we have to change the  CSS property for this HTML element. To change the CSS property,  simply adjust the header style ‘position’ value to ‘absolute’ by running the invoke() method. After this, add the image snapshot function with the customized file name and with ‘fullPage’ value for the capture option.

The test is now ready so let’s come back to Cypress Test Runner and start it. You can do it two ways:

1) By clicking on spec file name 

2) When there are multiple tests in the integration folder by clicking the ‘Run (x)  integration spec’ button on the right-hand side of the Test Runner. 

The difference between the above is essentially the image output. Clicking a single test will save a snapshot in the Cypress/snapshots/testFileName.js folder. Clicking the button though will create screen capture images in the Cypress/snapshots/All Integration Specs directory.

The initial run should go without any issues because (like I mentioned above) it creates baseline images and saves them to the snapshots folder. The next test runs will refer to those saved screenshots. 

If you ever need to create a new baseline you can run Cypress with:

npx cypress open --env updateSnapshots=true 

Let’s assume you made changes in the code and there were some global font adjustments. You ran the visual regression test for the homepage and it failed. What happened?

As you can see the new screenshot was different from the baseline one. The pictured error message also points out that you can check this difference in a comparison file saved to our some_local_path\cypress\snapshots\homepage.spec.js_diff_output_\Homepage-Test-1920,1080.diff.png.

This way we can exactly know what happened and review changes to the code.

 



And that’s it. Visual regression test, done!

If you’re keen to learn more about how visual regression can help your projects, head over to my other recent post, on testing with Webdriver.IO! Leave a comment below if you want me to cover anything specific related to this topic! 

Arek 

  • Image for What it Means to be an Umbraco Gold Partner News

    What it Means to be an Umbraco Gold Partner

  • Image for Website Security Checklist: The 4 Basics Build

    Website Security Checklist: The 4 Basics

  • Image for Azure Resource Naming Tips Build

    Azure Resource Naming Tips

  • Image for Powerful Content Automation Solutions for Better Web Editing Strategy

    Powerful Content Automation Solutions for Better Web Editing

Ready to collaborate ?

Get in touch to see how we can transform your digital presence.

Send us a message