Skip to main content

Visual Testing

info
What you'll learn​
  • The difference between functional and visual testing
  • How to use visual testing plugins
  • Best practices for visual testing

Introduction​

Cypress offers several solutions for testing an application including:

While many of our solutions, like Test Replay, Cypress Accessibility and UI Coverage, offer a visual display of the application, having an automated way to ensure your application visually looks as intended is crucial. This is where visual testing comes in.

Visual testing is a type of testing that focuses on the appearance of the application. It is a way to ensure that the application looks the same to the user after changes are made.

it('completes todo', () => {
  cy.visit('/') // opens TodoMVC running at "baseUrl"
  cy.get('.new-todo').type('write tests{enter}')
  cy.contains('.todo-list li', 'write tests').find('.toggle').check()

  cy.contains('.todo-list li', 'write tests').should('have.class', 'completed')
})

You could technically write a functional test asserting the CSS properties using the have.css assertion, but these may quickly become cumbersome to write and maintain, especially when visual styles rely on a lot of CSS styles.

cy.get('.completed').should('have.css', 'text-decoration', 'line-through')
cy.get('.completed').should('have.css', 'color', 'rgb(217,217,217)')

Your visual styles may also rely on more than CSS, perhaps you want to ensure an SVG or image has rendered correctly or shapes were correctly drawn to a canvas.

Luckily, Cypress gives a stable platform for including plugins that can perform visual testing.

Typically such plugins take an image snapshot of the entire application under test or a specific element, and then compare the image to a previously approved baseline image. If the images are the same (within a set pixel tolerance), it is determined that the web application looks the same to the user. If there are differences, then there has been some change to the DOM layout, fonts, colors or other visual properties that needs to be investigated.

For example, one can use the cypress-plugin-snapshots plugin and catch the following visual regression:

.todo-list li.completed label {
  color: #d9d9d9;
  /* removed the line-through */
}
it('completes todo', () => {
  cy.visit('/')
  cy.get('.new-todo').type('write tests{enter}')
  cy.contains('.todo-list li', 'write tests').find('.toggle').check()

  cy.contains('.todo-list li', 'write tests').should('have.class', 'completed')

  // run 'npm install cypress-plugin-snapshots --save'
  // capture the element screenshot and
  // compare to the baseline image
  cy.get('.todoapp').toMatchImageSnapshot({
    imageConfig: {
      threshold: 0.001,
    },
  })
})

This open source plugin compares the baseline and the current images side by side if pixel difference is above the threshold; notice how the baseline image (Expected result) has the label text with the line through, while the new image (Actual result) does not have it.

Baseline vs current image

Like most image comparison tools, the plugin also shows a difference view on mouse hover:

Highlighted changes

Tooling​

There are several published, open source plugins, listed in the Plugins Visual Testing section, and several commercial companies have developed visual testing solutions on top of Cypress listed below.

Applitools​

See Applitools' official docs and our tutorial.

Second joint webinar with Applitools with a focus on Component Testing

Percy​

See Percy's official docs and our tutorial.

info

The Cypress Real World App (RWA) uses the cy.percySnapshot() command provided by the Cypress Percy plugin to take visual snapshots throughout the user journey end-to-end tests

Check out the Real World App test suites to see these Percy and Cypress in action.

Happo​

See Happo's official docs and our webinar and blog.

Chromatic​

Chromatic leverages your existing Cypress setup—configuration, mocking, and tests—to enable visual testing of your application's UI. With the Chromatic plugin installed, Chromatic captures an archive of your UI while your Cypress tests are running.

See Chromatic's official docs and their blog.

Argos​

Argos integrates seamlessly with your existing Cypress setup—configuration, mocking, and tests—to enable visual testing of your application’s UI. With Argos set up, it captures and analyzes screenshots during your Cypress test runs to detect visual changes.

Check out Argos's official documentation for setup instructions and best practices.

Wopee.io​

Wopee.io seamlessly integrates with Cypress to enhance test coverage, expedite maintenance, and ensure more resilient test runs. Aiming for autonomous visual testing, Wopee.io allows you to easily incorporate visual validation into your existing Cypress tests, adding an extra layer of quality assurance.

See Wopee.io's official docs, our webinar and their blog and webinar.

Best practices​

As a general rule there are some best practices when visual testing.

Recognize the need for visual testing​

Assertions that verify style properties

cy.get('.completed').should('have.css', 'text-decoration', 'line-through')
  .and('have.css', 'color', 'rgb(217,217,217)')
cy.get('.user-info').should('have.css', 'display', 'none')
...

If your end-to-end tests become full of assertions checking visibility, color and other style properties, it might be time to start using visual diffing to verify the page appearance.

DOM state​

tip

Best Practice: Take a snapshot after you confirm the page is done changing.

For example, if the snapshot command is cy.mySnapshotCommand:

Incorrect Usage

// the web application takes time to add the new item,
// sometimes it takes the snapshot BEFORE the new item appears
cy.get('.new-todo').type('write tests{enter}')
cy.mySnapshotCommand()

Correct Usage

// use a functional assertion to ensure
// the web application has re-rendered the page
cy.get('.new-todo').type('write tests{enter}')
cy.contains('.todo-list li', 'write tests')
// great, the new item is displayed,
// now we can take the snapshot
cy.mySnapshotCommand()

Timestamps​

tip

Best Practice: Control the timestamp inside the application under test.

Below we freeze the operating system's time to Jan 1, 2018 using cy.clock() to ensure all images displaying dates and times match.

const now = new Date(2018, 1, 1)

cy.clock(now)
// ... test
cy.mySnapshotCommand()

Application state​

tip

Best Practice: Use cy.fixture() and network mocking to set the application state.

Below we stub network calls using cy.intercept() to return the same response data for each XHR request. This ensures that the data displayed in our application images does not change.

cy.intercept('/api/items', { fixture: 'items' }).as('getItems')
// ... action
cy.wait('@getItems')
cy.mySnapshotCommand()

Visual diff elements​

tip

Best Practice: Use visual diffing to check individual DOM elements rather than the entire page.

Targeting specific DOM element will help avoid visual changes from component "X" breaking tests in other unrelated components.

Component testing​

tip

Best Practice: Use Component Testing to test the individual components functionality in addition to end-to-end and visual tests.

See also​

Contents