Skip to main content

Cross Browser Testing

Cypress has the capability to run tests across multiple browsers. Currently, Cypress has support for Chrome-family browsers (including Electron and Chromium-based Microsoft Edge), WebKit (Safari's browser engine), and Firefox.

caution
Web Security

Tests that require the chromeWebSecurity configuration option to be disabled may experience issues in non-Chromium based browsers.

Excluding Electron, any browser you want to run Cypress tests in needs to be installed on your local system or CI environment. A full list of detected browsers is displayed within the browser selection menu of Cypress.

Cypress with Firefox selected as the browser

The desired browser can also specified via the --browser flag when using run command to launch Cypress. For example, to run Cypress tests in Firefox:

cypress run --browser firefox

To make launching of Cypress with a specific browser even more convenient, npm scripts can be used as a shortcut:

"scripts": {
"cy:run:chrome": "cypress run --browser chrome",
"cy:run:firefox": "cypress run --browser firefox"
}

Continuous Integration Strategies​

When incorporating testing of multiple browsers within your QA process, you must implement a CI strategy that provides an optimal level of confidence while taking into consideration test duration and infrastructure costs. This optimal strategy will vary by the type and needs of a particular project. This guide we present several strategies to consider when crafting the strategy for your project.

CI strategies will be demonstrated using the Circle CI Cypress Orb for its concise and readable configuration, but the same concepts apply for most CI providers.

info
Docker Images for Testing

The CI configuration examples within this guide use Cypress's Docker images to provision testing environments with desired versions of Node, Chrome, and Firefox.

Periodic Basis​

Generally, it is desired to run tests with each pushed commit, but it may not be necessary to do so for all browsers. For example, we can choose to run tests within Chrome for each commit, but only run Firefox on a periodic basis (i.e. nightly). The periodic frequency will depend on the scheduling of your project releases, so consider a test run frequency that is appropriate for the release schedule of your project.

info
Cron Scheduling

Typically CI providers allow for the scheduling of CI jobs via cron expressions. For example, the expression 0 0 * * * translates to "everyday at midnight" or nightly. Helpful online utilities are available to assist with creation and translation of cron expressions.

The following example demonstrates a nightly CI schedule against production (master branch) for Firefox:

version: 2.1
orbs:
cypress: cypress-io/cypress@3
workflows:
nightly:
triggers:
- schedule:
cron: '0 0 * * *'
filters:
branches:
only:
- master
jobs:
- cypress/run:
install-browsers: true
cypress-command: 'npx cypress run --browser firefox'
start-command: 'npm start'

Production Deployment​

For projects that exhibit consistently stable behavior across browsers, it may be better to run tests against additional browsers only before merging changes in the production deployment branch.

The following example demonstrates only running Firefox tests when commits are merged into a specific branch (develop branch in this case) so any potential Firefox issues can be caught before a production release:

version: 2.1
orbs:
cypress: cypress-io/cypress@3
workflows:
test_develop:
jobs:
- filters:
branches:
only:
- develop
- cypress/run:
install-browsers: true
cypress-command: 'npx cypress run --browser firefox'
start-command: 'npm start'

Subset of Tests​

We can choose to only run a subset of tests against a given browser. For example, we can execute only the happy or critical path related test files, or a directory of specific "smoke" test files. It is not always necessary to have both browsers always running all tests.

In the example below, the Chrome cypress/run job runs all tests against Chrome and reports results to Cypress Cloud using a (group) named chrome.

The Firefox cypress/run job runs a subset of tests, defined in the spec parameter, against the Firefox browser, and reports the results to Cypress Cloud under the group firefox-critical-path.

info

Note: The name under each cypress/run job which will be shown in the Circle CI workflow UI to distinguish the jobs.

version: 2.1
orbs:
cypress: cypress-io/cypress@3
workflows:
build:
jobs:
- cypress/run:
name: Chrome
start-command: 'npm start'
install-browsers: true
cypress-command: 'npx cypress run --browser chrome --record --group chrome'
- cypress/run:
name: Firefox
start-command: 'npm start'
cypress-command: 'npx cypress run --browser firefox --record --group
firefox-critical-path --spec
cypress/e2e/signup.cy.js,cypress/e2e/login.cy.js'

Parallelize per browser​

Execution of test files can be parallelized on a per group basis, where test files can be grouped by the browser under test. This versatility enables the ability to allocate the desired amount of CI resources towards a browser to either improve test duration or to minimize CI costs.

You do not have to run all browsers at the same parallelization level. In the example below, the Chrome dedicated cypress/run job runs all tests in parallel, across 4 machines, against Chrome and reports results to Cypress Cloud under the group name chrome. The Firefox dedicated cypress/run job runs a subset of tests in parallel, across 2 machines, defined by the spec parameter, against the Firefox browser and reports results to Cypress Cloud under the group named firefox.

version: 2.1
orbs:
cypress: cypress-io/cypress@3
workflows:
build:
jobs:
- cypress/run:
name: Chrome
cypress-command: 'npx cypress run --record --parallel --group chrome --browser chrome'
start-command: 'npm start'
parallelism: 4
install-browsers: true
- cypress/run:
name: Firefox
cypress-command:
'npx cypress run --record --parallel --group firefox --browser
firefox --spec
cypress/e2e/app.cy.js,cypress/e2e/login.cy.js,cypress/e2e/about.cy.js'
start-command: 'npm start'
parallelism: 2
install-browsers: true

Running Specific Tests by Browser​

There may be instances where it can be useful to run or ignore one or more tests when in specific browsers. For example, test run duration can be reduced by only running smoke-tests against Chrome and not Firefox. This type of granular selection of test execution depends on the type of tests and the level of confidence those specific tests provide to the overall project.

tip
Tip

When considering to ignore or only run a particular test within a given browser, assess the true need for the test to run on multiple browsers.

You can specify a browser to run or exclude by passing a matcher to the suite or test within the test configuration. The browser option accepts the same arguments as Cypress.isBrowser().

// Run the test if Cypress is running
// using the built-in Electron browser
it('has access to clipboard', { browser: 'electron' }, () => {
...
})

// Run the test if Cypress is run via Firefox
it('Download extension in Firefox', { browser: 'firefox' }, () => {
cy.get('#dl-extension')
.should('contain', 'Download Firefox Extension')
})

// Run happy path tests if Cypress is run via Firefox
describe('happy path suite', { browser: 'firefox' }, () => {
it('...')
it('...')
it('...')
})

// Ignore test if Cypress is running via Chrome
// This test is not recorded to Cypress Cloud
it('Show warning outside Chrome', { browser: '!chrome' }, () => {
cy.get('.browser-warning').should(
'contain',
'For optimal viewing, use Chrome browser'
)
})

See also​