Skip to main content

Trade-offs

End-to-End Only

Cypress automates the browser with its own unique architecture. While this unlocks the power to do things you will not find anywhere else, there are specific trade-offs that are made. There is no free lunch!

In this guide we will lay out what some of the trade-offs are - and specifically how you can work around them.

While at first it may seem like these are strict limitations in Cypress - we think you will soon realize that many of these boundaries are actually good to have. In a sense they prevent you from writing bad, slow, or flaky tests.

Permanent trade-offs:​

Temporary trade-offs:​

We have open issues where you can find a full list of things Cypress will eventually address, we wanted to highlight some of the more important temporary restrictions that Cypress will eventually address. PRs are welcome ;-)

Many of these issues are currently being worked on or are on our Roadmap.

Permanent trade-offs​

Automation restrictions​

Cypress is a specialized tool that does one thing really well: end-to-end testing web applications while they are under development. You should not use Cypress for things it is not designed for such as:

  • Indexing the web
  • Spidering links
  • Performance testing
  • Scripting 3rd party sites

There are other excellent tools that are optimized for doing each item listed above.

The sweet spot of Cypress is to be used as a tool to test your own application as you build it. It is built for developers and QA engineers, not manual testers or exploratory testing.

Inside the browser​

In case you missed it before - Cypress tests run inside of the browser! This means we can do things nobody else can. There is no object serialization or JSON wire protocols. You have real, native access to everything in your application under test. It is impossible for Cypress to 'miss' elements and it always knows the moment your application fires any kind of event.

But what this also means is that your test code is being evaluated inside the browser. Test code is not evaluated in Node, or any other server side language. The only language we will ever support is the language of the web: JavaScript.

This trade-off means it makes it a little bit harder to communicate with the back end - like your server or database. You will not be able to connect or import those server-side libraries or modules directly. Although you can require node_modules which can be used in the browser. Additionally, you have the ability to use Node to import or talk directly to your back end scripts using our Plugins API or cy.task().

To talk to your database or server you need to use the cy.exec(), cy.task(), or cy.request() commands. That means you will need to expose a way to seed and setup your database. This really is not that hard, but it might take a bit more elbow grease than other testing tools written in your back end language.

The trade-off here is that doing everything in the browser (basically all of your tests) is a much better experience in Cypress. But doing things outside of the browser may take a little extra work.

In the future we do have plans to release back end adapters for other languages.

Multiple browsers open at the same time​

Cypress does not support controlling more than 1 open browser at a time.

With that said, except in the most unusual and rare circumstances, you can still test most application behavior without opening multiple browsers at the same time.

You may ask about this functionality like this:

I'm trying to test a chat application. Can I run more than one browser at a time with Cypress?

Whether you are testing a chat application or anything else - what you are really asking about is testing collaboration. But, you don't need to recreate the entire environment in order to test collaboration with 100% coverage.

Doing it this way can be faster, more accurate, and more scalable.

While outside the scope of this article, you could test a chat application using the following principles. Each one will incrementally introduce more collaboration:

1. Use only the browser:​

    ↓
← browser →
    ↑

Avoid the server, invoke your JavaScript callbacks manually thereby simulating what happens when "notifications come in", or "users leave the chat" purely in the browser.

You can stub everything and simulate every single scenario. Chat messages, offline messages, connections, reconnections, disconnections, group chat, etc. Everything that happens inside of the browser can be fully tested. Requests leaving the browser could also be stubbed and you could assert that the request bodies were correct.

2. Stub the other connection:​

server → browser
            ↓
server ← browser
  ↓
(other connections stubbed)
  ↓
server → browser

Use your server to receive messages from the browser, and simulate "the other participant" by sending messages as that participant. This is certainly application specific, but generally you could insert records into the database or do whatever it takes for your server to act as if a message of one client needs to be sent back to the browser.

Typically this pattern enables you to avoid making a secondary WebSocket connection and yet still fulfills the bidirectional browser and server contract. This means you could also test edge cases (disconnections, etc) without actually handling real connections.

3: Introduce another connection:​

server → browser
            ↓
server ← browser
  ↓
server → other connection
            ↓
server ← other connection
  ↓
server → browser

To do this - you would need a background process outside of the browser to make the underlying WebSocket connection that you can then communicate with and control.

You can do this in many ways and here is an example of using an HTTP server to act as the client and exposing a REST interface that enables us to control it.

// Cypress tests

// tell the http server at 8081 to connect to 8080
cy.request('http://localhost:8081/connect?url=http://localhost:8080')

// tell the http server at 8081 to send a message
cy.request('http://localhost:8081/message?m=hello')

// tell the http server at 8081 to disconnect
cy.request('http://localhost:8081/disconnect')

And the HTTP server code would look something like this...

const client = require('socket.io:client')
const express = require('express')

const app = express()

let socket

app.get('/connect', (req, res) => {
  const url = req.query.url

  socket = client(url)

  socket.on('connect', () => {
    res.sendStatus(200)
  })
})

app.get('/message', (req, res) => {
  const msg = req.query.m

  socket.send(msg, () => {
    res.sendStatus(200)
  })
})

app.get('/disconnect', (req, res) => {
  socket.on('disconnect', () => {
    res.sendStatus(200)
  })

  socket.disconnect()
})

app.listen(8081, () => {})

This avoids ever needing a second open browser, but still gives you an end-to-end test that provides 100% confidence that the two clients can communicate with each other.

Contents