The Test Runner is a free, downloadable and open source (MIT license) application. This is always free to use. Our Dashboard Service is a web application that offers a variety of billing plans (including a free, open source plan) for when you want to record your test runs in CI.
Please see our Pricing Page for more details.
Currently our users use Cypress to control the viewport with the
cy.viewport() command to test responsive, mobile views in a website or web application.
The Cypress Test Runner is a hybrid application/framework/service all rolled into one. It takes a little bit of other testing tools, brings them together and improves on them.
beforeEach methods. Cypress isn’t different from Mocha, it actually uses Mocha under the hood. All of your tests will be written on top of Mocha’s
reporter for your tests failing / passing. It runs from the command line.
Cypress essentially replaces Karma because it does all of this already and much more.
Ruby specific tool that allows you to write integration tests for your web application is Capybara. In the Rails world, this is the go-to tool for testing your application. It uses Sauce Labs (or another headless driver) to interact with browsers. Its API consists of commands that query for DOM elements, perform user actions, navigate around, etc.
Cypress essentially replaces Capybara because it does all of these things and much more. The difference is that instead of testing your application in a GUI-less console, you would see your application at all times. You’d never have to take a screenshot to debug because all commands instantly provide you the state of your application while they run. Upon any command failing, you’ll get a human-readable error explaining why it failed. There’s no “guessing” when debugging.
Using Protractor provides a nice Promise-based interface on top of Selenium, which makes it easy to deal with asynchronous code. Protractor comes with all of the features of Capybara and essentially suffers from the same problems.
Cypress replaces Protractor because it does all of these things and much more. One major difference is that Cypress enables you to write your unit tests and integration tests in the same tool, as opposed to splitting up this work across both Karma and Protractor.
Also, Protractor is very much focused on
Sauce Labs also has a
manual testing mode, where you can remotely control browsers in the cloud as if they were installed on your machine.
Ultimately Sauce Labs and Cypress offer very different value propositions. Sauce Labs doesn’t help you write your tests, it takes your existing tests and runs them across different browsers and aggregates the results for you.
Cypress on the other hand helps you write your tests. You would use Cypress every day, building and testing your application, and then use Sauce Labs to ensure your application works on every browser.
A note about Cypress and Sauce Labs
Cypress’ API is written to be completely compatible for integration with Sauce Labs. It is our goal to offer full integration with Sauce Labs in the future, however, complete integration is not yet available.
Any and all. Ruby, Node, C#, PHP - none of that matters. Cypress tests anything that runs in the context of a browser. It is back end, front end, language and framework agnostic.
You can read about our currently available browsers here.
Cypress works in any CI provider.
No. But if you’re wanting to test parts of your application that are not easily testable, you’ll want to refactor those situations (as you would for any testing).
No. In fact Cypress’ architecture is very different from Selenium in a few critical ways:
- Cypress runs in the context of the browser. With Cypress it’s easier to inspect what is running in the browser, but harder to talk to the outside world. In Selenium it’s the exact opposite. Selenium runs outside of the browser where your application is running. Although Cypress is adding more commands every day that give you access to the outside world - like
Yes, technically; it’s sandboxed and has to follow the same rules as every other browser. That’s actually a good thing because it doesn’t require a browser extension, and it naturally will work across all browsers (which enables cross-browser testing).
All of these technologies together are coordinated and enable Cypress to work, which extends its capabilities far outside of the browser sandbox. Without these, Cypress would not work at all. For the vast majority of your web development, Cypress will work just fine, and already does work.
If you’re using some complex thumb-print, retinal-scan, time-based, key-changing, microphone, audial, decoding mechanism to log in your users, then no, Cypress won’t work with that. But seriously, Cypress is a development tool, which makes it easy to test your web applications. If your application is doing 100x things to make it extremely difficult to access, Cypress won’t magically make it any easier.
Because Cypress is a development tool, you can always make your application more accessible while in your development environment. If you want, disable complex steps in your authentication systems while you’re in your testing environment. After all, that’s why we have different environments! Normally you already have a development environment, a testing environment, a staging environment, and a production environment. So expose the parts of your system you want accessible in each appropriate environment.
In doing so, Cypress may not be able to give you 100% coverage without you changing anything, but that’s okay. Just use different tools to test the crazier, less accessible parts of your application, and let Cypress test the other 99%.
Just remember, Cypress won’t make a non-testable application suddenly testable. It’s on your shoulders to architect your code in an accessible manner.
Yes. Cypress works on anything rendered to a browser.
No. There are already lots of tools to do that. Using Cypress to test against a 3rd party application is not its intended use. It may work but will defeat the purpose of why it was created. You use Cypress while you develop your application, it helps you write your tests.
There is nothing currently built into Cypress to do this. Adding code coverage around end-to-end tests is much harder than unit tests and it may not be feasible to do in a generic way. You can read in more detail about code coverage here. You may find some other coverage utilities useful when writing end-to-end tests like:
Unit tests are not something we are really trying to solve right now. Most of the
cy API commands are useless in unit tests. The biggest benefit of writing unit tests in Cypress is that they run in a browser, which has debugger support built in.
We have internally experimented at doing DOM based component unit testing in Cypress - and that has the possibility of being an excellent “sweet spot” for unit tests. You’d get full DOM support, screenshot support, snapshot testing, and you could then use other
cy commands (if need be). But as I mentioned this isn’t something we’re actively pushing, it just remains a thing that’s possible if we wanted to go down that route.
With that said - we actually believe the best form of testing in Cypress is a combination of a “unit test” mixed with an “e2e test”. We don’t believe in a “hands off” approach. We want you to modify the state of your application, take shortcuts as much as possible (because you have native access to all objects including your app). In other words, we want you to think in unit tests while you write integration tests.
We believe unit tests and end-to-end tests have differences that should guide your choice.
|Unit tests||End-to-end tests|
|Focus on code||Focus on the features|
|Should be kept short||Can be long|
|Examine the returned result of actions||Examine side effect of actions: DOM, storage, network, file system, database|
|Important to developer workflow||Important to end user’s workflow|
In addition to the above differences, below are a few rules of thumb to decide when to write a unit test and when to write an end-to-end test.
- If the code you are trying to test is called from other code, use a unit test.
- If the code is going be called from the external system, like a browser, use an end-to-end test.
- If a unit test requires a lot of mocking and you have to bring tools like
sinon.jsto simulate a real world environment, you may want to rewrite it as an end-to-end test.
- If an end-to-end test does not go through the browser and instead calls the code directly, you probably want to rewite it as a unit test
Finally, unit and end-to-end tests are not that different and have common features. Good tests:
- Focus on and test just one thing.
- Are flake-free and do not fail randomly.
- Give you confidence to refactor code and add new features.
- Are easy to run both locally and on a continuous integration server.
Certainly, unit and end-to-end tests are NOT in opposition to each other and are complementary tools in your toolbox.