Branch Review
What you'll learn​
- How to compare test results between branches in Cypress Cloud
- How Branch Review surfaces the impact of Pull Requests on your test suite
- How to compare any two runs, if not using a Pull Request workflow
- Best practices for grouping test runs
Cypress Branch Review allows you to quickly identify the different impacts a Pull Request might have on your test suite in a single view. Compare which tests are failing, flaky, pending, added, or modified between the source and base branches. If enabled on your account, UI Coverage or Cypress Accessibility changes between runs will also appear here.
Branch Review is useful even if you do not have a Pull Request workflow, because it allows you to compare any two runs of your choice. These might might represent different builds of the application, changes in test code, or results from nightly runs. This page will use the example of a Pull Request made using the GitHub integration, but this integration is not required in order to use Branch Review.
A common scenario throughout the software development lifecycle (SDLC) is an engineer's feature branch that will be merged into the repo's develop or main branch. Here's how this scenario appears in Branch Review:
Without Branch Review, pinpointing changes in your test suite's results requires a manual side-by-side comparison between your newly-introduced branch's test runs and your base branch's test runs. This sub-optimal workflow often fails to answer the fundamental questions, what changed and why? You might be left wondering if the same tests are flaky between branches, when new failures were introduced, or if you added sufficient test coverage to your new branch.
Getting Started​
The key to the Branch Review area is a comparison between a Base run and a Changed run. The Changed run is the main subject of the comparison, often associated with an incoming Pull Request. The Base run is the starting point for comparison.
Accessing Branch Review​
There are a number of ways to get to Branch Review depending on where you are looking at your Cypress results. In all cases, once you've picked Changed run, you can adjust the automatically-selected Base run to any branch or run in the project to dial in the comparison you are most interested in.
From the project list​
The project card shows three active branches for the project. You can click through to any of these to enter Branch Review focused on that branch.
From the main navigation​
When a project is open, select "Branches" in the main navigation to see a full, filterable list of available branches and choose one to set as the source of the Changed run.
From the run overview page​
Click the branch name associated with the run. This will take you to Branch Review with that run pre-selected as the Changed run. This is a great way to investigate a specific run with an unexpected outcome in the tests.
From a Pull Request​
Click the "View all changes introduced in this branch" link at the bottom of the Test Results table. You will enter branch review with the current PR's branch and latest run pre-selected as the Changed run.
Integrating with Pull Requests ​
Pull Requests in Branch Review work by leveraging the Cypress Cloud GitHub integration to query the GitHub API for branches with Pull Requests. Your project will first need to be connected to a GitHub repository. If you haven't already, follow the steps in Install the Cypress GitHub app to connect your project to GitHub.
When Pull Requests are connected, you will be able to select runs by the associated PR:
Note: If you do not see PR # tags associated with a branch, you may need to pass PR numbers to runs as environment variables. This helps ensure Cypress Cloud can map PR data correctly.
Latest Runs​
If the selected branch has a PR associated and a PR # passed via CI, there is an additional callout on the top of the overview tab of the Latest runs page which directs to the Branch Review.
Branch Details​
Review Header​
The header includes the Git commit message, PR selector, PR status, and base and feature branch labels with the Cypress Cloud test run ID number (#).
If there are multiple Pull Requests open for the same branch, you can select the Pull Request you want to review from the dropdown to the right of the commit message.
Clicking the branch label will link you directly to the appropriate run overview and hovering on these elements exposes additional run meta data.
Review Test Status​
Note: Recording exactly one run for every commit is the best way to ensure complete and accurate comparisons. See our best practices on grouping multiple cypress run calls under one run.
When on the review screen, you will see the Failures, Flaky, Pending, Added, and Modified tabs. Each tab will show you the specs that fall into that category. You can click on a test to view the test details.
Review Test Status
- Details of which branches and runs are compared. If a Pull Request is detected, Pull Request details will be here.
- Links to key differences in test information like new failures and flaky tests. Note that all numbers and scores refer to the Changed run.
- UI Coverage changes.
- Cypress Accessibility changes.
- See what has changed in the specific status you have opened, for example:
total number introduced in your branch
total number decreased or resolved in your branch
total count, for example, 3 new and 1 existing
- The test state (failed, flaky, pending) is also indicated at the spec level
- new = the state was not previously captured, but now is captured
(newly pending tests can imply anit.skip()was not removed) - existing = the state was previously captured and now is still captured
- resolved = the state was previously captured but now is no longer captured
- new = the state was not previously captured, but now is captured
Review Test Comparison​
Navigating into the test detail view reveals a side-by-side comparison of the test results on both branches, as well as, artifacts so you can compare before vs. after your PR. This can help narrow the focus on changes introduced or resolved on either branch and elevate the source of flake within tests throughout different attempts. Review the test definition panel for diff snapshots to help quickly determine changes in test code.
Review Test Comparison
-
Base branch results
-
Feature branch results
-
Test Replay and artifacts
-
Test attempts (descending order)
-
Code change diffs
Troubleshooting​
Note on Available Data for Pull Requests​
Branch Review is a powerful tool to compare two branches with recorded runs to Cypress Cloud. There are factors that can impact what is available to review between a feature and base branch. For example, whether or not a branch at a particular commit has a run recorded to Cypress Cloud will affect what is displayed.
The following captures this scenario across both branches:
| Base Branch | Feature Branch | What Branch Review Displays |
|---|---|---|
| has run | has run | Comparison data using both found runs |
| has run | no run | Comparison data using last run on feature branch |
| no run | has run | Non-comparison data with found feature run |
| no run | no run | Non-comparison data using last run on feature branch |
If the required runs for comparison are not available, you can use the manual selection dropdowns to compare two suitable runs, if they exist in Cypress Cloud.
Best Practices​
Grouping Test Runs​
Cypress Cloud allows for grouping recorded tests together under a single run. This means multiple cypress run calls can be labeled and associated to a single run in Cypress Cloud.
Recording multiple test runs per commit without grouping will cause issues, as Branch Review relies on the latest run. Therefore, recording one run per commit and utilizing grouping is essential to improving effectiveness and ensure an accurate comparison between branches.
Missing Runs in Pull Request Comparisons​
If a run is missing on either the merge base or feature commit of the PR, a banner is displayed specifying which commit is missing the run with a link to it on GitHub.
See Also​
- Read about multi-repo implementation best practices in Cypress Cloud. Even if your test code and application code are not colocated, you maybe be able to connect your test runs to specific commits and pull requests in your application.