Branch Review
What you'll learn​
- What Branch Review is and why you should use it
- What test insights are available to review between branches
- How Branch Review works and what might impact available data
Overview​
Cypress Branch Review is designed to elevate your pull request review workflow. It allows you to quickly identify the impact a pull request might have on your test suite in a single view. Compare which tests are failing, flaky, pending, added, or modified between the source and base branches and prevent the merging of low-quality code.
A common scenario throughout the software development lifecycle (SDLC) is an engineer's feature branch that will be merged into the repo's develop or main branch.
Previously, pinpointing changes in your test suite's results required a manual side-by-side comparison between your newly-introduced branch's test runs and your base branch's test runs. This sub-optimal workflow often fails to answer the fundamental questions, what changed and why? You might be left wondering if the same tests are flaky between branches, when new failures were introduced, or if you added sufficient test coverage to your new branch.
Cypress Branch Review is currently only available for GitHub integrated projects. We are working on adding support for GitLab and Bitbucket soon.
Getting Started​
Branch Review works by leveraging the Cypress Cloud GitHub integration to query the GitHub API for branches with pull requests. Your project will first need to be connected to a GitHub repository. If you haven't already, follow the steps in Install the Cypress GitHub app to connect your project to GitHub.
Accessing Branch Review​
To access Branch Review, navigate to the Branches tab in the left sidebar and select the branch you want to review. In order to see a comparison, the branch you select must have a pull request.
Active branches, shown with available PR # tags
If you do not see PR # tags associated with a branch, you may need to pass PR numbers to runs as environment variables. This helps ensure Cypress Cloud can map PR data correctly.
Latest Runs​
If the selected branch has a PR associated and a PR # passed via CI, there is an additional callout on the top of the overview tab of the Latest runs page which directs to the Branch Review.
Branch Details​
View Options​
"Overview" is the default view of a branch for all projects.
Branch Overview
Callouts for the latest available Branch Review or a reminder to create a pull request in order to compare test run results.
Click "Review changes" for the full Branch Review.
Click any test status to reveal those changes.
Latest runs of the current branch for contextual guidance on the state of its test runs.
"Review" provides the detailed Branch Review insights for the branch's available pull requests, with the latest pull request selected by default.
"Runs" provides a list of all runs that are attributed to the current branch.
Review Header​
The header includes the Git commit message, PR selector, PR status, and base and feature branch labels with the Cypress Cloud test run ID number (#).
If there are multiple pull requests open for the same branch, you can select the pull request you want to review from the dropdown to the right of the commit message.
Clicking the branch label will link you directly to the appropriate run overview and hovering on these elements exposes additional run meta data.
Review Test Status​
When on the review screen, you will see the Failures, Flaky, Pending, Added, and Modified tabs. Each tab will show you the specs that fall into that category. You can click on a test to view the test details.
If a run is missing on either the merge base or feature commit of the PR, a banner is displayed specifying which commit is missing the run with a link to it on GitHub.
Recording exactly one run for every commit as the best/only way to ensure complete and accurate comparisons. See our best practices on grouping multiple cypress run calls under one run.
Review Test Status
- Review your PR information
- Get a glance of changes in your branch across key statuses
- See what has changed in a specific status, for example:
total number introduced in your branch
total number decreased or resolved in your branch
total count, for example, 3 new and 1 existing
- The test state (failed, flaky, pending) is also indicated at the spec level
- new = the state was not previously captured, but now is captured
(newly pending tests can imply anit.skip()was not removed) - existing = the state was previously captured and now is still captured
- resolved = the state was previously captured but now is no longer captured
- new = the state was not previously captured, but now is captured
Review Test Comparison​
Navigating into the test detail view reveals a side-by-side comparison of the test results on both branches, as well as, artifacts so you can compare before vs. after your PR. This can help narrow the focus on changes introduced or resolved on either branch and elevate the source of flake within tests throughout different attempts. Review the test definition panel for diff snapshots to help quickly determine changes in test code.
Review Test Comparison
Base branch results
Feature branch results
Test Replay and artifacts
Test attempts (decending order)
Code change diffs
GitHub Pull Request Comments​
The Cypress Cloud GitHub Integration offers detailed test information via PR comments. Follow the "Review all test suite changes" link to load the Branch Review between the feature and base branches.
Troubleshooting​
Note on Available Data​
Branch Review is a powerful tool to compare two branches with recorded runs to Cypress Cloud. There are factors that can impact what is available to review between a feature and base branch. For example, whether or not a branch at a particular commit has a run recorded to Cypress Cloud will affect what is displayed.
The following captures this scenario across both branches:
| Base Branch | Feature Branch | What Branch Review Displays |
|---|---|---|
| has run | has run | Comparison data using both found runs |
| has run | no run | Comparison data using last run on feature branch |
| no run | has run | Non-comparison data with found feature run |
| no run | no run | Non-comparison data using last run on feature branch |
Best Practices​
Grouping Test Runs​
Cypress Cloud allows for grouping recorded tests together under a single run. This means multiple cypress run calls can be labeled and associated to a single run in Cypress Cloud.
Recording multiple test runs per commit without grouping will cause issues, as Branch Review relies on the latest run. Therefore, recording one run per commit and utilizing grouping is essential to improving effectiveness and ensure an accurate comparison between branches.
See Also​
- Read about multi-repo implementation best practices in Cypress Cloud.