Branch Review

info

What you'll learn

• How Branch Review surfaces the impact of pull requests on your test suite
• How to compare test results between branches in Cypress Cloud
• Best practices for grouping test runs

Cypress Branch Review is designed to elevate your pull request review workflow. It allows you to quickly identify the impact a pull request might have on your test suite in a single view. Compare which tests are failing, flaky, pending, added, or modified between the source and base branches and prevent the merging of low-quality code.

A common scenario throughout the software development lifecycle (SDLC) is an engineer's feature branch that will be merged into the repo's develop or main branch.

Previously, pinpointing changes in your test suite's results required a manual side-by-side comparison between your newly-introduced branch's test runs and your base branch's test runs. This sub-optimal workflow often fails to answer the fundamental questions, what changed and why? You might be left wondering if the same tests are flaky between branches, when new failures were introduced, or if you added sufficient test coverage to your new branch.

info

Cypress Branch Review is currently only available for GitHub integrated projects. We are working on adding support for GitLab and Bitbucket soon.

Getting Started

Branch Review works by leveraging the Cypress Cloud GitHub integration to query the GitHub API for branches with pull requests. Your project will first need to be connected to a GitHub repository. If you haven't already, follow the steps in Install the Cypress GitHub app to connect your project to GitHub.

Accessing Branch Review

To access Branch Review, navigate to the Branches tab in the left sidebar and select the branch you want to review. In order to see a comparison, the branch you select must have a pull request.

Active branches, shown with available PR # tags

If you do not see PR # tags associated with a branch, you may need to pass PR numbers to runs as environment variables. This helps ensure Cypress Cloud can map PR data correctly.

Latest Runs

If the selected branch has a PR associated and a PR # passed via CI, there is an additional callout on the top of the overview tab of the Latest runs page which directs to the Branch Review.

Branch Details

View Options

"Overview" is the default view of a branch for all projects.

note

Branch Overview

1. Callouts for the latest available Branch Review or a reminder to create a pull request in order to compare test run results.

2. Click "Review changes" for the full Branch Review.

3. Click any test status to reveal those changes.

4. Latest runs of the current branch for contextual guidance on the state of its test runs.

"Review" provides the detailed Branch Review insights for the branch's available pull requests, with the latest pull request selected by default.

"Runs" provides a list of all runs that are attributed to the current branch.

Review Header

The header includes the Git commit message, PR selector, PR status, and base and feature branch labels with the Cypress Cloud test run ID number (#).

If there are multiple pull requests open for the same branch, you can select the pull request you want to review from the dropdown to the right of the commit message.

Clicking the branch label will link you directly to the appropriate run overview and hovering on these elements exposes additional run meta data.

Review Test Status

When on the review screen, you will see the Failures, Flaky, Pending, Added, and Modified tabs. Each tab will show you the specs that fall into that category. You can click on a test to view the test details.

info

If a run is missing on either the merge base or feature commit of the PR, a banner is displayed specifying which commit is missing the run with a link to it on GitHub.

info

Recording exactly one run for every commit as the best/only way to ensure complete and accurate comparisons. See our best practices on grouping multiple cypress run calls under one run.

note

Review Test Status

1. Review your PR information
2. Get a glance of changes in your branch across key statuses
3. See what has changed in a specific status, for example:
   • total number introduced in your branch
   • total number decreased or resolved in your branch
   • total count, for example, 3 new and 1 existing
4. The test state (failed, flaky, pending) is also indicated at the spec level
   • new = the state was not previously captured, but now is captured
     (newly pending tests can imply an it.skip() was not removed)
   • existing = the state was previously captured and now is still captured
   • resolved = the state was previously captured but now is no longer captured

Review Test Comparison

Navigating into the test detail view reveals a side-by-side comparison of the test results on both branches, as well as, artifacts so you can compare before vs. after your PR. This can help narrow the focus on changes introduced or resolved on either branch and elevate the source of flake within tests throughout different attempts. Review the test definition panel for diff snapshots to help quickly determine changes in test code.

note

Review Test Comparison

1. Base branch results

2. Feature branch results

3. Test Replay and artifacts

4. Test attempts (descending order)

5. Code change diffs

GitHub Pull Request Comments

The Cypress Cloud GitHub Integration offers detailed run information via PR comments. Follow the "View all changes introduced in this branch" link to load the Branch Review between the feature and base branches.

Troubleshooting

Note on Available Data

Branch Review is a powerful tool to compare two branches with recorded runs to Cypress Cloud. There are factors that can impact what is available to review between a feature and base branch. For example, whether or not a branch at a particular commit has a run recorded to Cypress Cloud will affect what is displayed.

The following captures this scenario across both branches:

Base Branch	Feature Branch	What Branch Review Displays
has run	has run	Comparison data using both found runs
has run	no run	Comparison data using last run on feature branch
no run	has run	Non-comparison data with found feature run
no run	no run	Non-comparison data using last run on feature branch

Best Practices

Grouping Test Runs

Cypress Cloud allows for grouping recorded tests together under a single run. This means multiple cypress run calls can be labeled and associated to a single run in Cypress Cloud.

Recording multiple test runs per commit without grouping will cause issues, as Branch Review relies on the latest run. Therefore, recording one run per commit and utilizing grouping is essential to improving effectiveness and ensure an accurate comparison between branches.

See Also

• Read about multi-repo implementation best practices in Cypress Cloud.