Skip to main content
UI CoveragePremium Solution

Block pull requests and set policies

Once you've used UI Coverage to find the untested parts of your app, the next step is keeping new gaps from creeping back in. This guide turns your UI Coverage report into a merge gate, so a pull request that adds an untested button, form, or page can't merge without a deliberate decision.

UI Coverage reports are generated server-side in Cypress Cloud from Test Replay data, so they add no overhead to your test runs, and nothing in your Cypress pipeline fails on its own when coverage drops. Enforcement is entirely opt-in: you choose a policy and apply it in your CI job with the Results API. With it you can:

  • Fail a build or block a pull request when overall coverage falls below a threshold, or when a critical view regresses.
  • Block only on new gaps by comparing each run against a stored baseline, so you can pay down existing coverage debt over time instead of all at once.
  • Post the result where reviewers work, linking straight to the report on the pull request.

Choose a policy​

The Results API gives you the run's coverage numbers; how you gate on them is the policy. Two models cover most teams, and you can combine them.

PolicyBlocks when…Best for
Coverage thresholdOverall or per-view coverage drops below a fixed percentageA codebase whose coverage is already where you want it, or a critical flow you hold to a high bar
New gaps vs. a baselineA run introduces untested elements that weren't there beforeAn existing app with known coverage debt you want to stop growing

A fixed threshold is simple but blunt: if your app sits at 62% coverage, a threshold of 80% blocks every pull request until you've paid down all the existing debt. A baseline compares each run against a known-good state and fails only on newly introduced gaps, which lets you hold the line today and improve incrementally. Most teams with an established suite start here.

Before you begin​

This guide assumes you've completed the Results API setup: your CI install step adds the @cypress/extract-cloud-results module, and a verification script runs after cypress run --record in the same CI build. A run also needs Test Replay enabled and must have been recorded within the last 7 days for the helper to find it. See the Results API FAQ for the full prerequisite list.

Every example below is a script that calls getUICoverageResults() and, when your policy isn't met, exits with a non-zero status to fail the CI step. See Wire the check into CI for how that failed step becomes a required status check that blocks the merge.

Enforce a coverage threshold​

The simplest policy fails the build when overall coverage drops below a floor, and holds critical flows like login and checkout to a higher standard:

scripts/verifyUICoverageResults.js
const { getUICoverageResults } = require('@cypress/extract-cloud-results')

getUICoverageResults()
  .then((results) => {
    const { summary, views } = results

    // Fail the build if overall coverage is below 80%.
    if (summary.coverage < 80) {
      throw new Error(
        `Project coverage is ${summary.coverage}%, below the minimum threshold of 80%.`
      )
    }

    // Hold critical flows to a higher standard.
    const criticalViews = [/login/, /checkout/]

    views.forEach((view) => {
      if (
        criticalViews.some((pattern) => pattern.test(view.displayName)) &&
        view.coverage < 95
      ) {
        throw new Error(
          `Critical view "${view.displayName}" coverage is ${view.coverage}%, below the required 95%. See: ${view.uiCoverageReportUrl}`
        )
      }
    })

    console.log('UI Coverage meets all thresholds.')
  })
  .catch((error) => {
    console.error(error.message)
    process.exit(1)
  })

The Results API examples cover more threshold variations, including reporting the score on the pull request, failing on untested navigation links, and ratcheting the floor up as coverage improves.

Block only on new gaps with a baseline​

A threshold treats all untested elements the same, whether they've been untested for a year or arrived in the pull request under review. Comparing a run against a stored baseline lets you ignore existing gaps and fail only when a change introduces new untested elements. This is what makes UI Coverage enforceable on a real codebase: you stop the debt from growing today, and pay it down on your own schedule without blocking every merge.

Why compare untested-element counts instead of the coverage score?​

The coverage score is the right number for tracking direction over time, but a poor signal for a per-run gate: adding a single test can reveal a whole page of previously unseen elements and lower the score even though your coverage improved.

The untested element and link counts per view provide a more direct signal for "did this change add gaps?" If a view's untested count rises above its baseline, or a brand-new view arrives carrying untested elements, the change introduced gaps, regardless of what the percentage did. That's the number the script below compares.

Baseline structure​

The baseline is a small JSON file you commit alongside your code. It records, for a known-good run, how many untested items each view had:

  • runNumber: the run the baseline was captured from, for reference.
  • runUrl: a link back to that run in Cypress Cloud.
  • views: a map of each view's displayName to its combined untested count (untestedElementsCount plus untestedLinksCount).
ui-coverage-baseline.json
{
  "runNumber": 68086,
  "runUrl": "https://cloud.cypress.io/projects/ypt4pf/runs/68086",
  "views": {
    "/": 0,
    "/login": 2,
    "/products/*": 4,
    "/checkout": 1
  }
}

Create your first baseline​

You need a known-good run to anchor the baseline against. Pick a recent run whose coverage reflects the state you want to hold the line at, typically a run on your base branch, then capture its per-view untested counts one of two ways.

From the comparison script. The script below is the same one you wire into CI. Run it once with no committed baseline and it writes the current run's numbers to ui-coverage-baseline.next.json without failing the build. Review that file, then commit it as ui-coverage-baseline.json so every later run compares against it. A CI runner is discarded after the job, so to capture that first candidate either run the script locally by replaying a recent run's CI context (see Required CI environment variables), or upload ui-coverage-baseline.next.json as a build artifact from the CI run.

View the comparison script

This script reads the committed baseline, fails the build when any view has more untested elements than its baseline (or when a new view ships untested elements), and writes the current run's numbers to a separate candidate file so you can promote them when you're ready to accept a new state. Running it the first time with no baseline just writes the candidate for you.

scripts/compareUICoverageBaseline.js
const fs = require('fs')
const { getUICoverageResults } = require('@cypress/extract-cloud-results')

const BASELINE_FILE = 'ui-coverage-baseline.json'
// Written on every run; promote it to BASELINE_FILE when you accept a new state.
const CANDIDATE_FILE = 'ui-coverage-baseline.next.json'

const hasBaseline = fs.existsSync(BASELINE_FILE)
const baseline = hasBaseline
  ? JSON.parse(fs.readFileSync(BASELINE_FILE, 'utf8'))
  : { runNumber: null, runUrl: null, views: {} }

getUICoverageResults()
  .then((results) => {
    const { runNumber, runUrl, uiCoverageReportUrl, summary, views } = results

    // A cancelled or incomplete run can under-report views, which would look
    // like a regression. Skip enforcement instead of failing for the wrong reason.
    if (summary.isPartialReport) {
      console.warn('Report is partial. Skipping baseline comparison.')
      return
    }

    const newGaps = []
    const improvements = []

    views.forEach((view) => {
      // Untested links are counted separately from untested elements.
      // Sum both so a newly unreachable page can't slip through.
      const current = view.untestedElementsCount + view.untestedLinksCount
      const previous = baseline.views[view.displayName]

      if (previous === undefined) {
        // A view with no baseline entry is new. Only flag it if it ships gaps.
        if (current > 0) {
          newGaps.push({
            view: view.displayName,
            previous: 0,
            current,
            url: view.uiCoverageReportUrl,
          })
        }
      } else if (current > previous) {
        newGaps.push({
          view: view.displayName,
          previous,
          current,
          url: view.uiCoverageReportUrl,
        })
      } else if (current < previous) {
        improvements.push({ view: view.displayName, previous, current })
      }
    })

    // Save the current run's numbers as the next candidate baseline.
    fs.writeFileSync(
      CANDIDATE_FILE,
      JSON.stringify(
        {
          runNumber,
          runUrl,
          views: Object.fromEntries(
            views.map((view) => [
              view.displayName,
              view.untestedElementsCount + view.untestedLinksCount,
            ])
          ),
        },
        null,
        2
      )
    )

    // On the first run there's no committed baseline to compare against, so seed
    // the candidate and exit cleanly. Commit it to start gating later runs.
    if (!hasBaseline) {
      console.log(
        `No committed baseline yet. Wrote ${CANDIDATE_FILE}; commit it as ${BASELINE_FILE} to start gating.`
      )
      return
    }

    improvements.forEach(({ view, previous, current }) =>
      console.log(`✅ ${view}: untested elements ${previous} → ${current}`)
    )

    if (newGaps.length > 0) {
      console.error('\n❌ New untested elements were introduced:\n')
      newGaps.forEach(({ view, previous, current, url }) =>
        console.error(`   ${view}: ${previous} → ${current} untested (${url})`)
      )
      console.error(
        '\nAdd tests for the new elements, or exclude them with App Quality configuration.'
      )
      console.error(`Full report: ${uiCoverageReportUrl}`)
      process.exit(1)
    }

    console.log('\n✅ No new untested elements against the baseline.')
  })
  .catch((error) => {
    console.error(`Could not compare UI Coverage results: ${error.message}`)
    process.exit(1)
  })

From your editor with an AI assistant. If you use an agent that supports Cypress Cloud MCP, you can build the baseline without wiring up CI first by having it pull the report and write the file:

Build a UI Coverage baseline with your AI assistant

Pulls the UI Coverage report from Cypress Cloud and writes a baseline file you can commit.

Using Cypress Cloud, pull the UI Coverage report for the latest run on main. Write a ui-coverage-baseline.json file with runNumber, runUrl, and a views object that maps each view's display name to its untested element count.

Review the generated file the same way, then commit it. See Work with AI agents for more on using Cloud MCP with UI Coverage.

Key concepts​

New untested elements​

A view has new untested elements when its combined untested count (untestedElementsCount plus untestedLinksCount) is higher than the baseline, or when a view absent from the baseline appears with untested elements. This is a coverage regression: the change added interactive elements or link destinations no test reaches. The script fails the build so the pull request can't merge until you cover them or exclude them.

Resolved gaps​

A view has resolved gaps when its combined untested count is lower than the baseline. This is an improvement: a change added coverage. The script reports these but doesn't fail the build, so progress is visible without blocking anyone.

View-level comparison​

Untested elements are compared per view (a URL pattern for end-to-end tests, or a spec file for component tests), so the failure message points at the exact page or component where the new gap appeared, rather than a single project-wide number. Views are matched by displayName, so renaming a route or changing how views are grouped changes the key: an unchanged page can then look like a new view and trip the gate. Re-promote the baseline after changes like that.

Wire the check into CI​

Add a step to the CI workflow that runs your Cypress tests, after the cypress run --record step, that installs the Results API module and executes your verification script:

test_cypress.yaml
name: My Workflow
on: push

jobs:
  run-tests:
    runs-on: ubuntu-24.04
    steps:
      - uses: actions/checkout@v7
      - run: npm install
      - name: Run Cypress tests
        run: npx cypress run --record
        env:
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
+     - name: Enforce UI Coverage policy
+       run: |
+          npm install --force https://cdn.cypress.io/extract-cloud-results/v1/extract-cloud-results.tgz
+          node ./scripts/compareUICoverageBaseline.js
+       env:
+         CYPRESS_PROJECT_ID: ${{ secrets.CYPRESS_PROJECT_ID }}
+         CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}

This example runs the baseline script. If you chose the threshold policy instead, run verifyUICoverageResults.js in that step, or run whichever script matches the policy you settled on.

When your script exits non-zero, the CI step fails, which surfaces as a failed status check on the pull request. To make that check block the merge rather than just show red, mark it as required in your version control provider's branch protection settings (for example, GitHub's required status checks). Because enforcement lives entirely in your CI job, you control exactly when a coverage gap is advisory versus blocking.

See the Results API CI setup for the equivalent step in GitLab, Jenkins, Azure, CircleCI, and other providers.

Use Profiles for PR-specific configuration​

A pull request check should be fast and focused, while your nightly regression run tracks everything. Profiles let you apply different app quality configuration to different runs based on their run tags, so both can share one Cypress Cloud project.

For example, this configuration defines a pr-critical profile that narrows PR runs to just the flows you gate merges on, while the base configuration still reports on everything for regression tracking:

App Quality Config
{
  "profiles": [
    {
      "name": "pr-critical",
      "comment": "Narrow, fast config for PR runs: only the flows we block merges on",
      "config": {
        "viewFilters": [
          { "pattern": "/checkout/*", "include": true },
          { "pattern": "/login", "include": true },
          { "pattern": "*", "include": false }
        ]
      }
    }
  ]
}

Record your PR runs with the matching tag, and Cypress Cloud applies the profile automatically:

cypress run --record --tag pr-critical

When you record more than one run in a single CI build, pass the same tag to the helper so it returns the right run's results:

Fetch results for a tagged run
getUICoverageResults({ runTags: ['pr-critical'] })

You can confirm which profile produced a report by reading the config property on the result, which is useful for asserting that a PR run actually used the narrow configuration you expect.

Best practices​

When to update the baseline​

Promote the candidate baseline (ui-coverage-baseline.next.json) to your committed ui-coverage-baseline.json when you:

  • Add tests that cover previously untested elements and want to lock in the gain.
  • Deliberately accept a new gap as known debt that shouldn't block deployments.
  • Want a fresh reference point after a large change to the application.

Commit the baseline to version control so it's versioned alongside your code and available to every CI run.

Skip incomplete or non-passing runs​

A run can be cancelled, time out, or produce a partial report. The example already skips comparison when summary.isPartialReport is true; you can also check runStatus and skip enforcement for statuses like cancelled or timedOut, so an incomplete run warns instead of failing the build for the wrong reason. See Skip enforcement for partial or non-passing runs.

Manage baselines across branches​

You may want different baselines for different branches. Store branch-specific baseline files, or use an environment variable to choose which baseline to load, so a long-lived feature branch doesn't fail against main's numbers.

Where to store the baseline​

  • Version control (recommended): commit the JSON file so it's reviewed and versioned with the code that changed the coverage.
  • CI artifacts: store baselines as build artifacts retrieved on later runs.
  • External storage: use a database if you need richer versioning across many projects.
info

This programmatic baseline complements the Branch Review UI, which visually compares two runs. Use the script to gate merges automatically in CI, and Branch Review for manual investigation during code review.

See also​

Contents