{
  "doc": {
    "id": "cloud/features/flaky-test-management",
    "title": "Detect and fix flaky tests in Cypress Cloud",
    "description": "Detect, score, alert on, and fix flaky tests with Cypress Cloud. Track test suite reliability across CI and prioritize the flaky tests that matter most.",
    "section": "cloud",
    "source_path": "/llm/markdown/cloud/features/flaky-test-management.md",
    "version": "fbc9225067c51c52ee13224e3b702cf8a025ec12",
    "updated_at": "2026-08-14T12:36:26.878Z",
    "headings": [
      {
        "id": "cloud/features/flaky-test-management#flaky-test-management",
        "text": "Flaky Test Management",
        "level": 1
      },
      {
        "id": "cloud/features/flaky-test-management#why-flaky-test-management",
        "text": "Why Flaky Test Management",
        "level": 2
      },
      {
        "id": "cloud/features/flaky-test-management#flake-detection",
        "text": "Flake Detection",
        "level": 2
      },
      {
        "id": "cloud/features/flaky-test-management#enable-test-retries",
        "text": "Enable test retries",
        "level": 3
      },
      {
        "id": "cloud/features/flaky-test-management#flagging-flaky-tests",
        "text": "Flagging flaky tests",
        "level": 3
      },
      {
        "id": "cloud/features/flaky-test-management#flake-alerting",
        "text": "Flake Alerting",
        "level": 2
      },
      {
        "id": "cloud/features/flaky-test-management#github",
        "text": "GitHub",
        "level": 3
      },
      {
        "id": "cloud/features/flaky-test-management#gitlab",
        "text": "GitLab",
        "level": 3
      },
      {
        "id": "cloud/features/flaky-test-management#bitbucket",
        "text": "Bitbucket",
        "level": 3
      },
      {
        "id": "cloud/features/flaky-test-management#slack",
        "text": "Slack",
        "level": 3
      },
      {
        "id": "cloud/features/flaky-test-management#microsoft-teams",
        "text": "Microsoft Teams",
        "level": 3
      },
      {
        "id": "cloud/features/flaky-test-management#debug-and-fix-flaky-tests",
        "text": "Debug and fix flaky tests",
        "level": 2
      },
      {
        "id": "cloud/features/flaky-test-management#investigate-flake-with-cloud-mcp",
        "text": "Investigate flake with Cloud MCP",
        "level": 2
      },
      {
        "id": "cloud/features/flaky-test-management#how-to-use-it",
        "text": "How to use it",
        "level": 3
      },
      {
        "id": "cloud/features/flaky-test-management#compare-flake-across-branches-with-branch-review",
        "text": "Compare flake across branches with Branch Review",
        "level": 2
      },
      {
        "id": "cloud/features/flaky-test-management#flaky-test-analytics",
        "text": "Flaky test analytics",
        "level": 2
      },
      {
        "id": "cloud/features/flaky-test-management#flake-severity",
        "text": "Flake severity",
        "level": 3
      },
      {
        "id": "cloud/features/flaky-test-management#failure-rate-vs-flake-rate",
        "text": "Failure rate vs. flake rate",
        "level": 3
      },
      {
        "id": "cloud/features/flaky-test-management#flake-analytics-across-all-projects-with-enterprise-reporting",
        "text": "Flake analytics across all projects with Enterprise Reporting",
        "level": 2
      },
      {
        "id": "cloud/features/flaky-test-management#export-flake-data-with-the-data-extract-api",
        "text": "Export flake data with the Data Extract API",
        "level": 2
      },
      {
        "id": "cloud/features/flaky-test-management#available-flake-reports",
        "text": "Available flake reports",
        "level": 3
      },
      {
        "id": "cloud/features/flaky-test-management#example-request",
        "text": "Example request",
        "level": 3
      },
      {
        "id": "cloud/features/flaky-test-management#see-also",
        "text": "See Also",
        "level": 2
      }
    ]
  },
  "chunks": [
    {
      "id": "cloud/features/flaky-test-management#why-flaky-test-management",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Why Flaky Test Management",
      "heading_level": 2,
      "content_markdown": "## Why Flaky Test Management\n\nA **flaky test** passes and fails across retries without any code change. It \"fails\" once, runs again on the same code, and passes. A single flaky test seems harmless. At scale, flake is one of the most expensive and corrosive problems a software organization faces:\n\n*   **It erodes trust in the entire test suite.** Once developers learn that a red build might just be \"the flaky one,\" they start ignoring failures, and real regressions slip through to production.\n*   **It drains engineering time.** Teams burn hours re-running pipelines, investigating failures that aren't real, and manually triaging which tests can be trusted. That's velocity spent on noise instead of shipping.\n*   **It slows down delivery.** Flake blocks merges, delays releases, and forces developers to context-switch away from feature work to babysit CI.\n*   **It hides the real picture.** Without data, \"the tests are flaky\" is just a vague complaint. There's no way to know whether reliability is improving or degrading over time, or which tests are actually responsible.\n\nFlaky Test Management turns flake from an invisible, anecdotal cost into **measurable, prioritized** work. Cypress Cloud automatically detects flake from your existing CI runs, scores each test by severity, tracks suite-wide reliability over time, and alerts the right people the moment new flake appears. The result is a clear picture of test suite health that helps teams keep shipping, focus effort on the few tests causing the most disruption, and show reliability trending in the right direction.\n\nBy enabling [test retries](/llm/markdown/app/guides/test-retries.md), Cypress Cloud can detect, flag, and track flaky tests from your recorded Cypress test runs in your CI/CD pipeline.\n\n**Prerequisites**\n\nFlaky Test Management works on runs [recorded to Cypress Cloud](/llm/markdown/cloud/get-started/setup.md) from your CI/CD pipeline, with [test retries](/llm/markdown/app/guides/test-retries.md#Configure-Test-Retries) enabled. Detection, analytics, and alerting require a [Team Cypress Cloud plan](https://cypress.io/pricing).\n",
      "section": "cloud",
      "anchors": [
        "why-flaky-test-management"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 401
    },
    {
      "id": "cloud/features/flaky-test-management#flake-detection",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Flake Detection",
      "heading_level": 2,
      "content_markdown": "## Flake Detection\n\n**Cypress Cloud Feature:** Test flake insights are available to organizations with a [Team Cypress Cloud plan](https://cypress.io/pricing).\n\nYou can't fix what you can't see. Flake Detection gives you a continuous, automated record of every flaky test in your suite, captured as it happens in CI, so you can assess severity and prioritize fixes instead of relying on developers to remember and report intermittent failures.\n\n### Enable test retries\n\nTest retries are the _fundamental mechanism_ that enables Cypress Cloud to detect flaky tests. Enabling test retries in your Cypress configuration is required to take advantage of any Flaky Test Management feature. Read more about [test retries](/llm/markdown/app/guides/test-retries.md) in the Cypress App guide.\n\n*   cypress.config.js\n*   cypress.config.ts\n\n```\nconst { defineConfig } = require('cypress')module.exports = defineConfig({  retries: {    runMode: 2,    openMode: 0,  },})\n```\n\n```\nimport { defineConfig } from 'cypress'export default defineConfig({  retries: {    runMode: 2,    openMode: 0,  },})\n```\n\nIf you want more control over the pass/fail result of any test deemed flaky by test retries, you may want to use [experimental test retries](/llm/markdown/app/references/experiments.md#Experimental-Flake-Detection-Features).\n\n### Flagging flaky tests\n\nTest runs containing flaky tests are flagged with a flaky-test count on the Cypress Cloud **Latest Runs** page.\n\nRuns containing flaky tests can be filtered in or out with the **Flaky** filter. This gives teams an at-a-glance view of how much flake is reaching their pipeline on any given day.\n",
      "section": "cloud",
      "anchors": [
        "flake-detection"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 305
    },
    {
      "id": "cloud/features/flaky-test-management#enable-test-retries",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Enable test retries",
      "heading_level": 3,
      "content_markdown": "### Enable test retries\n\nTest retries are the _fundamental mechanism_ that enables Cypress Cloud to detect flaky tests. Enabling test retries in your Cypress configuration is required to take advantage of any Flaky Test Management feature. Read more about [test retries](/llm/markdown/app/guides/test-retries.md) in the Cypress App guide.\n\n*   cypress.config.js\n*   cypress.config.ts\n\n```\nconst { defineConfig } = require('cypress')module.exports = defineConfig({  retries: {    runMode: 2,    openMode: 0,  },})\n```\n\n```\nimport { defineConfig } from 'cypress'export default defineConfig({  retries: {    runMode: 2,    openMode: 0,  },})\n```\n\nIf you want more control over the pass/fail result of any test deemed flaky by test retries, you may want to use [experimental test retries](/llm/markdown/app/references/experiments.md#Experimental-Flake-Detection-Features).\n",
      "section": "cloud",
      "anchors": [
        "enable-test-retries"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 145
    },
    {
      "id": "cloud/features/flaky-test-management#flagging-flaky-tests",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Flagging flaky tests",
      "heading_level": 3,
      "content_markdown": "### Flagging flaky tests\n\nTest runs containing flaky tests are flagged with a flaky-test count on the Cypress Cloud **Latest Runs** page.\n\nRuns containing flaky tests can be filtered in or out with the **Flaky** filter. This gives teams an at-a-glance view of how much flake is reaching their pipeline on any given day.\n",
      "section": "cloud",
      "anchors": [
        "flagging-flaky-tests"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 72
    },
    {
      "id": "cloud/features/flaky-test-management#flake-alerting",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Flake Alerting",
      "heading_level": 2,
      "content_markdown": "## Flake Alerting\n\n**Cypress Cloud Feature:** Test flake alerting is available to organizations with a [Team Cypress Cloud plan](https://cypress.io/pricing).\n\nDetection only pays off if the right people know about flake when it matters. Flake Alerting pushes flake notifications directly into the tools your team already works in, so flake is caught in the review process instead of compounding silently over time. Each integration below links to its full setup instructions.\n\nSource control integrations surface flake right where code is reviewed, and can even **block merges** while flaky tests are present:\n\n### GitHub\n\nEnable flake alerting on GitHub status checks in your project's GitHub integration settings.\n\nSee [Status checks](/llm/markdown/cloud/integrations/github.md#Status-checks) for full setup instructions.\n\nOnce enabled, GitHub PR comments show the number of flaky tests associated with the PR in the test summary and include a **Flakiness** section highlighting the specific tests that flaked, putting reliability data in front of reviewers before they merge.\n\n### GitLab\n\nCypress can post a **`cypress/flake`** commit status check that reports whether a run contained flaky tests, so you can stop merge requests with flaky tests from merging, along with merge request comments that summarize the run. Enable both in your project's GitLab integration settings.\n\nSee [status checks](/llm/markdown/cloud/integrations/gitlab.md#Status-checks) for full setup instructions.\n\n### Bitbucket\n\nCypress can post a **`cypress/flake`** commit status check that reports whether a run contained flaky tests, so you can stop pull requests with flaky tests from merging, along with pull request comments that summarize the run. Enable both in your project's Bitbucket integration settings.\n\nSee [status checks](/llm/markdown/cloud/integrations/bitbucket.md#Status-checks) for full setup instructions.\n\n### Slack\n\nEnable the **Flaky tests** notification preference in your Slack integration settings to be alerted whenever a run contains flaky tests, independent of whether the run passed or failed.\n\nSee [Flaky test alerts](/llm/markdown/cloud/integrations/slack.md#Flaky-test-alerts) for full setup instructions.\n\nOnce enabled, Cypress Cloud sends a Slack message whenever flaky tests are detected, so teams can respond in real time:\n\n### Microsoft Teams\n\nEnable the **Flaky tests** notification preference in your Microsoft Teams integration settings to post a message to your channel whenever a run contains flaky tests, independent of the run's pass/fail status. See [Flaky test alerts](/llm/markdown/cloud/integrations/microsoft-teams.md#Flaky-test-alerts) for full setup instructions.\n",
      "section": "cloud",
      "anchors": [
        "flake-alerting"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 477
    },
    {
      "id": "cloud/features/flaky-test-management#github",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "GitHub",
      "heading_level": 3,
      "content_markdown": "### GitHub\n\nEnable flake alerting on GitHub status checks in your project's GitHub integration settings.\n\nSee [Status checks](/llm/markdown/cloud/integrations/github.md#Status-checks) for full setup instructions.\n\nOnce enabled, GitHub PR comments show the number of flaky tests associated with the PR in the test summary and include a **Flakiness** section highlighting the specific tests that flaked, putting reliability data in front of reviewers before they merge.\n",
      "section": "cloud",
      "anchors": [
        "github"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 83
    },
    {
      "id": "cloud/features/flaky-test-management#gitlab",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "GitLab",
      "heading_level": 3,
      "content_markdown": "### GitLab\n\nCypress can post a **`cypress/flake`** commit status check that reports whether a run contained flaky tests, so you can stop merge requests with flaky tests from merging, along with merge request comments that summarize the run. Enable both in your project's GitLab integration settings.\n\nSee [status checks](/llm/markdown/cloud/integrations/gitlab.md#Status-checks) for full setup instructions.\n",
      "section": "cloud",
      "anchors": [
        "gitlab"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 71
    },
    {
      "id": "cloud/features/flaky-test-management#bitbucket",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Bitbucket",
      "heading_level": 3,
      "content_markdown": "### Bitbucket\n\nCypress can post a **`cypress/flake`** commit status check that reports whether a run contained flaky tests, so you can stop pull requests with flaky tests from merging, along with pull request comments that summarize the run. Enable both in your project's Bitbucket integration settings.\n\nSee [status checks](/llm/markdown/cloud/integrations/bitbucket.md#Status-checks) for full setup instructions.\n",
      "section": "cloud",
      "anchors": [
        "bitbucket"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 71
    },
    {
      "id": "cloud/features/flaky-test-management#slack",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Slack",
      "heading_level": 3,
      "content_markdown": "### Slack\n\nEnable the **Flaky tests** notification preference in your Slack integration settings to be alerted whenever a run contains flaky tests, independent of whether the run passed or failed.\n\nSee [Flaky test alerts](/llm/markdown/cloud/integrations/slack.md#Flaky-test-alerts) for full setup instructions.\n\nOnce enabled, Cypress Cloud sends a Slack message whenever flaky tests are detected, so teams can respond in real time:\n",
      "section": "cloud",
      "anchors": [
        "slack"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 77
    },
    {
      "id": "cloud/features/flaky-test-management#microsoft-teams",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Microsoft Teams",
      "heading_level": 3,
      "content_markdown": "### Microsoft Teams\n\nEnable the **Flaky tests** notification preference in your Microsoft Teams integration settings to post a message to your channel whenever a run contains flaky tests, independent of the run's pass/fail status. See [Flaky test alerts](/llm/markdown/cloud/integrations/microsoft-teams.md#Flaky-test-alerts) for full setup instructions.\n",
      "section": "cloud",
      "anchors": [
        "microsoft-teams"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 56
    },
    {
      "id": "cloud/features/flaky-test-management#debug-and-fix-flaky-tests",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Debug and fix flaky tests",
      "heading_level": 2,
      "content_markdown": "## Debug and fix flaky tests\n\nDetecting and scoring flake tells you _which_ tests to fix and in what order. The next step is finding _why_ a test flaked, and that is where [Test Replay](/llm/markdown/cloud/features/test-replay.md) comes in.\n\nFrom a flaky test's details panel, open any flaky run in Test Replay to replay the exact attempts as they ran in CI, with full access to the DOM, network requests, console logs, and element state at the moment of failure. Comparing a passing attempt against a failing one on the same code is often the fastest way to pinpoint the timing issue, race condition, or environmental dependency causing the flake.\n",
      "section": "cloud",
      "anchors": [
        "debug-and-fix-flaky-tests"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 144
    },
    {
      "id": "cloud/features/flaky-test-management#investigate-flake-with-cloud-mcp",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Investigate flake with Cloud MCP",
      "heading_level": 2,
      "content_markdown": "## Investigate flake with Cloud MCP\n\n[Cloud MCP](/llm/markdown/cloud/integrations/cloud-mcp.md) connects your AI coding assistant (Cursor, Claude, VS Code Copilot, and others) directly to Cypress Cloud, closing the gap between CI results and your editor. Instead of manually triaging which failures are real, you can ask your agent to pull flake data from Cloud and act on it without leaving your workflow.\n\nFor flake, the agent can use the `cypress_get_flaky_tests` tool to identify the flaky tests in a run, which directly answers the question reviewers ask most: **\"Are these failures real regressions, or known flakiness?\"** Paired with `cypress_get_failed_tests`, which returns error details, stack traces, and a Test Replay link for each attempt, your agent can separate signal from noise and jump straight to a root-cause investigation.\n\nCloud MCP is included on **every Cypress Cloud plan at no additional cost**. Flaky test data returned through MCP is plan-specific, so it is available only when your organization is on a plan that includes flake reporting (the [Team plan](https://cypress.io/pricing) or higher).\n\n### How to use it\n\n1.  **Connect Cloud MCP.** Have an admin enable the integration, then authenticate and add the server to your AI client. See the [Cloud MCP setup guide](/llm/markdown/cloud/integrations/cloud-mcp.md#Configuring-the-Cloud-MCP).\n2.  **Prompt your agent.** Mention **\"Cypress Cloud\"** so the agent selects the right tools, give it context (a branch, commit, or Run URL), and define the workflow you want, not just the question.\n\nFor example, to audit recent flake and look for patterns:\n\n> \"List all flaky tests from the last 5 runs in Cypress Cloud. Are there any common patterns in the error messages?\"\n\nThis turns flake triage into a conversation: the agent surfaces the flaky tests, groups the errors, and can follow up with the Test Replay context needed to fix them.\n",
      "section": "cloud",
      "anchors": [
        "investigate-flake-with-cloud-mcp"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 388
    },
    {
      "id": "cloud/features/flaky-test-management#how-to-use-it",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "How to use it",
      "heading_level": 3,
      "content_markdown": "### How to use it\n\n1.  **Connect Cloud MCP.** Have an admin enable the integration, then authenticate and add the server to your AI client. See the [Cloud MCP setup guide](/llm/markdown/cloud/integrations/cloud-mcp.md#Configuring-the-Cloud-MCP).\n2.  **Prompt your agent.** Mention **\"Cypress Cloud\"** so the agent selects the right tools, give it context (a branch, commit, or Run URL), and define the workflow you want, not just the question.\n\nFor example, to audit recent flake and look for patterns:\n\n> \"List all flaky tests from the last 5 runs in Cypress Cloud. Are there any common patterns in the error messages?\"\n\nThis turns flake triage into a conversation: the agent surfaces the flaky tests, groups the errors, and can follow up with the Test Replay context needed to fix them.\n",
      "section": "cloud",
      "anchors": [
        "how-to-use-it"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 167
    },
    {
      "id": "cloud/features/flaky-test-management#compare-flake-across-branches-with-branch-review",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Compare flake across branches with Branch Review",
      "heading_level": 2,
      "content_markdown": "## Compare flake across branches with Branch Review\n\nWhen deciding whether a Pull Request is safe to merge, the key question is usually: _did my changes introduce new flake, or was this test already unstable?_ [Branch Review](/llm/markdown/cloud/features/branch-review.md) answers this directly by comparing the changed branch against its base in a single view, so reviewers hold authors accountable for the flake they introduce without getting blocked by flake they didn't.\n\nThe **Flaky** tab lists every spec that flaked, marking each result as:\n\n*   **new**: flake not previously captured, and possibly introduced by this branch\n*   **existing**: flake already present on the base branch\n*   **resolved**: flake that was present before but no longer appears\n\nThis lets reviewers focus only on the flake a branch is responsible for, and confirm that a stability fix actually took effect. From there you can open a test to compare attempts side by side with [Test Replay](/llm/markdown/cloud/features/test-replay.md) and the code diff to track down the root cause.\n",
      "section": "cloud",
      "anchors": [
        "compare-flake-across-branches-with-branch-review"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 215
    },
    {
      "id": "cloud/features/flaky-test-management#flaky-test-analytics",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Flaky test analytics",
      "heading_level": 2,
      "content_markdown": "## Flaky test analytics\n\n**Cypress Cloud Feature:** Test flake analytics are available to organizations with a [Team Cypress Cloud plan](https://cypress.io/pricing).\n\nThe flaky test analytics page gives a bird's-eye view of flake across your project, making it easy to understand reliability at a glance and back it up with data. It includes:\n\n*   A plot of the **number of flaky tests over time**, so you can see reliability trending up or down release over release.\n*   The overall **flakiness level** of the entire project: a single health metric you can track and set goals against.\n*   The **number of flaky tests grouped by severity**, so effort goes where the pain is greatest.\n*   A filterable **log of every flaky test**, ordered by severity, ready to triage and assign.\n\n### Flake severity\n\nFlake **severity** is determined by how often a test flakes, its **flake rate**. Severity lets teams prioritize objectively: fix the handful of high-severity tests disrupting the most builds before chasing rare, low-impact flake.\n\nFlake rate is the share of recent runs affected by flake, calculated as the number of runs where the spec had a flaky test divided by the total number of runs in the current window.\n\n| Severity | Flake rate |\n| --- | --- |\n| — | 0% |\n| Low | \\>0-10% |\n| Medium | \\>10-50% |\n| High | \\>50% |\n\nSelecting any flaky test case opens a details panel with the context engineers need to find and fix the root cause quickly:\n\n*   A historical log of the latest flaky runs\n*   The most common errors across the test case's runs\n*   The related test case changelog\n*   A plot of failure rate and flake rate over time\n\nTogether, these test-level details shorten the time from \"this test is flaky\" to \"here's why,\" reducing the investigation cost of every fix.\n\n### Failure rate vs. flake rate\n\nWhen test retries are enabled, it's important to distinguish **failure rate** from **flake rate**. Both are tracked over time for each flaky test on the analytics page as shown below.\n\nA test flagged as flaky may still have ultimately passed after retries. The status of an individual retry attempt is separate from the final test status.\n\nFor example, a project configured to retry failing tests up to three times may fail the first two attempts and pass on the third, a final status of passing.\n\nBecause of this, a test can show a **zero final failure rate while still exhibiting flake**.\n\nThis is exactly why flake is so easy to miss without dedicated tracking: the build goes green, but the underlying instability, and its cost, is still there.\n",
      "section": "cloud",
      "anchors": [
        "flaky-test-analytics"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 591
    },
    {
      "id": "cloud/features/flaky-test-management#flake-severity",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Flake severity",
      "heading_level": 3,
      "content_markdown": "### Flake severity\n\nFlake **severity** is determined by how often a test flakes, its **flake rate**. Severity lets teams prioritize objectively: fix the handful of high-severity tests disrupting the most builds before chasing rare, low-impact flake.\n\nFlake rate is the share of recent runs affected by flake, calculated as the number of runs where the spec had a flaky test divided by the total number of runs in the current window.\n\n| Severity | Flake rate |\n| --- | --- |\n| — | 0% |\n| Low | \\>0-10% |\n| Medium | \\>10-50% |\n| High | \\>50% |\n\nSelecting any flaky test case opens a details panel with the context engineers need to find and fix the root cause quickly:\n\n*   A historical log of the latest flaky runs\n*   The most common errors across the test case's runs\n*   The related test case changelog\n*   A plot of failure rate and flake rate over time\n\nTogether, these test-level details shorten the time from \"this test is flaky\" to \"here's why,\" reducing the investigation cost of every fix.\n",
      "section": "cloud",
      "anchors": [
        "flake-severity"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 243
    },
    {
      "id": "cloud/features/flaky-test-management#failure-rate-vs-flake-rate",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Failure rate vs. flake rate",
      "heading_level": 3,
      "content_markdown": "### Failure rate vs. flake rate\n\nWhen test retries are enabled, it's important to distinguish **failure rate** from **flake rate**. Both are tracked over time for each flaky test on the analytics page as shown below.\n\nA test flagged as flaky may still have ultimately passed after retries. The status of an individual retry attempt is separate from the final test status.\n\nFor example, a project configured to retry failing tests up to three times may fail the first two attempts and pass on the third, a final status of passing.\n\nBecause of this, a test can show a **zero final failure rate while still exhibiting flake**.\n\nThis is exactly why flake is so easy to miss without dedicated tracking: the build goes green, but the underlying instability, and its cost, is still there.\n",
      "section": "cloud",
      "anchors": [
        "failure-rate-vs-flake-rate"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 179
    },
    {
      "id": "cloud/features/flaky-test-management#flake-analytics-across-all-projects-with-enterprise-reporting",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Flake analytics across all projects with Enterprise Reporting",
      "heading_level": 2,
      "content_markdown": "## Flake analytics across all projects with Enterprise Reporting\n\nThe analytics above focus on a single project. To understand flake across your _entire organization_, [Enterprise Reporting](/llm/markdown/cloud/features/analytics/enterprise-reporting.md#Flaky-Tests) provides a dedicated **Flaky Tests** dashboard in Cypress Cloud.\n\n**Cypress Cloud Feature:** Enterprise Reporting is available to organizations with an [Enterprise Cypress Cloud plan](https://cypress.io/pricing).\n\nThe **Flaky Tests** tab surfaces flaky test occurrences and rates for your whole organization or a selected set of projects, as a point-in-time snapshot or trended over time. Because it reports flake **rate** (the percentage of runs that flake), you can compare projects fairly regardless of how often each one runs, so a high-volume project doesn't look worse simply because it executes more tests. The raw data behind every chart can also be downloaded for use in your own tools.\n",
      "section": "cloud",
      "anchors": [
        "flake-analytics-across-all-projects-with-enterprise-reporting"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 173
    },
    {
      "id": "cloud/features/flaky-test-management#export-flake-data-with-the-data-extract-api",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Export flake data with the Data Extract API",
      "heading_level": 2,
      "content_markdown": "## Export flake data with the Data Extract API\n\nThe in-app analytics answer \"how healthy is this project?\" The [Data Extract API](/llm/markdown/cloud/integrations/data-extract-api.md) answers the same question across your _entire organization_, in a format you own. It lets you pull flake metrics into your own dashboards and reports, so test reliability sits alongside the other metrics your team tracks and trends can be followed release over release without anyone logging into Cypress Cloud.\n\n**Cypress Cloud Feature:** The **Data Extract API** is available to organizations with an [Enterprise Cypress Cloud plan](https://cypress.io/pricing).\n\n### Available flake reports\n\nThe API exposes five flake-focused reports. Each is requested with its `report_id` and returns CSV, JSON, or XLSX for a date range you specify.\n\n| Report | `report_id` | Answers |\n| --- | --- | --- |\n| Flake detail over time | `flake-per-project-over-time` | How does flaky test **count** per project change day over day and week over week? |\n| Flaky rate per project | `flake-rate-per-project` | What **percentage** of runs flake in each project, normalizing out projects that simply run more often? |\n| Flaky rate per project over time | `flake-rate-per-project-over-time` | How is each project's flaky **rate** trending over the period? |\n| Flaky test details | `flaky-test-details` | Every individual flaky result, including commit, branch, author, errors, and a `test_replay_url` to jump straight to the failure. |\n| Top flaky tests per project | `top-flaky-per-project` | Which specs flake the most, with a Test Replay link to a recent occurrence for triage? |\n\n**Flaky rate** is reported as a whole number. For example, 1 flaky test over 4 runs has a flaky rate of `25`. This mirrors the [flake rate](#Flaky-test-analytics) used in the in-app analytics.\n\n### Example request\n\nSubstitute your organization's API key for `YOUR-API-KEY` and choose the report with `report_id`:\n\n```\nhttps://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=flake-rate-per-project&export_format=csv&start_date=2025-03-01\n```\n\nThe `flaky-test-details` report is available in near real time, roughly 30 minutes after a run completes, so it is well suited to automated pipelines that react to new flake. For the full list of parameters, formats, and limits, see the [Data Extract API](/llm/markdown/cloud/integrations/data-extract-api.md#Flaky) documentation.\n",
      "section": "cloud",
      "anchors": [
        "export-flake-data-with-the-data-extract-api"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 464
    },
    {
      "id": "cloud/features/flaky-test-management#available-flake-reports",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Available flake reports",
      "heading_level": 3,
      "content_markdown": "### Available flake reports\n\nThe API exposes five flake-focused reports. Each is requested with its `report_id` and returns CSV, JSON, or XLSX for a date range you specify.\n\n| Report | `report_id` | Answers |\n| --- | --- | --- |\n| Flake detail over time | `flake-per-project-over-time` | How does flaky test **count** per project change day over day and week over week? |\n| Flaky rate per project | `flake-rate-per-project` | What **percentage** of runs flake in each project, normalizing out projects that simply run more often? |\n| Flaky rate per project over time | `flake-rate-per-project-over-time` | How is each project's flaky **rate** trending over the period? |\n| Flaky test details | `flaky-test-details` | Every individual flaky result, including commit, branch, author, errors, and a `test_replay_url` to jump straight to the failure. |\n| Top flaky tests per project | `top-flaky-per-project` | Which specs flake the most, with a Test Replay link to a recent occurrence for triage? |\n\n**Flaky rate** is reported as a whole number. For example, 1 flaky test over 4 runs has a flaky rate of `25`. This mirrors the [flake rate](#Flaky-test-analytics) used in the in-app analytics.\n",
      "section": "cloud",
      "anchors": [
        "available-flake-reports"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 261
    },
    {
      "id": "cloud/features/flaky-test-management#example-request",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "Example request",
      "heading_level": 3,
      "content_markdown": "### Example request\n\nSubstitute your organization's API key for `YOUR-API-KEY` and choose the report with `report_id`:\n\n```\nhttps://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=flake-rate-per-project&export_format=csv&start_date=2025-03-01\n```\n\nThe `flaky-test-details` report is available in near real time, roughly 30 minutes after a run completes, so it is well suited to automated pipelines that react to new flake. For the full list of parameters, formats, and limits, see the [Data Extract API](/llm/markdown/cloud/integrations/data-extract-api.md#Flaky) documentation.\n",
      "section": "cloud",
      "anchors": [
        "example-request"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 84
    },
    {
      "id": "cloud/features/flaky-test-management#see-also",
      "doc_id": "cloud/features/flaky-test-management",
      "heading": "See Also",
      "heading_level": 2,
      "content_markdown": "## See Also\n\n*   Read answers to common flake questions in the [Cypress Cloud FAQ](/llm/markdown/cloud/faq.md#Flaky-Test-Management).\n*   Debug flaky runs by replaying the exact CI attempt with [Test Replay](/llm/markdown/cloud/features/test-replay.md).\n*   Query flake from your AI coding assistant with [Cloud MCP](/llm/markdown/cloud/integrations/cloud-mcp.md).\n*   Compare flake between branches before merging with [Branch Review](/llm/markdown/cloud/features/branch-review.md).\n*   Access organization-wide flake reports with the [Data Extract API](/llm/markdown/cloud/integrations/data-extract-api.md#Flaky).\n",
      "section": "cloud",
      "anchors": [
        "see-also"
      ],
      "path": "/llm/json/chunked/cloud/features/flaky-test-management.json",
      "token_estimate": 79
    }
  ]
}