{ "doc": { "id": "accessibility/guides/detect-changes", "title": "Detect changes | Cypress Accessibility Documentation", "description": "Monitor accessibility issues in production by running scheduled Cypress tests against live environments, capturing dynamic content changes, and generating automated reports for a comprehensive accessibility overview.", "section": "accessibility", "source_path": "/llm/markdown/accessibility/guides/detect-changes.md", "version": "e6988a974973e9090ce70406c38cb2b9e0eac9fa", "updated_at": "2026-05-15T15:50:22.536Z", "headings": [ { "id": "accessibility/guides/detect-changes#detect-and-manage-changes", "text": "Detect and manage changes", "level": 1 }, { "id": "accessibility/guides/detect-changes#branch-review", "text": "Branch Review", "level": 2 }, { "id": "accessibility/guides/detect-changes#branch-review-use-cases", "text": "Branch Review use cases", "level": 3 }, { "id": "accessibility/guides/detect-changes#speed-up-code-reviews-and-deployments", "text": "Speed up code reviews and deployments", "level": 4 }, { "id": "accessibility/guides/detect-changes#compare-reports-during-local-development-without-waiting-for-ci", "text": "Compare reports during local development (without waiting for CI)", "level": 4 }, { "id": "accessibility/guides/detect-changes#see-whats-changing-between-scheduled-runs", "text": "See what's changing between scheduled runs", "level": 4 }, { "id": "accessibility/guides/detect-changes#note", "text": "Note", "level": 3 }, { "id": "accessibility/guides/detect-changes#results-api", "text": "Results API", "level": 2 }, { "id": "accessibility/guides/detect-changes#results-api-use-cases", "text": "Results API use cases", "level": 3 }, { "id": "accessibility/guides/detect-changes#fail-a-build-based-on-new-failed-accessibility-rules", "text": "Fail a build based on new failed accessibility rules", "level": 4 }, { "id": "accessibility/guides/detect-changes#fail-a-build-based-on-deviation-from-a-desired-score", "text": "Fail a build based on deviation from a desired score", "level": 4 }, { "id": "accessibility/guides/detect-changes#alert-specific-teams-about-failures-they-own", "text": "Alert specific teams about failures they own", "level": 4 }, { "id": "accessibility/guides/detect-changes#analytics", "text": "Analytics", "level": 2 }, { "id": "accessibility/guides/detect-changes#example-accessibility-charts", "text": "Example accessibility charts", "level": 3 }, { "id": "accessibility/guides/detect-changes#overall-score-top-projects-and-score-trend", "text": "Overall score, top projects, and score trend", "level": 4 }, { "id": "accessibility/guides/detect-changes#severity-breakdown-for-the-selected-period", "text": "Severity breakdown for the selected period", "level": 4 }, { "id": "accessibility/guides/detect-changes#weekly-average-violations-per-run-by-rule", "text": "Weekly average violations per run by rule", "level": 4 }, { "id": "accessibility/guides/detect-changes#weekly-average-failed-rules-per-run-by-impact", "text": "Weekly average failed rules per run by impact", "level": 4 }, { "id": "accessibility/guides/detect-changes#weekly-average-element-violations-per-run-by-impact", "text": "Weekly average element violations per run by impact", "level": 4 }, { "id": "accessibility/guides/detect-changes#enterprise-reporting", "text": "Enterprise Reporting", "level": 4 }, { "id": "accessibility/guides/detect-changes#analytics-use-cases", "text": "Analytics use cases", "level": 3 }, { "id": "accessibility/guides/detect-changes#track-progress-and-spot-trends", "text": "Track progress and spot trends", "level": 4 }, { "id": "accessibility/guides/detect-changes#pull-high-level-metrics-for-ai-analysis", "text": "Pull high level metrics for AI analysis", "level": 4 }, { "id": "accessibility/guides/detect-changes#set-goals-and-minimum-standards", "text": "Set goals and minimum standards", "level": 4 }, { "id": "accessibility/guides/detect-changes#combine-approaches", "text": "Combine approaches", "level": 2 } ] }, "content": { "type": "root", "children": [ { "type": "heading", "depth": 1, "children": [ { "type": "text", "value": "Detect and manage changes" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Cypress Accessibility supports three main paths to observing and acting on changes in the accessibility report for your projects:" } ] }, { "type": "list", "ordered": false, "start": null, "spread": false, "children": [ { "type": "listItem", "spread": false, "checked": null, "children": [ { "type": "paragraph", "children": [ { "type": "text", "value": "Run comparisons with " }, { "type": "link", "title": null, "url": "/llm/markdown/accessibility/core-concepts/compare-reports.md", "children": [ { "type": "text", "value": "Branch Review" } ] } ] } ] }, { "type": "listItem", "spread": false, "checked": null, "children": [ { "type": "paragraph", "children": [ { "type": "text", "value": "The " }, { "type": "link", "title": null, "url": "/llm/markdown/accessibility/guides/results-api.md", "children": [ { "type": "text", "value": "Results API" } ] }, { "type": "text", "value": " to fail builds or trigger alerts when results don't meet your standards" } ] } ] }, { "type": "listItem", "spread": false, "checked": null, "children": [ { "type": "paragraph", "children": [ { "type": "link", "title": null, "url": "/llm/markdown/cloud/features/analytics/enterprise-reporting.md#Cypress-Accessibility", "children": [ { "type": "text", "value": "Analytics" } ] }, { "type": "text", "value": " for high-level trend-spotting and analysis" } ] } ] } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "In this guide we'll discuss how these work and some common situations where we might use them together." } ] }, { "type": "heading", "depth": 2, "children": [ { "type": "text", "value": "Branch Review" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Branch Review helps you spot new issues quickly by comparing accessibility results from two Cypress runs." } ] }, { "type": "list", "ordered": false, "start": null, "spread": false, "children": [ { "type": "listItem", "spread": false, "checked": null, "children": [ { "type": "paragraph", "children": [ { "type": "text", "value": "Pinpoint when and where problems have been introduced" } ] } ] }, { "type": "listItem", "spread": false, "checked": null, "children": [ { "type": "paragraph", "children": [ { "type": "text", "value": "Understand and debug easily with fully rendered page snapshots" } ] } ] }, { "type": "listItem", "spread": false, "checked": null, "children": [ { "type": "paragraph", "children": [ { "type": "text", "value": "Share links to specific new issues with all the context needed to dive in" } ] } ] } ] }, { "type": "heading", "depth": 3, "children": [ { "type": "text", "value": "Branch Review use cases" } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Speed up code reviews and deployments" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Use Branch review when reviewing a pull request made by another developer or before you create one of your own. You can compare the report from your changed branch with the equivalent report on the main branch. The comparison allows you to see any issues that your changes have improved or fixed completely, as well as net-new issues introduced, and any new instances of existing failures." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Since this reduces the amount of violations to review to just what was affected by the PR code changes, you won't spend any time looking for what's relevant -- and if there is no difference, you'll know your PR does not affect the accessibility score and is safe to merge." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "See the main " }, { "type": "link", "title": null, "url": "/llm/markdown/accessibility/core-concepts/compare-reports.md", "children": [ { "type": "text", "value": "Accessibility Branch Review docs" } ] }, { "type": "text", "value": " for a video demo of this workflow." } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Compare reports during local development (without waiting for CI)" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "You might expect that every Cypress Accessibility report requires a full standard Cypress test run in your Continuous Integration (CI) pipeline." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "A common accessibility workflow is to " }, { "type": "link", "title": null, "url": "/llm/markdown/accessibility/guides/local-development.md", "children": [ { "type": "text", "value": "create small Cypress Cloud recordings from your local development environment" } ] }, { "type": "text", "value": " to preview the effects of any code changes. This allows for fast analysis of just a single test or two, or as much as you need to have confidence your changes are working -- especially if you are actively remediating accessibility issues." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "To get a high-quality comparison for a small set of tests, record the tests locally from your main branch, and then on the branch with changes. In Cypress Cloud, you will be able to compare these two runs knowing they cover exactly the same scope." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "This completely sidesteps the usual \"push, build, run tests, report\" cycle you might be used to, and is great when you need fast feedback." } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "See what's changing between scheduled runs" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Many teams have some Cypress projects that run on a schedule and target a QA, staging, or production environment. These runs produce Cypress Accessibility reports but are not connected to specific pull requests with code changes. Branch Review works perfectly in these situations, even if all runs are on the same branch." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Comparing reports on a regular schedule can let you see if accessibility issues have been introduced between this week and last week, for example, and then find the specific run where the changes happened. That \"diff report\" can can be forwarded to engineering or product teams as the basis of a ticket." } ] }, { "type": "heading", "depth": 3, "children": [ { "type": "text", "value": "Note" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Comparing two runs in detail with Branch Review requires a good understanding of the runs themselves. The comparison is most useful if the runs are testing the same pages and workflows, and if each run fully passed all tests. This means that Cypress saw all the same states and variations of the application in both contexts. Your runs will likely never be 100% perfectly matching, but the closer they are, the better." } ] }, { "type": "heading", "depth": 2, "children": [ { "type": "text", "value": "Results API" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Cypress Accessibility's " }, { "type": "link", "title": null, "url": "/llm/markdown/accessibility/guides/results-api.md", "children": [ { "type": "text", "value": "Results API" } ] }, { "type": "text", "value": " provides rich data related to the outcome of any accessibility report for a Cypress run. It supports adding a step in CI to block pull requests or trigger other side effects. To use the Results API, you'll need to " }, { "type": "link", "title": null, "url": "/llm/markdown/accessibility/guides/results-api.md#Installation", "children": [ { "type": "text", "value": "add a step in your CI process" } ] }, { "type": "text", "value": ". This comes with a helper function which will automatically match your CI build with the related run in Cypress Cloud." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Once you receive the results, you can perform any action you need to. Below are some of the common ones customers have told us about, where the results are compared with expectations." } ] }, { "type": "heading", "depth": 3, "children": [ { "type": "text", "value": "Results API use cases" } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Fail a build based on new failed accessibility rules" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "The Results API provides a combined report with all the failing rules included. This means you can easily prepare a list of known, permitted failing rules to compare with the results from any run, and choose to fail only if a previously passing rule has started to fail. This is described in detail in the guide related to " }, { "type": "link", "title": null, "url": "/llm/markdown/accessibility/block-pull-requests.md", "children": [ { "type": "text", "value": "blocking pull requests and setting policies" } ] }, { "type": "text", "value": "." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "This concept is simple but powerful because it provides extremely high confidence that any failed build is related to a specific recent code, content, or testing change. Especially if there are a high number of existing failures, the actual number of failed elements detected can vary from run-to-run, especially if existing broken components are reused in new ways." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Rules that are fully passing do not have any of these problems, and for an extreme action like failing a pipeline, it's important to have high confidence that a problem is actually a new issues and can be traced to a specific build. With this confidence in place, the existing failed rules can be addressed one by one to get fully passing." } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Fail a build based on deviation from a desired score" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "The Results API includes the specific failed element count and accessibility percentage score for the accessibility report. This makes it easy to set a score threshold and fail a build if the score falls below that expectation. This is especially useful when taking a monitoring approach, or wanting a broader, less specific standard than one based on specific rules." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Changes in percentage score are directly related to changes in the number of elements detected failing accessibility rules throughout the run. Alerting or failing a pipeline for scores going down is a great way to know when you need to jump into Branch Review to understand the underlying causes of the change in score." } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Alert specific teams about failures they own" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "In addition to the consolidated accessibility report, accessibility failures are listed per View in the results API. A View is always related to a specific URL pattern or component test, which means in many projects it is possible to connect accessibility issues in the Results API with the team responsible for solving issues in that part of the application." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "As an example, customers can use this functionality to trigger Slack webhooks that notify a particular channel of the issues in specific areas. The Results API provides direct links to the Views and to each failed rule within that View, making it easy to forward useful links to the correct places, or jump right into debugging from the CI output." } ] }, { "type": "heading", "depth": 2, "children": [ { "type": "text", "value": "Analytics" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Analytics live under the " }, { "type": "link", "title": null, "url": "/llm/markdown/cloud/features/analytics/enterprise-reporting.md#Cypress-Accessibility", "children": [ { "type": "text", "value": "Cypress Accessibility" } ] }, { "type": "text", "value": " tab of " }, { "type": "link", "title": null, "url": "/llm/markdown/cloud/features/analytics/enterprise-reporting.md", "children": [ { "type": "text", "value": "Enterprise Reporting" } ] }, { "type": "text", "value": " in Cypress Cloud, alongside other org-wide metrics on the same page." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "The same aggregates you see in the UI can be pulled programmatically: every chart is backed by data you can " }, { "type": "link", "title": null, "url": "/llm/markdown/cloud/features/analytics/enterprise-reporting.md#Download-data-from-Cloud", "children": [ { "type": "text", "value": "download from Cloud" } ] }, { "type": "text", "value": " or request through the " }, { "type": "link", "title": null, "url": "/llm/markdown/cloud/integrations/data-extract-api.md", "children": [ { "type": "text", "value": "Data Extract API" } ] }, { "type": "text", "value": " (see " }, { "type": "link", "title": null, "url": "/llm/markdown/cloud/features/analytics/enterprise-reporting.md#Retrieve-your-test-data-through-API", "children": [ { "type": "text", "value": "Retrieve your test data through API" } ] }, { "type": "text", "value": " on the Enterprise Reporting page)." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "That API access supports many possible uses, including:" } ] }, { "type": "list", "ordered": false, "start": null, "spread": false, "children": [ { "type": "listItem", "spread": false, "checked": null, "children": [ { "type": "paragraph", "children": [ { "type": "text", "value": "Integrating Cypress Accessibility data with other quality metrics in your internal dashboards" } ] } ] }, { "type": "listItem", "spread": false, "checked": null, "children": [ { "type": "paragraph", "children": [ { "type": "text", "value": "Creating one-off reports about improvements or successes (for example showing the benefits over time that come from the adoption of your design system)" } ] } ] }, { "type": "listItem", "spread": false, "checked": null, "children": [ { "type": "paragraph", "children": [ { "type": "text", "value": "AI workflows: react to trends to find specific runs and drill into views, rules, and elements using " }, { "type": "link", "title": null, "url": "/llm/markdown/cloud/integrations/cloud-mcp.md", "children": [ { "type": "text", "value": "Cloud MCP" } ] }, { "type": "text", "value": " tools and " }, { "type": "link", "title": null, "url": "/llm/markdown/accessibility/work-with-ai-agents.md", "children": [ { "type": "text", "value": "Work with AI agents" } ] }, { "type": "text", "value": "." } ] } ] } ] }, { "type": "heading", "depth": 3, "children": [ { "type": "text", "value": "Example accessibility charts" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "The following screenshots illustrate typical cards on the Cypress Accessibility tab and how you might interpret them." } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Overall score, top projects, and score trend" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "This layout gives a north-star percentage for the filtered scope, a quick comparison of which projects score highest, and trends in topline scores over time. Use Start date, End date, Team, Projects, and Branch filters to zoom from org-wide trends to a single branch." } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Severity breakdown for the selected period" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "The severity breakdown summarizes failed rule categories across the entire set of runs included in your filters." } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Weekly average violations per run by rule" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Tracking the number of failed elements per rule over time helps separate out permanent changes from brief ones." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "If something is merged with new accessibility failures, but the failures are quickly fixed, those are unlikely to appear here - they are blips with little impact." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "On the other hand, when a rule has new failures introduced that are not quickly repaired, that will be clear in this chart - as the `region` rules above shows." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Likewise, when a rule's failures are fixed, a downward line here means the fix \"stuck\" and was picked up consistently in later runs." } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Weekly average failed rules per run by impact" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "This view emphasizes how many distinct rules fail per run on average, stacked by impact. It complements severity counts by showing whether breadth of rule failures is growing or shrinking week over the time period in question." } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Weekly average element violations per run by impact" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Element-level violations by impact highlight volume and volatility: spikes often indicate widespread markup or content issues." } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Enterprise Reporting" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "For more context on the Enterprise Reporting product and other tabs, see " }, { "type": "link", "title": null, "url": "/llm/markdown/cloud/features/analytics/enterprise-reporting.md", "children": [ { "type": "text", "value": "Enterprise Reporting" } ] }, { "type": "text", "value": " and the " }, { "type": "link", "title": null, "url": "/llm/markdown/cloud/features/analytics/overview.md", "children": [ { "type": "text", "value": "Analytics" } ] }, { "type": "text", "value": " overview. For endpoint details and export formats, use the " }, { "type": "link", "title": null, "url": "/llm/markdown/cloud/integrations/data-extract-api.md", "children": [ { "type": "text", "value": "Data Extract API" } ] }, { "type": "text", "value": " documentation." } ] }, { "type": "heading", "depth": 3, "children": [ { "type": "text", "value": "Analytics use cases" } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Track progress and spot trends" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "This level of reporting is ideal for documenting progress and spotting unexpected trends. When you see a project's score dropping over the long term, you can explore further with Branch Review to understand what changes are driving the score." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "You can compare reports before and after any shifts in scores to determine which accessibility issues were introduced." } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Pull high level metrics for AI analysis" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "You can dive deeper with " }, { "type": "link", "title": null, "url": "/llm/markdown/cloud/integrations/cloud-mcp.md", "children": [ { "type": "text", "value": "Cloud MCP" } ] }, { "type": "text", "value": " for your AI agents to look into issue ownerships remediation planning, and configuration." } ] }, { "type": "heading", "depth": 4, "children": [ { "type": "text", "value": "Set goals and minimum standards" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "Seeing all projects in one place gives a good sense of each project's relative level and helps inform what values you might want to use for comparison when implementing a policy using the Results API." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "This also provides useful planning insights, because you can see your historical pace of improvements. If some projects are improving faster than others, you will be able to recognize this and understand how to replicate that success, or create realistic OKRs on a project basis." } ] }, { "type": "heading", "depth": 2, "children": [ { "type": "text", "value": "Combine approaches" } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "As you can see above, there are times you may want to move between multiple systems to track down an issue. For example, if the Results API signals that a build on a feature branch has failed your accessibility standard, you might click through to the run itself, and then compare it with the main branch in Branch Review. That might be the end of the journey, but if the main branch is also failing your policy, you can use high-level analytics to see when the score originally changed, and find the specific cause by comparing two runs from around that time. This helps detect where code was force merged, or when content served in the application had accessibility regression unrelated to code changes." } ] }, { "type": "paragraph", "children": [ { "type": "text", "value": "As you use these tools, remember to hit the \"Submit Feedback\" button any time you see something that could be improved to help you understand what's important and what's changing." } ] } ] }, "token_estimate": 2593 }