Accessibility checks for changed website pages in local CI

@axe-core/playwright gives the full axe rules in a test you control. Its lead is narrow; keep Cypress or Puppeteer if you already use them. Pa11y CI fits teams wanting configuration instead of test code, with a status check in front.

PRACTICAL PICKSPreferredPickOur Preferred PickChanged pages in local CI$0

@axe-core/playwright

Deque’s free package runs the full axe-core rules inside Playwright Test, after any clicks or sign-in you script, and fails the test when a rule fails.

Its lead is narrow: 4.03 out of 5 against wick-a11y’s 3.98; you keep test code and add a response-status check.

Why @axe-core/playwright is our Pick
Newcomer assessment

No Rising Star selected

The November 6 search found no newer tool with the required release, Windows/macOS, rendered-page gate and safety evidence established.

AllyCat checks source files; Includa is unfinished alpha software; other newer candidates lack required eligibility evidence.

Who is this article for? Small website teams checking named changed routes locally and in CI before publishing, while keeping manual accessibility review in the workflow.

About 31 min read

What Preferred Pick, Rising Star and Ready to Rock mean

The quick list

Start with @axe-core/playwright for a new suite. Keep Cypress or Puppeteer if you already use them; choose Pa11y CI for a configuration file, with a status check in front.

These eight packages have no software charge for local checks, checked November 6, 2026. Existing computer and CI capacity are assumed. Licenses: MPL-2.0 for Deque’s axe packages, Apache-2.0 for Playwright, Lighthouse CI and IBM’s checker, LGPL-3.0-only for Pa11y CI, MIT for cypress-axe and wick-a11y. Cypress Cloud is optional. We could not confirm current prices for every alternative. Your costs are CI time, browser downloads and the time to write and maintain the checks. IBM’s package sends install-time telemetry unless you opt out.

The comparison at a glance

Compare what fails the build, which rules run, what you can do before a scan and how current each project is.

Start with @axe-core/playwright; its lead over wick-a11y is narrow. Cypress and Puppeteer integrations suit existing suites. Other choices follow their fit for changed pages in local CI. All tools remain selectable.

Automated accessibility checks for named local website pages
Feature@axe-core/playwrightwick-a11ycypress-axe@axe-core/puppeteerLighthouse CIIBM accessibility-checkerPa11y CIaxe-core CLI
Editorial score out of 54.03 · Preferred Pick, narrow lead3.983.883.833.803.753.683.38
First year / three years, software charges$0 / $0$0 / $0$0 / $0$0 / $0$0 / $0$0 / $0$0 / $0$0 / $0
Best fitPreferred Pick for a new suite; full axe rules after any interaction; narrow leadCypress teams wanting HTML reports and annotated screenshots; narrow second placeTeams already using Cypress; keep test code and explicit failure checksTeams already using Puppeteer; add assertions and error handlingTeams already using Lighthouse CI for performance that want missing pages to failA second opinion from IBM’s own rules, with baselines and spreadsheet reportsTeams wanting a configuration file and readable output, plus a separate status checkA quick one-off axe check where a matching ChromeDriver is already managed
How you set it upTest file in JavaScript or TypeScript; Playwright can start your local site firstInstall plugin and Cypress peers; register tasks and support commands; write testsInstall plugin and peers; add test commands and a logging taskInstall Puppeteer and the integration; write a test and report handlingConfiguration file, then collect and assert; built-in server for a static build folderText file listing pages, optional YAML settings, one commandJSON or JavaScript configuration and one command; Chrome comes with PuppeteerURLs on the command line; ChromeDriver must match your Chrome version
Page that returns an error statusFails only if your test checks the response status; navigation does not stop on 404 or 500Check the navigation response and expected page content in your Cypress testCheck navigation and expected page content in the Cypress testCheck the navigation response in your script; logging alone is not a gateCollection fails: Lighthouse stops on 4xx and 5xx responses. An earlier check on October 1, 2026 stopped on a 404Not documentedScanned as if it were your page; add a status check first. An earlier check on October 1, 2026 passed a missing pageNo status check; fails only if the error page itself breaks a rule
Rulesaxe-core 4.14: 91 WCAG A and AA and best-practice rules by defaultaxe-core through cypress-axe; configure all required rules and severitiesaxe-core supplied as a peer dependency; choose a supported current versionaxe-core 4.14, like the PickA subset of axe-core; 0.16.0 upgrades to Lighthouse 13.6.0; older audit counts do not describe itIBM’s own engine; IBM_Accessibility policy by defaultHTML_CodeSniffer by default; axe-core 4.13 instead or as wellaxe-core 4.14, the same rules as @axe-core/playwright
Before the scanAny scripted step: click, type, sign in; Chromium, Firefox or WebKitCypress interactions before scanning; one accessibility analysis per testScan after Cypress interactions; check the state visitors useAny interactions you script in Puppeteer before analyze()A Puppeteer script before collection, mainly for signing inNone from the command line; use its test-framework integrationsPer-page actions such as click, type and wait; viewport and headersA fixed delay after load; no interaction
Release record (dates checked stated below)4.14.0, October 15, 2026; Playwright Test 1.64.1, October 14, 20263.0.2, August 16, 2026; Cypress 15.10+ handling; checked November 6, 20261.7.0, August 29, 2025; Cypress 15 support; checked November 6, 20264.14.0, October 15, 2026; checked November 6, 20260.16.0, November 2, 2026; Lighthouse 13.6.0; checked November 6, 20264.0.34, September 8, 20265.0.0, October 5, 2026, with Pa11y 10.0.0 from August 28, 20264.14.0, October 15, 2026
Price and license$0; MPL-2.0, with Playwright under Apache-2.0$0; MIT plugin and local Cypress App; Cloud optional$0; MIT plugin, local Cypress App; Cloud is optional$0; MPL-2.0 integration; no paid runner service required$0; Apache-2.0$0; Apache-2.0$0; LGPL-3.0-only$0; MPL-2.0
Reader reviewsNo reviews yet. Write a reviewNo reviews yet. Write a reviewNo reviews yet. Write a reviewNo reviews yet. Write a reviewNo reviews yet. Write a reviewNo reviews yet. Write a reviewNo reviews yet. Write a reviewNo reviews yet. Write a review

Software charges checked November 6, 2026 in project documentation; existing computer and CI capacity assumed. cypress-axe, Deque’s Puppeteer integration and Lighthouse CI releases were checked that day. wick-a11y’s README and release listing were checked November 6. Other feature evidence and download counts remain dated October 19, 2026.

No automated check certifies accessibility. Deque says axe-core finds about 57% of WCAG issues, by its own count of issues in its audits; in the UK Government Digital Service’s audit of 142 planted barriers, no single tool found more than 40%. W3C WAI says tools cannot check every accessibility aspect and can mislead. A page that returns a normal status with “not found” text passes every status check here.

All features and sources 13 criteria

Open the groups that matter to you. They compare the same selected tools. “Planned” means future work, not an available feature. Sources include the published conditions.

Build gate and findings

Show 5 criteria
Build gate and findings
Feature@axe-core/playwrightwick-a11ycypress-axe@axe-core/puppeteerLighthouse CIIBM accessibility-checkerPa11y CIaxe-core CLI
Page that returns an error statusFails only if your test checks the response status; navigation does not stop on 404 or 500Check the navigation response and expected page content in your Cypress testCheck navigation and expected page content in the Cypress testCheck the navigation response in your script; logging alone is not a gateCollection fails: Lighthouse stops on 4xx and 5xx responses. An earlier check on October 1, 2026 stopped on a 404Not documentedScanned as if it were your page; add a status check first. An earlier check on October 1, 2026 passed a missing pageNo status check; fails only if the error page itself breaks a rule
Allowing known issuesExclude elements, disable rules or snapshot a fingerprint of rule and elementsRule, context and severity filters; warning-only settings do not fail the buildScope scans, configure axe, filter impact; skipFailures must stay falseExclude elements or disable rules; add your own baseline comparisonPer-audit assertions; maxLength counts items, not which elements failBaseline files; a scan fails when results differ from the baselineIgnore issue codes or set a threshold; keep the threshold at 0 for nowDisable rules or exclude parts of the page
Open reports affecting the gate4.14.0 can hang on some sandboxed iframes; a development-branch fix is not yet releasedonlyWarnImpacts and impact filters can leave issues as warnings; keep every required severity failingskipFailures logs violations without failing; filters can omit issuesThe example logs results and catches errors; add a failing assertionA fixed error and a new one cancel out under count assertions (1145)Not reviewed in depth; README notes a Jest parallel-run error and Content Security Policy blockingUnloaded pages pass with a threshold of 1 or more (368); pages skipped after a browser crash (282)Chrome tab crash about four minutes into a CI scan (1338)
Rulesaxe-core 4.14: 91 WCAG A and AA and best-practice rules by defaultaxe-core through cypress-axe; configure all required rules and severitiesaxe-core supplied as a peer dependency; choose a supported current versionaxe-core 4.14, like the PickA subset of axe-core; 0.16.0 upgrades to Lighthouse 13.6.0; older audit counts do not describe itIBM’s own engine; IBM_Accessibility policy by defaultHTML_CodeSniffer by default; axe-core 4.13 instead or as wellaxe-core 4.14, the same rules as @axe-core/playwright
Where you see the problemFailed test lists violations; full results can be attached to the HTML reportCypress log, terminal table and HTML report with element guidance and annotated screenshotsCypress log; terminal detail needs callback and cy.taskResults object; format and retain it yourselfTerminal names the audits you assert; elements are in the saved HTML or JSON reportJSON, CSV, HTML or Excel reportsTerminal: element, code snippet and fix advice; JSON and other reportersTerminal: rule, link and elements; JSON file on request
Sources for build gate and findings

Checked October 19, 2026 in Playwright’s accessibility guide and navigation reference, the Pa11y CI README and Pa11y navigation code, Lighthouse’s error definitions and audit configuration, IBM’s configuration guide, the axe-core CLI README and each project’s issue tracker. The October 1, 2026 observations come from an earlier hands-on check made before our research-only method.

Pages, setup and upkeep

Show 8 criteria
Pages, setup and upkeep
Feature@axe-core/playwrightwick-a11ycypress-axe@axe-core/puppeteerLighthouse CIIBM accessibility-checkerPa11y CIaxe-core CLI
First year / three years, software charges$0 / $0$0 / $0$0 / $0$0 / $0$0 / $0$0 / $0$0 / $0$0 / $0
Before the scanAny scripted step: click, type, sign in; Chromium, Firefox or WebKitCypress interactions before scanning; one accessibility analysis per testScan after Cypress interactions; check the state visitors useAny interactions you script in Puppeteer before analyze()A Puppeteer script before collection, mainly for signing inNone from the command line; use its test-framework integrationsPer-page actions such as click, type and wait; viewport and headersA fixed delay after load; no interaction
How you set it upTest file in JavaScript or TypeScript; Playwright can start your local site firstInstall plugin and Cypress peers; register tasks and support commands; write testsInstall plugin and peers; add test commands and a logging taskInstall Puppeteer and the integration; write a test and report handlingConfiguration file, then collect and assert; built-in server for a static build folderText file listing pages, optional YAML settings, one commandJSON or JavaScript configuration and one command; Chrome comes with PuppeteerURLs on the command line; ChromeDriver must match your Chrome version
Release record (dates checked stated below)4.14.0, October 15, 2026; Playwright Test 1.64.1, October 14, 20263.0.2, August 16, 2026; Cypress 15.10+ handling; checked November 6, 20261.7.0, August 29, 2025; Cypress 15 support; checked November 6, 20264.14.0, October 15, 2026; checked November 6, 20260.16.0, November 2, 2026; Lighthouse 13.6.0; checked November 6, 20264.0.34, September 8, 20265.0.0, October 5, 2026, with Pa11y 10.0.0 from August 28, 20264.14.0, October 15, 2026
Security advisory evidencePlaywright: CVE-2025-59288, fixed in 1.55.1; none for @axe-core/playwrightNo dependency audit established; keep Cypress, axe and browser dependencies currentNo published plugin advisory found November 6; dependencies not exhaustively checkedNo published integration advisory found November 6; keep the browser currentNone listedNone listedNone listedNone listed
Price and license$0; MPL-2.0, with Playwright under Apache-2.0$0; MIT plugin and local Cypress App; Cloud optional$0; MIT plugin, local Cypress App; Cloud is optional$0; MPL-2.0 integration; no paid runner service required$0; Apache-2.0$0; Apache-2.0$0; LGPL-3.0-only$0; MPL-2.0
What leaves your machineNo account or upload documented; browsers download during setupLocal report files include page screenshots and offending elements; protect retained reportsKeep the local runner workflow; cloud recording is optionalNo hosted scan required; keep reports localNothing with the filesystem target; the public-storage target makes reports public for days to weeksInstall-time IBM Telemetry, with a documented opt-out; rules fetched during runs by defaultNo account or upload documentedNo account or upload documented
Weekly npm downloads, October 12 to 18, 2026About 9.4 millionNot checkedAbout 357,000Not checkedAbout 1.2 million, including performance useAbout 14,000About 63,000About 67,000
Sources for pages, setup and upkeep

Release dates and licenses from each project’s release page and npm record, incumbent advisories from OSV, download counts from the npm downloads API, all on October 19, 2026. The cypress-axe advisory cell uses the GitHub advisory search and project security page checked November 6, 2026; the Puppeteer cell uses the Deque integration security page checked November 6. These bounded searches do not establish dependency safety. No separate advisory or dependency audit was established for wick-a11y; its cell records that limit. Data handling from Lighthouse CI’s services disclaimer and IBM’s package documentation. Downloads are a rough signal of use, not a count of teams.

What matters when choosing

@axe-core/playwright, run inside Playwright Test, is our Practical Pick for a small website team that wants its changed pages checked automatically, on a laptop and in CI, before it publishes. Deque’s free package runs the full axe-core rule set on any page Playwright opens, and a failed check fails the test like any other assertion. Because the check is ordinary test code, you can open a menu, fill in a form or sign in before scanning, and Playwright can start your local site before the run. Version 4.14.0, released October 15, 2026, uses axe-core 4.14. Among the five packages whose downloads we checked on October 19, it was by far the most used: npm counted about 9.4 million downloads of the package in the week to October 18, 2026, against about 67,000 for axe-core CLI and 63,000 for Pa11y CI. Downloads are a rough signal of use, not a count of teams. Playwright accessibility testing, releases, npm weekly downloads

The most important compromise is that you write and keep a small test file rather than a configuration file. Playwright also does not fail a missing page by itself: its documentation says page.goto does not throw when the server returns a 404 or 500, so add one line that checks the response status before the scan. For known problems you cannot fix yet, Playwright’s guide suggests recording a fingerprint of each one, its rule and affected elements, so a new error on another element still fails the test. Page navigation, known issues

The lead is narrow: @axe-core/playwright scores 4.03 out of 5, ahead of wick-a11y at 3.98, cypress-axe at 3.88 and @axe-core/puppeteer at 3.83. wick-a11y adds HTML reports and screenshots to Cypress checks. Choose a Cypress or Puppeteer integration if your team already uses that runner; Playwright remains the clearest starting point for a new suite.

If you will not write test code, choose Pa11y CI and add a status check before it scans. It reads a list of pages or a sitemap from one configuration file, can click and type before scanning, and prints each problem with the element and a suggested fix. But Pa11y tests whatever page comes back, including a “not found” page, and two open bugs can let a page that never ran pass the build. If you already use Lighthouse CI for performance budgets, it is the one tool here that fails a missing page without extra work, but it runs a subset of axe’s rules. Version 0.16.0, released November 2, 2026, upgrades to Lighthouse 13.6.0. IBM’s accessibility-checker is the choice for a second opinion from a different rule engine; axe-core CLI is the quickest one-off check if you already manage a matching ChromeDriver. Pa11y CI, Lighthouse CI releases

Every tool here is limited by what automation can find. Deque says axe-core finds “on average 57% of WCAG issues automatically”. Its own March 2021 study gave that figure from more than 2,000 of its audits, counted by number of issues rather than by WCAG success criteria. When the UK Government Digital Service planted 142 barriers on one page, the best single tool found 40%, axe 29% and HTML_CodeSniffer, Pa11y’s default engine, 20% (results last updated April 2018). W3C’s accessibility initiative says tools “can not determine accessibility, they can only assist in doing so.” A passing run means no automated rule failed on that page in that state. Keyboard use, screen-reader output, wording and most problems in context still need a person. axe-core README, Deque study, March 10, 2021, GDS tool audit, W3C WAI, updated May 13, 2024

No Rising Star selected. The November 6, 2026 search found no newer tool with all the required release, Windows/macOS availability, rendered-page gate and safety evidence established. AllyCat checks source files rather than rendered routes, Includa is explicitly unfinished alpha software, and wick-a11y was already public before the two-year newcomer window. AllyCat, Includa, wick-a11y.

We could not confirm current prices for every alternative.

Disclosure: Practical Picks articles are researched and written with AI, currently Anthropic’s Claude. Playwright documents test agents that run with Claude Code, and Deque advertises its paid axe MCP Server for use with Claude Code. We have no relationship with Microsoft, Deque or Anthropic over this article, and these Claude links carried no weight in our judgment. Playwright test agents, axe MCP Server

@axe-core/playwright

Preferred Pick · changed pages in local CI

$0MPL-2.0 · runs inside Playwright Test (Apache-2.0)

Choose @axe-core/playwright when someone on the team can keep a small test file. Check each page’s HTTP status in the test, then scan with the full axe rules in the state your visitors see.

Reasons to choose it

  • Full axe-core 4.14 rules, current as of October 2026
  • Scan after clicks, form entries or signing in
  • Playwright can start your local site before the run

What to watch

  • You write and maintain test code
  • A 404 or 500 does not fail the test unless you check the status
  • Published 4.14.0 can hang on some sandboxed iframes; the fix is on the development branch

Playwright’s own accessibility guide shows the pattern: open the page, run new AxeBuilder({ page }).analyze() and expect the list of violations to be empty. You can limit a scan to part of the page, exclude elements, restrict rules to WCAG tags such as wcag2aa, disable specific rules, and attach the full results to Playwright’s report so the person fixing the page sees every affected element. A shared fixture keeps those settings the same across test files. The guide itself says automated tests “can detect some common accessibility problems such as missing or invalid properties” and “cannot detect all types of WCAG violations”. Accessibility testing guide

What makes it the strongest fit is control over the page state. Many accessibility failures appear only after an interaction: an open menu, a dialog, a form showing errors. In a test you reach that state with ordinary Playwright steps and then scan, in Chromium, Firefox or WebKit. Playwright’s webServer option can “launch a local dev server before running your tests”, so the same command works on a laptop and in CI. Deque’s package is free under the MPL-2.0 license and Playwright under Apache-2.0; neither needs an account. Playwright Test 1.64.1 was released October 14, 2026. Web server, package record, Playwright Test release

Read the full @axe-core/playwright analysis and sources

Add the status check yourself. Playwright documents that navigation “will not throw an error when any valid HTTP status code is returned by the remote server”, including 404 and 500, and returns the response so you can read response.status(). A test that skips this check will scan a “not found” page and may pass. A page that returns 200 with “not found” text passes this check too, so assert something that only the real page contains. page.goto

Choose another tool if nobody on the team will write or review test code. Open reports in Deque’s tracker include one, from October 8, 2026, where a scan hangs on a sandboxed iframe that does not allow scripts until the test times out, which fails the test but hides the cause; a fix was merged into the development branch on October 27, but the November 6 release page still lists 4.14.0, which predates it. Release record, checked November 6, 2026 OSV lists one advisory for the playwright package, fixed in 1.55.1 in October 2025: macOS scripts that reinstall branded Chrome and Edge downloaded installers without checking the server’s certificate (CVE-2025-59288). The same axe rules are available in Cypress through the third-party cypress-axe package and in Deque’s Puppeteer and Selenium packages, if your team already uses one of those. Issue 1469, advisory

Used @axe-core/playwright? Share your experience to help other readers choose.

wick-a11y

Existing Cypress tests · detailed reports

$0MIT plugin · local runner

Choose wick-a11y if your Cypress team wants HTML reports with element guidance and annotated screenshots. Keep every required severity failing.

Reasons to choose it

  • HTML reports and annotated screenshots
  • Scan after Cypress interactions
  • Local runner; Cloud optional

What to watch

  • One accessibility analysis per test
  • Warning and impact settings can weaken the gate
  • You maintain tasks, configuration and test code

wick-a11y runs axe through cypress-axe after the interactions in your Cypress test, then adds a terminal table and an HTML report with affected elements, fix guidance and annotated screenshots. The MIT plugin needs no paid Cypress Cloud subscription. It scores 3.98 out of 5 against the Pick’s 4.03: a narrow gap, with stronger ready-made reports than plain cypress-axe but the same need to maintain browser tests. Choose it when your team already uses Cypress and wants reports it can share with the people fixing the page. README, checked November 6, 2026

Register the plugin’s tasks and support commands, visit each changed route in a test, and scan after the relevant interaction. Keep all required severities in the failing set: onlyWarnImpacts produces warnings, and impact filters can omit problems. You still need to check that the intended page loaded. The plugin allows only one accessibility analysis per test, so use separate tests for different states. Its reports include screenshots and page elements; protect them if your pages contain private data. Configuration and limits

Read the full wick-a11y analysis and sources

Version 3.0.2 is listed on August 16, 2026; the current README documents Cypress 15.10+ configuration and compatibility differences for older versions. We did not establish a complete dependency security audit. Its additional report detail does not establish a higher detection rate than the underlying axe engine. It was already public before November 2024, so it is not a newcomer for this comparison. Release listing and README, checked November 6, 2026

Used wick-a11y? Share your experience to help other readers choose.

cypress-axe

Existing Cypress tests

$0MIT plugin · local runner

Keep Cypress if your team already uses it. Run axe after the interactions that matter, keep skipFailures false and add terminal reporting.

Reasons to choose it

  • Full axe checks in Cypress tests
  • Scan after page interactions
  • Local runner; paid Cloud is optional

What to watch

  • You maintain test code
  • Detailed terminal output needs a callback and logging task
  • Filters and skipFailures can hide violations

Choose cypress-axe if your changed-page tests already run in Cypress. The MIT-licensed plugin injects axe into a page after cy.visit(), then cy.checkA11y() asserts that no violations were found. You can scan again after opening a menu or submitting a form, scope the check to elements and select impact levels. Cypress’s own guide documents this approach separately from its paid cloud accessibility service. The local plugin does not require that service. Plugin README, Cypress guide, pricing, checked November 6, 2026.

It scores 3.88 out of 5, close to the Pick. You still maintain test code, and terminal reports with detailed violations need a callback and a cy.task logging task. Leave skipFailures false: setting it true logs violations while allowing the test to pass. Review filters and navigation assertions so an omitted check cannot hide a regression. Version 1.7.0, released August 29, 2025, adds Cypress 15 support. No independent current detection comparison establishes an accuracy advantage over the Pick’s axe rules. Releases, checked November 6, 2026.

Used cypress-axe? Share your experience to help other readers choose.

@axe-core/puppeteer

Existing Puppeteer scripts

$0MPL-2.0 · assertions supplied by your test

Keep Puppeteer if it already opens your changed routes. Assert that violations are empty, fail on scan errors and check navigation responses.

Reasons to choose it

  • Same axe generation as the Pick
  • Script page interactions before scanning
  • Scope scans and configure rules

What to watch

  • Logging results alone does not fail CI
  • You add report formatting and retention
  • Some sandboxed frames need the unreleased fix

Deque’s free Puppeteer integration runs axe on a Page or Frame you already control. Choose it when Puppeteer already opens the changed routes and reaches the states you need. It supports selectors, exclusions, tags and rule configuration; version 4.14.0 uses the same axe generation as the Pick. API guide, releases, checked November 6, 2026.

It scores 3.83 out of 5. The example logs results and catches errors; it is not by itself a failing CI test. Assert that violations are empty, make scan errors fail the process, check navigation responses and retain the report. A Content Security Policy can require bypassing it before navigation to inject axe. As with the Pick, the development branch contains an iframe fix that the latest published release does not yet include. The extra gate and reporting code is the main reason to start with Playwright Test for a new suite.

Used @axe-core/puppeteer? Share your experience to help other readers choose.

Lighthouse CI

Also consider · already using Lighthouse CI

$0Apache-2.0 · bundles Lighthouse 13.6.0

Add accessibility assertions to Lighthouse CI if you already run it for performance. It fails a page that returns an error status with no extra step, but assert named audits and keep reports on your own disk.

Reasons to choose it

  • Stops on 4xx and 5xx responses by itself
  • Category and per-audit assertions
  • Built-in server for a static build folder

What to watch

  • Runs a subset of axe-core’s rules
  • Version 0.16.0 upgrades to Lighthouse 13.6.0; older audit counts no longer apply
  • The public-storage upload target makes reports public

Lighthouse CI collects Lighthouse reports for a list of URLs, or for HTML files it finds in a static build folder that it serves itself, and then checks them against assertions. Lighthouse treats a page that answers with a 4xx or 5xx status as a failed load, and its command line exits with an error, so lhci collect stops the build before any score is calculated. In an earlier hands-on check on October 1, 2026, before we adopted our research-only method, Lighthouse CI 0.15.1 stopped on a missing local page with “Status code: 404”. One exception: its static server’s single-page-app option serves index.html for unknown paths, so a missing route then loads normally. Lighthouse error definitions, CLI exit on runtime errors, configuration, earlier check record

Its accessibility checks are a subset of axe-core’s. In the older Lighthouse 12.6.1 used for our October research, the accessibility category had 63 automated audits, against 91 WCAG and best-practice rules that axe-core 4.14 runs by default. The category score is a weighted average with no partial credit, and audits with zero weight, such as empty-heading in that version, do not affect it. Assert those audits individually if they matter to you. A category assertion on its own prints only the score; in the October 1 check, the terminal gave the failing page’s score, 0.72, and nothing more until we added assertions for named audits, and the affected elements appeared only in the saved report. A 2019 demonstration built a deliberately inaccessible page with a perfect Lighthouse score. Lighthouse 12.6.1 configuration, axe-core rules, scoring, updated October 22, 2025, matuzo.at, May 31, 2019

Read the full Lighthouse CI analysis and sources

Lighthouse CI 0.16.0, released November 2, 2026, upgrades to Lighthouse 13.6.0, so the long gap between releases is over. That improves its case for teams already using it, but its narrower accessibility coverage and report workflow still make it a different choice from the Pick. The older 63-audit count above describes 12.6.1, not the new release. The August report about count assertions remains relevant: a fixed error and a new one can cancel out when only the number of failing items matters. Releases, checked November 6, 2026, issue 1145, checked October 19, 2026

It is free under the Apache-2.0 license. Keep reports on your own disk with the filesystem upload target. The quick-start temporary-public-storage target makes reports “available to anyone on the internet with the link”, is run by a third party, Eris Ventures LLC, and keeps them for three days to five weeks. Upload targets, services disclaimer

Used Lighthouse CI? Share your experience to help other readers choose.

IBM accessibility-checker

Second opinion · a different rule engine

$0Apache-2.0 · install-time telemetry

Consider IBM’s accessibility-checker beside an axe-based tool when you want a second rule set, baselines and spreadsheet reports. Plan for its telemetry opt-out and network access during runs.

Reasons to choose it

  • IBM’s own rules, independent of axe
  • Baselines fail a scan only when results change
  • JSON, CSV, HTML and Excel reports

What to watch

  • Installing it agrees to IBM Telemetry unless you opt out
  • Fetches its rules during runs by default
  • Error-page handling not documented

IBM’s accessibility-checker package, part of its Equal Access Toolkit, uses IBM’s own rule engine rather than axe, so it is a genuinely different second opinion. Its achecker command scans a text file listing pages, files or folders and writes JSON, CSV, HTML or Excel reports. By default it fails on violations and on potential violations that need review, and its source sets exit code 1 when any page fails or errors. Baseline files record known results, so later scans fail only when the results change. The same package works inside Playwright, Puppeteer, Selenium and Jest tests. Version 4.0.34 was released September 8, 2026, and IBM published releases in every month from June to September 2026. It is free under the Apache-2.0 license. Package README, configuration, command-line source, releases

Know what it sends and fetches. The README says the package uses IBM Telemetry and that “by installing this package as a dependency you are agreeing to telemetry collection”, with an opt-out documented by IBM. By default it uses the latest rule archive, loaded during the run from a content delivery network or IBM’s rule server, so a CI job needs network access, results can change when IBM updates its rules, and a page’s Content Security Policy can block the engine. A “versioned” setting pins the rules to those current when your installed version was released. Its documentation does not say how it treats a page that returns an error status. Configuration and telemetry, known issues

Read the full IBM accessibility-checker analysis and sources

Sparkbox’s February 2024 review of IBM’s browser extension, which uses the same engine, found 12 of 23 planted problems, including 9 of the 13 it expected a tool to find, with “at least one false positive”. That is one page and an older version, not a current detection rate. Choose another tool if you want page interactions without writing test code, or if your CI cannot reach the internet. Sparkbox review

Used IBM accessibility-checker? Share your experience to help other readers choose.

Pa11y CI

Also consider · no test code

$0LGPL-3.0-only · one configuration file

Choose Pa11y CI when you want a configuration file instead of test code. Request every page first and stop if any returns an error, keep the threshold at 0, and include one page with a known error.

Reasons to choose it

  • URL list or sitemap, with per-page click, type and wait actions
  • HTML_CodeSniffer and axe-core rules, separately or together
  • Readable terminal output with element and fix advice

What to watch

  • Scans a “not found” page as if it were yours
  • Open bugs let unloaded pages pass with a threshold of 1 or more
  • Pages after a browser crash can go unreported

Pa11y CI 5.0.0, released October 5, 2026, runs Pa11y 10 against a list of URLs or every address in a sitemap, with options to rewrite sitemap addresses to your local server and to exclude some. Each page can have its own actions, such as click element #tab-1 or wait for path to be /myaccount, its own viewport and headers, and a screenshot. It runs HTML_CodeSniffer by default and can run axe-core 4.13 instead or as well, which gives two independent rule sets in one pass. Exit codes are documented: 0 when pages pass, 1 for a technical fault such as a bad configuration, 2 when at least one page fails. It is free under the LGPL-3.0-only license, and Pa11y 10 needs Node.js 22.13 or later. Pa11y CI README, Pa11y 10 release, actions

Pa11y does not check whether the page loaded. Its code opens the address and tests whatever the browser shows, without reading the HTTP status, and a contributor answering a 2020 request wrote that “it sure doesn’t look like there is a facility for this in pa11y right now.” In an earlier hands-on check on October 1, 2026, before we adopted our research-only method, Pa11y CI 4.1.1 tested a local server’s 404 page as if it were the intended page, reported no errors and exited 0. Before the scan, request every target address and stop the build if any returns an error status. Navigation code, issue 538, earlier check record, October 1, 2026

Read the full Pa11y CI analysis and sources

Keep the threshold at its default of 0 for now. A maintainer reported on October 6, 2026 that with --threshold set to 1 or more, a run in which pages failed to load but had no accessibility issues exits 0; a fix was proposed but not merged when we checked. A separate open report, from September 2025, says that if the browser crashes between pages, the remaining pages are neither tested nor reported; turning off useIncognitoBrowserContext avoided it for the reporter. Add one page with a known error to your list, so a run that silently stops testing still fails. Issue 368, issue 228, issue 282

In the same October 1 check, Pa11y CI 4.1.1 named all three planted defects on a small test page, an unlabeled email field, an image without alternative text and low-contrast text, with the element, the code snippet and the measured contrast ratio. That matches what its documentation describes, but one small page says nothing about how much it finds on real sites. Pa11y CI and Pa11y have each had three releases in 2026. Earlier check record, releases

Used Pa11y CI? Share your experience to help other readers choose.

axe-core CLI

Quick one-off axe check

$0MPL-2.0 · needs a matching ChromeDriver

Use axe-core CLI for a quick check of a few URLs with the full axe rules when your machine or CI image already keeps ChromeDriver in step with Chrome. Add a status check before it.

Reasons to choose it

  • Same axe-core 4.14 rules as the Pick
  • Several URLs in one command; --exit fails the run
  • Rule links and elements in the terminal

What to watch

  • ChromeDriver must match your Chrome version
  • No interaction before the scan; not a crawler
  • Does not read the HTTP status

Deque’s command line takes one or more URLs, runs axe-core and prints each rule that failed with a link and the affected elements; --exit makes it return 1 when any rule fails, and results can be saved as JSON. Version 4.14.0, released October 15, 2026, runs axe-core 4.14, the same rules as the Pick. Its README says it “is not a crawler” and suggests a test-framework integration if you find yourself checking dozens of pages. It cannot click or type before scanning; it can only wait a fixed time after load. It is free under the MPL-2.0 license; Deque’s terms for its free browser extension do not apply to it. CLI README, releases

It drives Chrome through ChromeDriver, and the two must match. Version 4.14.0 depends on ChromeDriver 152, so if your Chrome is a different major version you need to install a matching driver, as the README describes, or point --chromedriver-path at one. In an earlier hands-on check on October 1, 2026, before we adopted our research-only method, axe-core CLI 4.13.0 would not start until a matching ChromeDriver was added; it then reported all three planted defects with selectors and rule links. CLI README, package record, earlier check record

Read the full axe-core CLI analysis and sources

It does not read the HTTP status either: it loads pages through WebDriver, which does not report one. In the October 1 check it failed a missing page only because the server’s error page itself broke two accessibility rules; a well-built error page could pass. Add a status check before it. An open report from May 2026 describes Chrome crashing about four minutes into a scan on GitHub Actions; the crash stopped the run rather than passing it. Page loading code, issue 1338

Used axe-core CLI? Share your experience to help other readers choose.

Which product are we missing?

Know a product that belongs in this article? Tell us what it does differently and why it deserves a closer look. Newcomers welcome.

Readers and product teams can suggest candidates. Suggestions inform our research; they do not guarantee inclusion or a recommendation.

Suggest a product

Opens an email draft with this article attached as a link. Nothing is sent until you send it.

No email app? Send the product name, website, and this article’s title to admin@practicalpicks.guide.

What people are saying

No Practical Picks reader reviews are available yet for these tools. The most useful public evidence is in the projects’ issue trackers, and it concentrates on the same point: a build should fail when a page was not really checked. Pa11y CI’s maintainers have open bugs for pages that fail to load but still let the build pass when a threshold is set, and for pages skipped after a browser crash. A Lighthouse CI user showed in August 2026 that a count-based assertion can pass when one known error is fixed and a new one appears; Playwright’s guide suggests recording known issues by rule and affected elements instead. Pa11y CI issue 368, issue 282, Lighthouse CI issue 1145, Playwright guide

Reports about the engines themselves are narrower. A July 2026 report against Deque’s Selenium package describes a contrast failure missed at a narrow window width in Chrome 151 because the text was treated as partly covered; it is open and unconfirmed by Deque. Independent reviews by Sparkbox, a web development firm, found 10 of 13 tool-detectable problems with the axe extension and no false positives in December 2020, and 9 of 13 with IBM’s extension in February 2024. These are single-page results from different years, not current detection rates, and a reported failure mode does not show how often it happens. Issue 1403, Sparkbox axe review, Sparkbox IBM review

How we researched

This is public-source research. For this review we did not install or run any tool, scan any page, create accounts or contact a maker. No maker supplied access or payment, and our links carry no affiliate tracking. We first compared Lighthouse CI, Pa11y CI and axe-core CLI on October 1, 2026, including a small hands-on check described below. On October 19, 2026 we rechecked documentation, release records, source code, issue trackers, security advisories and licenses, added two tools, and made the decision again from public evidence.

We wrote this for a small website team with a local preview and a CI pipeline that wants a list of changed pages checked automatically before publishing, and wants the build to fail for the right reasons. The fixed weights are gate reliability, including pages that do not load (25%); actionable findings (20%); page states and routes (15%); setup and upkeep (15%); project health and security (15%); and software cost (10%). The final November 6, 2026 ordering starts with @axe-core/playwright (4.03), wick-a11y (3.98), cypress-axe (3.88) and @axe-core/puppeteer (3.83). Pa11y CI (3.68) remains the configuration-first alternative, while Lighthouse CI (3.80) suits an existing performance pipeline and stops on missing pages. The scores are editorial judgments from the documented configurations, not measured detection rates.

We compared the Cypress and Puppeteer integrations because setup, assertions and reports matter even with the same axe engine. Other scored options were Salesforce sa11y’s WebdriverIO integration (3.80), @axe-core/webdriverjs and @axe-core/webdriverio (both 3.75), A11yEco CLI (3.53), Lighthouse CLI with assertions (3.48), accessio-scan (3.40), Unlighthouse (3.38), and AccessLint CLI (3.28). An unknown first release prevents a newcomer award, but does not remove a product from this comparison. WebDriver tools require driver management; bare Lighthouse needs assertion code; Unlighthouse emphasizes whole-site category budgets. Salesforce sa11y, WebdriverIO integration, Lighthouse, Unlighthouse, A11yEco CLI, accessio-scan, AccessLint, checked November 6, 2026.

Component tools such as jest-axe, vitest-axe and Storybook’s accessibility add-on do not replace a list of changed website routes. Storybook can run real-browser checks; its component focus, rather than a simulated browser, is the reason it is outside this comparison. GitHub’s scanner is an issue-filing GitHub Action, and paid hosted monitoring is a different workflow from the account-free local checks considered here.

For newer tools, we revisited the October 19 candidates and checked project documentation, the GitHub accessibility-testing directory, repository search and W3C’s tools overview on November 6, 2026. npm search and version histories were inaccessible. AccessLint’s explicit macOS CLI support, accessio-scan’s platform and independence evidence, and A11yEco’s first release and CLI safety evidence were not established; no newcomer qualified. Omi A11Y and a11y-ai lacked individual product evidence. Happo needs hosted credentials; MagentaA11y is documentation. UUV, current QualWeb, sitespeed’s axe plugin, Accessibility Insights Action, be-a11y and a11ymcp remain unscored because their current documentation did not establish a complete rendered-page gate for this workflow. SortSite documents a paid Developer CI edition, outside the account-free, no-required-purchase setup considered here. UUV, QualWeb, sitespeed.io, Accessibility Insights, be-a11y, a11ymcp, Happo, MagentaA11y, SortSite Developer.

In an earlier hands-on check on October 1, 2026, before we adopted our research-only method, we served two original pages on our own machine: one with an unlabeled email field, an image without alternative text and low-contrast text, and a corrected copy. Pa11y CI 4.1.1, axe-core CLI 4.13.0 and Lighthouse CI 0.15.1, all using Chrome for Testing 152.0.7977.54 on Linux, reported the three defects and nothing on the corrected page. On a missing page, Lighthouse CI stopped with a 404 error, Pa11y CI passed, and axe-core CLI failed only because the error page had its own accessibility problems. These observations match what the tools’ source code and documentation say. They did not decide the Pick, which was not part of that check. Earlier check record, October 1, 2026

ToolPublic sourcesWhat remains uncertain
@axe-core/playwrightPlaywright accessibility, navigation and web-server guides; npm release records; Deque’s issue tracker; OSV advisory; npm download counts.Detection rate on real sites, behavior on pages with complex iframes, how much test code a given site needs.
Pa11y CIREADME and exit codes, Pa11y release notes and navigation code, issue tracker.When the open threshold and crash fixes ship, current detection rate of HTML_CodeSniffer.
Lighthouse CIConfiguration and services documentation, November 2 release notes, older Lighthouse source, scoring guide, issue tracker.Behavior on your pages under Lighthouse 13.6.0; the versioned category configuration was checked November 6, 2026.
IBM accessibility-checkerPackage README and configuration, command-line source, releases, Sparkbox review of the extension.How it treats error pages, exactly what the telemetry collects, operation without network access.
axe-core CLIREADME, package record, page-loading code, releases, issue tracker.Stability on long scans in CI, ChromeDriver matching on your runner.
cypress-axePlugin README and releases; Cypress accessibility guide and pricing, checked November 6, 2026.Site-specific test code, navigation failures and plugin compatibility on your runner.
@axe-core/puppeteerDeque’s API guide and releases, checked November 6, 2026.The assertions, error handling and report formatting your team adds.
NewcomersAccessLint monorepo and October 19 package record, accessio-scan source/package and announcement, A11yEco CLI documentation, Includa alpha notice, November 6, 2026.Both-OS support, same-market ownership for accessio-scan, A11yEco first release and CLI safety; no Rising Star qualifies on established evidence.
wick-a11yREADME configuration, reports, limits and release listing, checked November 6, 2026.Exact dependency safety and behavior on your pages; reports do not prove greater detection coverage.

None of this establishes how many problems any tool finds on your pages, how often it reports false problems, or how long a run takes. Every result depends on the page state, the browser and the rule version. Next substantive review is due December 18, 2026, or sooner if new documentation changes these recommendations or Pa11y CI fixes its open gate bugs. Found a changed release or behavior? Contact the editors.

Sources and evidence

Sources were checked October 19, 2026 unless another date is given. Project material establishes documented options, source behavior and published releases, not results on your site. Detection figures are their publishers’ own measurements. Download counts are the npm registry’s weekly figures for October 12 to 18, 2026.

This is a researched comparison, not a product review. Method: evidence standard, editorial picks.