|
1 | 1 | --- |
2 | | -title: Automated validation |
| 2 | +title: Automated testing |
3 | 3 | layout: default |
4 | 4 | parent: Test for accessibility |
5 | 5 | description: Learn about automated accessibility testing and validation. |
6 | | -nav_order: 5 |
| 6 | +nav_order: 2 |
| 7 | +contributors: |
| 8 | + - Rian Rietveld |
7 | 9 | --- |
8 | 10 |
|
9 | 11 | # Automated accessibility testing |
10 | 12 |
|
11 | | -## WCAG 2 AA automated validation |
| 13 | +With automated accessibility testing, it is necessary to check against the generated Document Object Model (DOM) of your web project. This is different from, for example, PHP code standard checks where you can perform checks on the code base. You need to run checks on a working site. |
12 | 14 |
|
13 | | -The generated DOM, the front end of your web project, must conform to WCAG 2 AA. You can use automated validation to get a limited scope of your success at this. |
| 15 | +Automated accessibility testing cannot catch all accessibility errors, so additional manual testing is still needed. Testing in the browser and manual keyboard testing need to be part of your regular workflow. |
14 | 16 |
|
15 | | -Recommended: |
| 17 | +## Recommended testing tools |
16 | 18 |
|
17 | | -- [aXe browser addon](https://www.deque.com/aXe/) for Chrome and FireFox. The addon adds a tab to your inspector with a validate button. After validating you see the errors and warnings for that particular webpage and how to fix them. Make sure to also test in different views and with the menu open and closed, for example. |
18 | | -- [HTML_CodeSniffer](http://squizlabs.github.io/HTML_CodeSniffer/) |
19 | | -- Accessibility Inspector in the [Firefox Developer Tools](https://developer.mozilla.org/en-US/docs/Tools). A good read about this by Marco Zehe: How to use [NVDA and Firefox to test your web pages for accessibility](https://www.marcozehe.de/how-to-use-nvda-and-firefox-to-test-your-web-pages-for-accessibility/) |
| 19 | +### Axe-core |
20 | 20 |
|
21 | | -More tools: |
| 21 | +[Axe-core](https://github.com/dequelabs/axe-core-npm) by Deque is an accessibility testing engine for websites and other HTML-based user interfaces. |
22 | 22 |
|
23 | | -- [React-a11y](https://github.com/reactjs/react-a11y), Identifies accessibility issues in your React.js elements |
24 | | -- Static AST checker for [a11y rules on JSX elements](https://github.com/evcohen/eslint-plugin-jsx-a11y) |
25 | | -- More [Toolbars and toolkits ](https://make.wordpress.org/accessibility/handbook/which-tools-can-i-use/useful-tools/#toolbars-toolkits) in the Accessibility Handbook |
| 23 | +[axe-core-npm](https://github.com/dequelabs/axe-core-npm) offers packages like @axe-core/cli and @axe-core/react, which can be used for automated accessibility testing powered by axe core. |
26 | 24 |
|
27 | | -## Automated testing |
| 25 | +[Playwright Accessibility testing](https://playwright.dev/docs/accessibility-testing) uses Axe. Accessibility tests work just like any other Playwright test. You can either create separate test cases or integrate accessibility scans and assertions into your existing test cases. NPM package: [@axe-core/playwright](https://www.npmjs.com/package/@axe-core/playwright). |
28 | 26 |
|
29 | | -The big difference between PHP and JavaScript code standard checks and accessibility checks is that the accessibility checks need to be performed on the generated DOM, not on the code base. You need to run checks on a working WordPress site. This can be done using automation that sets up your environment dynamically, but still requires a fully built and run installation. |
| 27 | +Axe is also available as [aXe Devtools browser addon](https://www.deque.com/axe/devtools/extension/). The addon adds a tab to your inspector with a "validate" button. After validating you see the errors and warnings for that particular webpage and how to fix them. Available as a free limited version and as a paid Pro version. |
30 | 28 |
|
31 | | -There are several command line tools for automated testing like [aXe-cli](https://github.com/dequelabs/axe-cli) and [pa11y](https://github.com/pa11y/pa11y). |
| 29 | +[Google LightHouse for Chrome](https://developer.chrome.com/docs/lighthouse) uses axe-core under the hood. |
32 | 30 |
|
33 | | -{: .callout .info } |
34 | | -**Info:** Automated accessibility testing doesn’t catch all the issues, rarely more than 30%. Testing in the browser and manual keyboard testing still needs to be part of your workflow. |
| 31 | +### IBM Equal Access Accessibility Checker |
35 | 32 |
|
36 | | -### Setup for aXe-cli |
| 33 | +[The IBM Equal Access Accessibility Checker](https://github.com/IBMa/equal-access/wiki) is an open-source and freely available set of tools for web developers and accessibility auditors, documented in the IBM Equal Access Wiki. |
37 | 34 |
|
38 | | -Install axe-core for CLI first: |
| 35 | +- The **accessibility-checker-engine** uses a set of rules that map to accessibility standards to detect accessibility issues in web content and applications. |
| 36 | +- The **accessibility-checker-extension** integrates into browser DevTools, providing an integrated scanning experience, a keyboard checker mode visualization, and helps users quickly identify the source of accessibility issues, understand what to do, and try fixes. The Checker is also available as a package for CI/CD environments and automated testing frameworks. |
39 | 37 |
|
40 | | -``` |
41 | | -npm install axe-cli -g |
42 | | -npm install chromedriver –g` |
43 | | -``` |
44 | 38 |
|
45 | | -Then run axe in the command line: |
| 39 | +### W3C Validators and tools |
46 | 40 |
|
47 | | -``` |
48 | | -axe url -b c |
49 | | -``` |
| 41 | +The W3C maintains a page with [links to the test tools](https://www.w3.org/developers/tools/) they provide, like the Nu HTML Checker, CSS Validator, and Link Checker. Their information includes the W3C API, providing consistent access to specifications and other W3C data. |
50 | 42 |
|
51 | | -The url can be any url, also a local one. You get a report of the accessibility issues for that url. The **-b** stands for browser, **c** for chrome, as this browser gives the best results (better than the default PhantomJS). You will get the errors and warnings in your console. |
| 43 | +### Pa11y |
| 44 | +[Pa11y](https://github.com/pa11y/pa11y), runs accessibility tests on your pages via the command line or Node.js, so you can automate your testing process. |
52 | 45 |
|
53 | | -You can provide more (space separated) urls to the command like |
| 46 | +## Resources |
54 | 47 |
|
55 | | -``` |
56 | | -axe url url2 url3 -b c |
57 | | -``` |
| 48 | +Research by Adrian Roselli: [Comparing Manual and Free Automated WCAG Reviews](https://adrianroselli.com/2023/01/comparing-manual-and-free-automated-wcag-reviews.html). This recommended post is updated regularly. |
58 | 49 |
|
59 | | -or call a file with urls like: |
60 | | - |
61 | | -``` |
62 | | -axe $( cat list-of-urls.txt ) -b c |
63 | | -``` |
64 | | - |
65 | | -**Note**: You can run [axe-cli on more than one url](https://make.wordpress.org/accessibility/handbook/best-practices/test-for-web-accessibility/test-for-web-accessibility-frontend-code/#setup-for-axe-cli) in one command, but axe-cli is not build to run on a large amount of urls or on a complete site, axe-cli is not a crawler. Deque Labs recommends to use the [axe-webdriverjs](https://www.npmjs.com/package/axe-webdriverjs), a chainable aXe API for Selenium’s WebDriverJS for testing on a large amounts of urls. |
66 | | - |
67 | | -### Setup for pa11y |
68 | | - |
69 | | -The [setup for pa11y](https://github.com/pa11y/pa11y) is well documented in the GitHub repository. |
70 | | - |
71 | | - |
72 | | -## Browser Toolbars & Toolkits |
73 | | - |
74 | | -- [aXe accessibility testing tool](https://www.deque.com/products/axe/) — available as browser extension and npm module ([axe-core](https://github.com/dequelabs/axe-core)) |
75 | | -- [WAVE](http://wave.webaim.org/toolbar) — run the WAVE accessibility evaluation tool within Firefox and Chrome. |
76 | | -- Accessibility inspector in the FireFox developer tools |
77 | | -- [AInspector for WCAG Accessibility Evaluation](https://addons.mozilla.org/en-US/firefox/addon/ainspector-wcag/) — Inspect web pages for potential accessibility issues. |
78 | | -- [Tota11y](http://khan.github.io/tota11y/) — An accessibility visualization toolkit that can be dragged into your bookmarks bar or installed as a plugin. |
79 | | -- [Total Validator](http://www.totalvalidator.com/) — an HTML validator, accessibility validator, spell checker, and broken link checker all rolled into one tool. Free & commercial versions available. |
80 | | -- [HeadingsMap](https://chrome.google.com/webstore/detail/headingsmap/flbjommegcjonpdmenkdiocclhjacmbi?hl=es) — A [Chrome](https://chromewebstore.google.com/detail/headingsmap/flbjommegcjonpdmenkdiocclhjacmbi?pli=1) or [Firefox extension](https://addons.mozilla.org/en-us/firefox/addon/headingsmap/) that shows the structure of headings on a webpage. |
81 | | -- [Visual ARIA Bookmarklet](http://whatsock.com/training/matrices/visual-aria.htm) — A bookmarklet to visually display ARIA attributes on a webpage. |
82 | | -- [Accessibility Testing Tools for Desktop and Mobile Websites](https://www.24a11y.com/2017/accessibility-testing-tools-desktop-mobile-websites/) – A review of testing tools by Paul J. Adam. |
83 | | -- [Accessibility Bookmarklets](https://www.digitala11y.com/accessibility-bookmarklets-testing/) – A collection of bookmarklets useful for accessibility testing. |
0 commit comments