Auditing websites for accessibility is imperative. Ignore accessibility and you unintentionally exclude 56.7mm people (Source: US Census Bureau, 2012) from having access to your website. But, before you can do that, you need to audit your site and find out what problems exist. To manually audit every single page on medium-to-large sites can be time-consuming, if not impossible. When a large digital education service provider asked us to audit a sampling of 800 landing pages and 30 micro sites for accessibility (a total of about 4,000 pages)...in one month...we needed to find a better way. Pa11y Dashboard to the rescue. Pa11y Dashboard provides an open source interface for keeping track of automated accessibility tests over time. It served as a handy tool that we could reference in our extensive written report. It also allowed us to see common patterns across various sites quickly and provide custom solutions.
The Dashboard
Out of the box, the front page of Pa11y Dashboard lists a filterable overview of all tasks. By default, automated audits are performed daily at 12:30 a.m., but the timing can be reconfigured in the Cron settings. Through the dashboard, you can also create, edit, and delete your tasks, and manually trigger an audit. Check out the Pa11y Dashboard demo to see it in action.
We customized the layout of the dashboard home page
The Task Pages
Individual task pages detail the errors, notices, and warnings, giving a short description of each along with their location in the HTML. Export these reports in CSV or JSON format, as you need. Also, a graph illustrates the delta in errors, warnings, and notices over time. A task holds a user readable name, a URL to track, the accessibility standard to perform tests against, timeout settings, HTTP Basic Authentication credentials, and the optional ability to ignore certain rules.
Errors, warnings, and notices
Pa11y Dashboard will output three types of messages. The system detects any clear-cut guideline violations and reports them as "errors." The system also issues "warnings" for items it identifies, but that require manual, human verification to confirm the guideline violations. And, "notices" are items that the system cannot detect, but still require a keen human eye (or ear) to ensure the web page meets a11y requirements. Warnings and notices are just as important as errors to verify and resolve to achieve standards compliance.
What's under the hood?
- Pa11y Dashboard: A web UI to view and manage scheduled audit tasks, built with Node.js, Express for routing, Handlebars for templating, and LESS for CSS preprocessing.
- Pa11y Webservice: The Pa11y JSON web service that allows you to run tests against multiple scheduled URLs with PhantomJS, and allows you to store and query results from a MongoDB database.
- Pa11y: A Node.js module, which provides a command-line and javascript interface for HTML Codesniffer tests. The module relies on PhantomJS. Pa11y was originally created by a group of front-end developers at Nature Publishing Group (later Springer Nature), and continues today as its own organization on Github.
- HTML Codesniffer: A client-side javascript application, which performs accessibility tests against the DOM. It contains rules and tests for Section 508, and all three levels of WCAG 2.0 web accessibility standards. It also includes a modal UI to run these tests in the browser via a bookmarklet.
Customizations
Because the Pa11y Dashboard is fully free and open source, it allowed us to make customizations easily to serve our client's specific needs.
We created a page listing all the errors, warnings, and notices found across the audit. For every issue found, it is indicated how often it occurs, on how many pages, and lists all the pages where it is found. This gives our client a bird’s-eye view of all the audited sites and a quick insight into the most common issues. We added permalinks so we could reference individual issues easily from our written audit.
Out of the box, Pa11y Dashboard does not accommodate crawling entire sites. We wrote a custom script to collect and import all URLs from the micro sites, and adjusted the UI to visually bundle micro site URLs together.
Importing tasks in batches
As previously mentioned, adding new tasks can be done through the UI. If you need to add a lot of tasks, you can leverage the Pa11y Webservice Client library that is included in the dashboard. This way, you can import your tasks in batches with a simple script.
1. Make sure the dashboard is up and running. Detailed instructions on how to set it up can be found on the Pa11y Dashboard Github page.
2. Next, create an CSV file and place it in the /data folder. We'll name it pa11y-tasks.js. It lists all the values we need to create a new task. Optionally, there are more values that can be added to tasks: ignore, timeout, wait, username, and password (for HTTP Basic Authentication).
"name","url","standard"
"Hillary for America","https://www.hillaryclinton.com/","WCAG2AA"
"Donald J Trump for President","https://www.donaldjtrump.com/","WCAG2AA"
"Mike Herchel for President","http://imwithherchel.com/","WCAG2AAA"3. Install the CSV parser module.
npm install csv-parser --save4. Add the following JavaScript file. We named it pa11y-import.js. In it we read the CSV file and import the tasks. It leverages the config file from the dashboard to create a client endpoint to store our data.
var fs = require('fs');
var csv = require('csv-parser');
var createClient = require('pa11y-webservice-client-node');
var config = require('./config');
var client = createClient('http://' + config.webservice.host + ':' + config.webservice.port + '/');
// Read the CSV file.
fs.createReadStream('./data/pa11y-tasks.csv')
.pipe(csv())
.on('data', function(data) {
// Create task.
client.tasks.create({
name: data.name,
url: data.url,
standard: data.standard
}, function (error, task) {
// Error and success handling.
if (error) {
console.error('Error:', error);
}
if (task) {
console.log('Imported:', task.name);
}
});
}).on('end', function () {
console.log('All tasks imported!');
});5. Finally, run the import script once. Optionally replace the Node environment value if you are using development, or test.
NODE_ENV=production node pa11y-import.jsThat's it! All your tasks should now be imported.
The future of Pa11y Dashboard
While Pa11y Dashboard in its current form is a very helpful tool, plans are being made for a successor project, named Sidekick. Among proposed solutions are full site audits, increased focus on Continuous Integration and deployment, and showing a “diff” that compares the current results to the last run. If you are interested in learning more about where the project is headed or getting involved, I suggest checking out the Sidekick proposal page.
The importance of manual audits
While we were able to do some really powerful and interesting things with the Pa11y Dashboard, keep in mind that in the end, optimizing your site for accessibility is more than getting rid of errors and warnings by audit tools. While tools like this one can give you a good overview of some of the most important issues, and are certainly a big help, no tool can completely replace manually assessing your pages. Using manual review techniques such as navigating your site by keyboard only and with a screen reader are crucial for determining whether or not you have an accessible website. Make sure every part of your site is navigable, and content can be accessed by every user. Even better, collect feedback from users with disabilities. They’ll likely catch things even the best pros won’t.
As part of this audit, we empowered our client to be able to perform manual audits, and find issues that can’t be found using automated tests. Helena’s on-site visit was critical in this process to instil enthusiasm and share practical knowledge. Ultimately, what really matters is the user experience.