Automating Accessibility Testing in Your CI/CD Pipelines with Axe

While trying to release an application as soon as possible, making sure it is accessible to everyone, including users with disabilities, can be a daunting task. With continuous releases and tight deadlines, manual accessibility testing often takes a back seat.

This not only harms user experience but also exposes your organization to legal risks and damages your brand reputation.

You can solve this problem by integrating tools like Axe directly into your continuous integration and deployment workflows, enabling you to automatically catch accessibility issues early in the development process pre-production.

In this article, we will examine why it's so important to include Axe within your CI/CD pipelines and walk through practical steps using Selenium WebDriver and Puppeteer.

At the end of the article, you will be familiar with setting up Axe with these frameworks as well as best practices.

Prerequisite

This article assumes that you have a fair understanding of:

1. Selenium or Cypress
2. Javascript
3. NPM

Why Automate Accessibility Testing in Your CI/CD Pipeline?

Imagine releasing a new feature only to discover it is unsuitable for people using screen readers or keyboard navigation. Your users will be frustrated, your brand reputation will suffer, and, worse, it might lead to legal implications in certain industries.

That’s where automating accessibility tests comes in. Automated accessibility testing within your CI/CD pipeline offers several key benefits.

1. Catching issues as soon as they're introduced makes them easier and less costly to fix.
2. Maintaining adherence to accessibility standards throughout development.
3. Reducing manual testing efforts, freeing your team to focus on building features.

Getting Started with Axe for Automated Testing

We'll cover how to set up Axe with two popular frameworks:

• Selenium WebDriver
• Cypress

Choose the one that fits your project's needs.

Automate Accessibility Testing with Selenium and Axe

By combining Selenium with Axe, you can perform automated accessibility audits during your testing process, catching issues early.

Bringing Axe into your Selenium tests is a great way to enhance your test suite with some accessibility checks. That way, every time you run your regression tests, you also test for accessibility issues, ensuring that new features comply with accessibility standards.

Installation

To get started, install Selenium WebDriver and Axe for WebDriverJS in your project directory:

    npm install selenium-webdriver axe-webdriverjs

Writing the Test Script

Let's create an accessibility test script that integrates with your existing Selenium tests. Create a file named accessibility.test.js

const { Builder } = require("selenium-webdriver");
const axeBuilder = require("axe-webdriverjs");

(async function accessibilityTest() {
  let driver = await new Builder().forBrowser("chrome").build();

  try {
    await driver.get("<https://yourapplication.com>");
    const results = await axeBuilder(driver).analyze();

    if (results.violations.length > 0) {
      console.error("Accessibility violations found:");
      results.violations.forEach((violation) => {
        console.error(
          `${violation.help}: ${violation.nodes.length} instance(s)`
        );
      });
      process.exit(1);
    } else {
      console.log("No accessibility violations found.");
    }
  } finally {
    await driver.quit();
  }
})();

This script initializes a Chrome browser, navigates to your app, and runs Axe's accessibility checks. If violations are found, it logs them and exits with a non-zero status code, causing the CI/CD pipeline to fail the build.

Selenium is ideal if you're already using it for functional tests and want to add accessibility checks without adopting a new tool.

Using Axe with Cypress

Integrating Axe into your Cypress tests makes doing accessibility checks super easy. Here's how you can use Axe and Cypress:

Cypress offers a developer-friendly environment for end-to-end testing with real-time reloads and debugging capabilities. By integrating Axe, you can add accessibility checks to your Cypress tests, catching issues as part of your development process.

Installation

Install Cypress and the Axe integration:

    npm install cypress @axe-core/core @axe-core/cypress

In your cypress/support/commands.js import the Axe commands: 

    import '@axe-core/cypress';

Writing the Test

Create a test file accessibility.spec.js and add:

describe("Accessibility Test", () => {
  it("Should have no detectable accessibility violations on load", () => {
    cy.visit("<https://yourapplication.com>");
    cy.injectAxe();
    cy.checkA11y(null, null, (violations) => {
      if (violations.length) {
        cy.wrap(violations).each((violation) => {
          cy.log(`${violation.id} ${violation.help}`);
        });
        throw new Error("Accessibility violations found");
      }
    });
  });
});

This test navigates to your application, injects the Axe-core library, and runs accessibility checks. If violations are detected, it logs them and throws an error to fail the test, ensuring issues are caught during the CI/CD process.

Integrating Axe into Github Actions

In this section, we'll see how to integrate our tests into our CI/CD pipeline to automate accessibility testing. Although we’ll be covering just Github Actions, you can apply the same principle to other CI/CD platforms like Jenkins, CircleCI, and GitLab.

Integrating with GitHub Actions

To setup Axe automation using Github Actions, your code has to be hosted on Github. The following workflow file will run your accessibility test on code changes.

Create a workflow file at .github/workflows/accessibility.yml:

name: Accessibility Testing

on:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Set up Node.js
        uses: actions/setup-node@v2
        with:
          node-version: "14"
      - name: Install Dependencies
        run: npm install
      - name: Run Accessibility Tests
        run: |
          npx cypress run

This workflow triggers when you push your code to the main branch. It then installs dependencies and runs your accessibility tests. If any issues are found, the build fails.

Applying Integration Principles to Other CI/CD Platforms

No matter which platform you use, the integration steps are similar:

1. Install Dependencies: Ensure your CI environment has the necessary tools.
2. Run Accessibility Tests: Execute test scripts during the build.
3. Handle Failures: Configure the pipeline to fail the build if violations exceed acceptable thresholds.

Best Practices For Integrating Your Automated Tests with CI/CD

Simply adding tests to your CI/CD pipeline isn't enough, you need to ensure that these tests are well integrated and managed to maximize their benefits. Here are some best practices to make the integration of accessibility testing smooth within your development workflow:

Setting Failure Thresholds

When you first implement automated accessibility testing, you might discover a large number of existing issues. To prevent your builds from failing constantly due to minor violations, it's important to set acceptable failure thresholds.

for example:

const violationThreshold = 5;
if (results.violations.length > violationThreshold) {
  console.error("Too many accessibility violations!");
  process.exit(1);
}

Team Collaboration

Accessibility should be a shared responsibility across your team. Encourage collaboration between developers, designers, QA engineers, and product managers to ensure that accessibility is considered at every stage of development.

In Summary

Adding automated tests to your CI/CD process might not be the easiest process for you and your team. But if you do want to try it out this article can put you on the right track. Here are some key takeaways: make sure to automate early, collaborate with your entire team throughout the process, and be in a monitoring and improvement loop for your automated tests.