Accessibility Testing in CI/CD

Shift left: Catch accessibility issues before they reach production.

⚙️ DevOps automation

Why automate accessibility testing?

💡 The earlier you catch bugs, the cheaper they are to fix

Accessibility issues found after launch cost much more to fix than those caught during development. Automated testing catches part of them instantly, on every commit: single tools found 17–41% of known barriers in a UK Government Digital Service test (2017), and Deque’s 2021 study found automated tests covered 57% of issues by volume. The rest still need manual and assistive-technology testing.

Benefits of CI/CD integration

Prevent regressions: New code doesn't break existing accessibility
Immediate feedback: Developers learn about issues in minutes
Consistent enforcement: Every PR gets the same checks
Documentation: Build reports track accessibility over time
Culture shift: Makes accessibility part of "done"

The accessibility testing pyramid

            /\
           /  \ Manual testing (10%)
          / 👤 \  - Screen reader, keyboard
         /      \  - Usability studies
        /--------\ 
       /          \ Integration tests (30%)
      /     🔍     \  - axe-core in e2e tests
     /              \  - Storybook a11y addon
    /----------------\
   /                  \ Unit tests (60%)
  /         ⚡          \  - Component-level axe
 /                      \  - eslint-plugin-jsx-a11y
/------------------------\

Automate what you can (bottom), manually test what you must (top).

axe-core: The industry standard

axe-core by Deque is the most widely-used accessibility testing engine. It powers most automated tools.

What it catches

Missing alt text
Color contrast failures
Missing form labels
Invalid ARIA
Heading structure issues
Keyboard accessibility problems
And 80+ more rules

What it can't catch

Whether alt text is meaningful
Logical focus order
Whether content makes sense
Complex interaction patterns
Cognitive accessibility

GitHub Actions setup

Basic workflow with axe

# .github/workflows/accessibility.yml
name: Accessibility Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  a11y-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Build project
        run: npm run build
      
      - name: Start server
        run: npm run start &
        
      - name: Wait for server
        run: npx wait-on http://localhost:3000
      
      - name: Run accessibility tests
        run: npm run test:a11y
      
      - name: Upload results
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: a11y-results
          path: a11y-results/

Using pa11y-ci

# package.json
{
  "scripts": {
    "test:a11y": "pa11y-ci"
  },
  "devDependencies": {
    "pa11y-ci": "^3.0.0"
  }
}

# .pa11yci.json
{
  "defaults": {
    "standard": "WCAG2AA",
    "runners": ["axe"],
    "chromeLaunchConfig": {
      "args": ["--no-sandbox"]
    }
  },
  "urls": [
    "http://localhost:3000/",
    "http://localhost:3000/about",
    "http://localhost:3000/contact",
    "http://localhost:3000/products"
  ]
}

Jest + React Testing Library

Setup jest-axe

npm install --save-dev jest-axe @testing-library/react

Component test example

import { render } from '@testing-library/react';
import { axe, toHaveNoViolations } from 'jest-axe';
import Button from './Button';

expect.extend(toHaveNoViolations);

describe('Button', () => {
  it('should have no accessibility violations', async () => {
    const { container } = render(
      <Button onClick={() => {}}>Click me</Button>
    );
    
    const results = await axe(container);
    expect(results).toHaveNoViolations();
  });

  it('should have no violations when disabled', async () => {
    const { container } = render(
      <Button disabled>Disabled button</Button>
    );
    
    const results = await axe(container);
    expect(results).toHaveNoViolations();
  });
});

Testing specific rules

it('should have sufficient color contrast', async () => {
  const { container } = render(<Alert type="warning">Warning!</Alert>);
  
  const results = await axe(container, {
    rules: {
      'color-contrast': { enabled: true }
    }
  });
  
  expect(results).toHaveNoViolations();
});

Cypress integration

Install cypress-axe

npm install --save-dev cypress-axe axe-core

Setup in cypress/support/e2e.js

import 'cypress-axe';

// Custom command to check accessibility
Cypress.Commands.add('checkA11y', (context, options) => {
  cy.injectAxe();
  cy.checkA11y(context, options, (violations) => {
    // Log violations to console with details
    violations.forEach((violation) => {
      const nodes = violation.nodes.map(n => n.target).join(', ');
      cy.log(`${violation.impact}: ${violation.description} (${nodes})`);
    });
  });
});

E2E test example

describe('Homepage accessibility', () => {
  beforeEach(() => {
    cy.visit('/');
    cy.injectAxe();
  });

  it('should have no accessibility violations on load', () => {
    cy.checkA11y();
  });

  it('should have no violations after opening modal', () => {
    cy.get('[data-cy="open-modal"]').click();
    cy.get('[data-cy="modal"]').should('be.visible');
    cy.checkA11y('[data-cy="modal"]');
  });

  it('should have no violations in dark mode', () => {
    cy.get('[data-cy="theme-toggle"]').click();
    cy.checkA11y(null, {
      rules: {
        'color-contrast': { enabled: true }
      }
    });
  });
});

Playwright integration

Install @axe-core/playwright

npm install --save-dev @axe-core/playwright

Test example

import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test.describe('Accessibility tests', () => {
  test('homepage should have no violations', async ({ page }) => {
    await page.goto('/');
    
    const results = await new AxeBuilder({ page })
      .withTags(['wcag2a', 'wcag2aa'])
      .analyze();
    
    expect(results.violations).toEqual([]);
  });

  test('form page should be accessible', async ({ page }) => {
    await page.goto('/contact');
    
    // Fill form to test all states
    await page.fill('#name', 'Test User');
    await page.fill('#email', 'test@example.com');
    
    const results = await new AxeBuilder({ page }).analyze();
    expect(results.violations).toEqual([]);
  });
});

Storybook a11y addon

Test components in isolation during development.

Install

npm install --save-dev @storybook/addon-a11y

Configure in .storybook/main.js

module.exports = {
  addons: [
    '@storybook/addon-a11y'
  ]
};

Run in CI

# Using storybook test runner
npm install --save-dev @storybook/test-runner axe-playwright

# package.json
{
  "scripts": {
    "test:storybook": "test-storybook"
  }
}

ESLint for JSX accessibility

Install eslint-plugin-jsx-a11y

npm install --save-dev eslint-plugin-jsx-a11y

ESLint config

// .eslintrc.js
module.exports = {
  plugins: ['jsx-a11y'],
  extends: ['plugin:jsx-a11y/recommended'],
  rules: {
    // Enforce alt text
    'jsx-a11y/alt-text': 'error',
    
    // Require labels for inputs
    'jsx-a11y/label-has-associated-control': 'error',
    
    // No onClick on non-interactive elements
    'jsx-a11y/click-events-have-key-events': 'error',
    'jsx-a11y/no-static-element-interactions': 'error',
    
    // Enforce heading order
    'jsx-a11y/heading-has-content': 'error'
  }
};

This catches issues at write-time in the IDE, before code is even committed.

Reporting and tracking

Generate HTML reports

// Custom reporter for pa11y
const reporter = {
  results: (results) => {
    const html = generateHTMLReport(results);
    fs.writeFileSync('a11y-report.html', html);
  }
};

// Or use axe-html-reporter
const { createHtmlReport } = require('axe-html-reporter');
createHtmlReport({ results });

Track metrics over time

Number of violations by severity
Pages with zero violations
Violations by rule type
Time to fix accessibility bugs

Dashboard integration

Send metrics to your monitoring dashboard:

// After running axe
const metrics = {
  total_violations: results.violations.length,
  critical: results.violations.filter(v => v.impact === 'critical').length,
  serious: results.violations.filter(v => v.impact === 'serious').length,
  pages_tested: urls.length
};

// Send to your metrics system
await sendToDatadog(metrics);
await sendToGrafana(metrics);

Best practices

Do

✅ Run tests on every PR
✅ Fail the build for critical/serious violations
✅ Test all major pages and states
✅ Include accessibility in code review checklist
✅ Combine automated + manual testing

Don't

❌ Assume 100% automated pass = accessible
❌ Disable tests to make builds pass
❌ Only test the homepage
❌ Ignore "minor" violations forever

Progressive enforcement

// Start by warning only
cy.checkA11y(null, null, null, { skipFailures: true });

// Then fail on critical only
cy.checkA11y(null, {
  rules: {
    'critical': { enabled: true }
  }
});

// Finally, fail on all WCAG 2.1 AA
cy.checkA11y(null, {
  runOnly: {
    type: 'tag',
    values: ['wcag2a', 'wcag2aa', 'wcag21aa']
  }
});

Web & Apps