"It's clear that artificial intelligence can greatly enhance the automation and speed of accessibility evaluation and testing. But it is equally important to recognise its current limitations…. Combining AI tools with human expertise ensures a more comprehensive and accurate approach to complying with accessibility laws and standards and creating an inclusive experience for all users."
Introduction
I recently marked my 20th anniversary of working as a software testing professional. For close to a decade, I’ve been learning about digital accessibility, making digital apps, websites and products work for as many people as possible. And I decided to write, talk and even teach accessibility to others, since I didn’t see many others teaching when I started my work in accessibility.
With the widespread adoption of artificial intelligence (AI) tools, we are beginning to see AI-assisted accessibility tools appear on the market.
These products generally fall into three categories.
- Evaluation tools: point out anything that they see as a potential issue
- Evaluate and suggest tools: highlight and suggest ways to address or fix the issues they find
- Evaluate and fix tools: apply fixes directly to the code under test, based on their findings
These tools, particularly the ones that simply fix things for you, sound great! But will they solve all the problems? Unfortunately, no. Will they help? Yes, in the right circumstances and within reasonable scope.
So, in this article, I will look at the tools that are out there already. I'll examine what they might offer, especially in relation to planned web content accessibility guidelines (WCAG) updates. And I'll run through the pros and cons of using artificial intelligence to support accessibility evaluation and testing.
What's available now?
I've limited the scope of this article to tools that are commercially available as add-ons or standalone products. Excluded are vendor overlays, which have inherent problems and have been the subject of many posts and articles. I am not endorsing any tools either. Instead, I'll share publicly available information on them to demonstrate what is out there.
There are two main types of tools on the market at the moment. Some are of just one type and some are hybrids.
- Automated checks: Tools in this category run automated checks for compliance with accessibility guidelines and give you a report. (Some of these checks may return suggestions that go above and beyond what the guidelines require.) These tools generally focus on systematic or binary identifiable elements. Systematic elements include HTML, headings, or tags such as header, main, and footer. Binary identifiable elements are either present or not, such as alternative text (alt text).
-
Guided tests: Tools in this category combine automated checks with a series of steps to verify information. Adding AI to these tools means that the AI can generate answers that contain information targeted to the system under test. Some tools will ask the user to confirm information throughout the testing process to make sure the AI has accurate information before it offers solutions. An example of a guided test might be:
- The tool is run on the application.
- The tool identifies a potential issue, such as a missing label.
- The tool shows a list of steps to fix the problem. It may even display the problem code snippet with instructions on how to replace it in the code base.
Both of these categories include tools that can create visual renderings of the system under test, highlighting visual focus and error states and offering solutions where they find errors. Others can scan code against the WCAG, while some tools offer a walkthrough of the guidelines, providing testing instructions and explanations. And others can integrate with design or architecture tools, highlighting potential accessibility issues before testing begins.
In addition to the examples above, certain tools offer accessibility evaluation support in specific areas:
- Readability
- Text simplification
- Caption generators
- Speech transcription
- Content reading and summary
- Translation
- Voice assistants
- Enhanced facial recognition
- Computer vision
Finally, there are code assistants like GitHub Copilot.
Given the range of tools, what can and can’t they offer us?
Will the tools keep up with upcoming WCAG changes?
While we don’t know when the final version of the WCAG 3.0 (Project Silver) update will be released, there is enough information to understand some of the changes it will make.
The update will include some new criteria, and the approach to testing against the guidelines will be reframed. At present, testing results in a pass or fail. Under the new version, there will be ‘atomic’ pass or fail outcomes as well as ‘holistic’ outcomes.
Among the goals for the new guidelines are:
- Cover even more users
- Include newer technology such as augmented and virtual reality
- Present simpler and easier to understand guidelines
- Allow broader testing options with clearer outcomes
- Suggest a potential rating scale (bronze, silver, gold)
This is the most significant change to the guidelines since they were first released. However, no tool I’m aware of has mentioned the upcoming changes in their documentation or marketing materials. While the guidelines will hardly make the AI-assisted tools obsolete, there may be a large gap in coverage for a while.
Bearing these changes in mind, what are the pros and cons to using AI to assist accessibility testing?
AI-assisted accessibility evaluation features: pros and cons
As you'll see, there's a lot of potential in enhancing accessibility evaluation with AI. However, human tester support will be crucial where the tools are weaker, as discussed in the "cons" section for each feature.
AI-powered image recognition for alt text evaluation
Pros
AI can quickly generate alt text for large numbers of images, saving time and effort for content creators. Advanced AI can understand the content of an image and provide descriptive alt text that enhances accessibility.
Cons
AI may generate accurate descriptions, but might not understand when an image is purely decorative. This could lead to confusion for the user, who may wonder, why am I hearing about a bunch of flowers when I’m buying underwear?!? And AI might miss more subtle context or even the intended purpose of an image. Your product could end up with inappropriate or inadequate descriptions for its images.
Improved analysis of semantic structure
Pros
As long as it keeps up with changes in the HTML specification, AI can help ensure consistent use of headings and ARIA (accessible rich internet applications) roles. This will help reduce errors in semantic structuring. And AI can analyse complex and large-scale websites efficiently, ensuring that all elements follow semantic rules.
Cons
AI might not fully grasp or understand correctly the meaning or importance of the content it analyses, leading to suggestions that disrupt the logical flow or user experience. Valid deviations might be flagged as errors, resulting in unnecessary changes or false positives.
Enhanced colour contrast testing with AI
Pros
- Comprehensive analysis: AI can analyse colour contrast across various states (hover, active, and so on) and simulate different visual impairments, providing a thorough evaluation.
- Real-time adjustments: AI can offer real-time suggestions or adjustments to colour schemes during the design process.
Cons
AI might flag colours as non-compliant if it doesn't fully consider design context or user preferences. Complex visual backgrounds or patterns where contrast is more challenging to assess may lead to failed or false reporting.
Intelligent simulation of keyboard navigation
Pros
AI can simulate complex user interactions, uncovering accessibility issues that might be missed by manual testing. It can test various scenarios rapidly, identifying potential keyboard traps or inaccessible elements without the need for extensive manual input.
Cons
AI testing might not represent real user behaviour, especially in edge cases, leading to overlooked issues or missed opportunities. Over-reliance on these tests might lead developers to assume that all keyboard navigation issues have been addressed when some complex scenarios remain untested.
Automated analysis of content readability and understandability
Pros
AI can analyse text for readability, looking for patterns such as overuse of capitalisation, complicated language, and the use of undefined acronyms. To do this, it uses scales or scores to measure how difficult something is to read. These checks help ensure that content meets guidelines for cognitive accessibility. And AI can suggest simpler alternatives for complex language, making content more accessible to a broad audience.
Cons
AI might oversimplify content, stripping it of necessary detail or nuance, especially in technical or specialised contexts. An automated tool may not take into consideration cultural nuances or context-specific language, leading to suggestions that are inappropriate or insensitive.
To wrap up
It's clear that artificial intelligence can greatly enhance the automation and speed of accessibility evaluation and testing. But it is equally important to recognise its current limitations. There is no doubt that the tools will continue to improve, but it feels highly unlikely they will completely replace humans in any testing space.
AI's "intelligence" is based on averages and datasets, which can present only the most common scenarios, leading to unintended bias. It is important that we understand this when looking at AI outputs, especially those like image descriptions.
Combining AI tools with human expertise ensures a comprehensive and accurate approach to complying with accessibility laws and standards and creating an inclusive experience for all users.
Tools to explore
-
Axe DevTools and Artificial Intelligence (AI)
- Deque is known for sharing knowledge and resources such as its accessible components library. A feature of their axe DevTools suite is their guided tests. These take you step by step through tests on your site. They ask questions and combine “computer vision with syntax analysis to find potential issues.”
-
Accessibility Desk AI Accessibility Toolkit
- Accessibility Desk shares a free toolkit with a number of individual tools associated with improving accessibility. Things like helping writing alternative text, accessibility statement generator, code validator, easy read to improve readability and an assessment tool where you answer questions.
-
Stark suite of integrated accessibility tools
- Stark claims to ‘supercharge accessibility’ through being ‘the only end-to-end solution from design and code to live product.’ Certainly a bold claim. Starting at the design phase it has plug-ins and integration for Figma, Sketch and Github. It offers reporting, AI-powered automation and a compliance centre. It quotes an impressive number of high profile clients from IBM to Microsoft and Deloitte. It combines AI insights with checklist flows with ‘scan and fix’ capability.
-
Top 18 AI Testing Tools in 2024 - Code Intelligence
- AI Tools list where some can be used for accessibility testing or reviews. Not specifically accessibility AI tools.
For more information
- Web Content Accessibility Guidelines 2.2
- Accessibility Overlay Fact Sheet
- Flesch-Kincaid readability tests - Wikipedia
- Web Accessibility Initiative - Accessible Rich Internet Applications Overview
- American ADA Legal Cases & The Impact On Global Accessibility, Helen Burge
- How to Advocate for Accessibility Testing and Implementing It on Your Teams, Janique Morris
- AI for accessibility conformance testing? Why? Steve F
