There are a number of tools like Axe as the tool we use to assess accessibility. Where do the tools fall short?

For my AMA on Accessibility Testing, Gary Hawkes asked. "There are a number of tools like Axe as the tool we use to assess accessibility. Where do the tools fall short?"

This is a great question, and probably one I should write an article about, especially now that AI accessibility checkers are coming to the market.

While they can find quite a few things, there are a lot of false positives involved, which is why you see more 'warnings', things for a person to check, than actual errors on most checkers. The missing piece is those elements (parts of the software) that cannot be systematically seen or understood, alongside human feelings. Below are a few examples to give an indication, and they would probably be much better presented in a table, but I'll do my best. Below, I'll try to explain the area, what they can and cannot do.

Keyboard navigation: Checkers can confirm if elements exist, but cannot tell if they are all reachable, in a logical order, if there are 'traps' that you cannot navigate away from (think pop-ups with no escape or close button), or if something might feel 'off', 'wrong', or 'frustrating' to a person.

Alternate text (Alt-text): Checkers can confirm that text for a screen reader to announce is present. It cannot understand if the text is actually descriptive, makes sense, or is not needed at all. It is a common myth that all images need alt-text. Most are decorative, so they do not. They also cannot tell whether your user journey needs additional text specifically for screen readers to make it comparable.

Visual indicators: Knowing where you are on the page as a keyboard user is very important. Not many checkers can tell if they are present. None can say if they are active on focus, sufficient to be seen or have the correct contrast.

There are lots more areas they cannot help with. Confusing layouts, missing context, readability, usability, state changes, menu interactions, zoom or magnification. They are useful, but only as a smaller part of any strategy. Hope that helps, Gary, and anyone else reading.

Check out more answers in the AMA about Accessibility Testing collection.