How do I know if an accessibility issue is actually impacting real users or just something that shows up in automated testing?
This is a fantastic question. I've seen first hand how huge the gap between automated testing and real-world impact is. And you're right to question it.
Automated tools catch maybe 40% of actual accessibility issues. That figure varies depending on who you ask. The point remains though. Automated tools don't catch everything. But what matters more is that automated tools also flag stuff that maybe doesn't matter much to your users.
So then, how do you figure out what's actually important?
Talk to real people with disabilities.
Nothing beats watching someone navigate your site with a screen reader or try to use it. Or seeing someone use voice control to fill in a form. User testing reveals the stuff that truly blocks people versus annoyances automated tools flag.
You could look at your analytics too. Are people dropping off at specific points? Unusually high bounce rates on certain pages might signal accessibility barriers, especially if you're seeing patterns. Still, that's not as reliable as actually asking people.
And don't forget that context matters a lot. A colour contrast issue might be flagged by tools, but not impact users if the text isn't critical. But that same issue could be a real problem on a form or navigation.
Here are some of the issues I've seen impact real users:
- keyboard navigation that breaks or traps users
- missing alt text on important images like graphs
- forms that screen readers can't fill in
- basically anything that makes core functionality impossible without a mouse
By all means, fix issues that automated tests find. But honestly, if you can swing it, pay actual users with disabilities to test your site. It's the most reliable way to separate real problems from testing noise.