Last week, I wrote about the cost of reworking a feature because an accessibility issue has been identified. Reducing this cost will improve collaboration between designers and developers and benefit not just accessibility, but also the overall quality and efficiency of your development process.
Today, I thought it would be good to look at a more user-centric metric and an important leading indicator for accessibility maturity.
The number of customer complaints related specifically to accessibility since the last release
So far, the two metrics we looked at didn't have anything to do with your customers. But the signals coming from them are the most important indicators of accessibility you can have. This direct feedback from your customers about the real-world impact of your accessibility efforts will highlight areas where you still need to work on.
You can run all the automated and manual tests you want. Those results are still lab results. They're not real-world uses cases. The fact is, the way you and I use a screen reader can differ quite a lot from how an every-day screen reader user relies on their software. The way I think about it is, just because I have a drivers' licence doesn't mean I have the reflexes of a Formula 1 racing driver.
So unlike automated tests and other internal audits, customer feedback provide real-world insights into how accessibility issues affect your actual users. They will also come across accessibility issues you didn't catch during testing, especially when using specific assistive technologies or in particular use cases.
The number of customer complaints measures real impact and is a good compliance indicator. A high number of complaints could signal legal risks and directly shows how accessibility is affecting user satisfaction.
How do you track this
This one is quite straight forward to track. For each release, keep a tally of the number of customer complaints. You don't need to prioritise anything at this point, but do your best to triage issues. If something is critical, then it means customers can't work with your website at all. Everything else should go in the same bucket.
The goal is to track this number across releases and see it go down.
Strategies to reduce the number
- Proactive testing. Before you release, do some manual accessibility testing, including testing with a diverse group of users with disabilities.
- Accessibility-focused sprints. If you're overwhelmed with too many, consider dedicating specific development sprints to addressing known accessibility issues.
- Automated checks. Implement automated accessibility checks in your CI/CD pipeline.
- User testing. Recruit some users with disabilities and conduct regular usability testing sessions.
- Documentation. It may be that your team doesn't know what they need to do. Give them access to clear accessibility documentation.
But wait, what if I have no customer complaints?
There may be a few reasons for this.
- You did such a great job, everyone's happy. Good for you!
- You don't have enough traffic and therefore not enough data points. That's okay. As you grow, look out so that number doesn't go up.
- There's no way for customers who encounter problems to feedback accessibility issues specifically. Review your accessibility statement and make sure there's a way for them to get in touch with you.
As you focus on reducing the number of accessibility-related customer complaints, you'll often see broader improvements in your development process. You'll also develop stronger connections with your user base, leading to more loyal customers and positive word-of-mouth recommendations. It's likely a differentiator in the market.