At its core, accessibility is about ensuring that digital products can be used by everyone, regardless of disability. This is the long term overarching goal. The problem with thinking long term is that you'll never know you got there until you're past that point.
This makes it hard to measure progress towards the goal. If you can't measure where you currently are towards the goal, you don't have any idea if you're going in the right direction and when you'll get there. You can't make any adjustments to your current process either.
If you don't know if you need to make adjustments and what those adjustments are, you're throwing darts in the dark. You might hit a target, but you don't know if it's the right target until you turn on the lights.
Success often hinges on the combined impact of small habits and regular routines. That's to say, ensuring your website is accessible depends on the work you put in, starting from the planning stages, all the way to testing and releasing new features for your users.
This is great news, because inputs are easier to track and measure. They are also the only thing you do have control over.
This led me to the idea of leading and lagging indicators, and which metric to use when.
Lagging indicators
A lagging indicator measures current performance. It will tell you how far away from your goal you are. I consider an audit, a completed VPAT or any document that describes the current state of your website a lagging indicator. It's easy to measure, but you have no control over changing its result. Even worse, you have no control over the result of any future similar measurement.
In other words, running axe DevTools on your website won't tell you what will happen if you run the same scan on the same page in 2 months.
Leading indicators
A leading indicator on the other hand looks at the short term benchmarks you can implement to inform how likely you are to meet the overall goal.
- The number of critical accessibility issues identified in production
- The number of accessibility issues in your backlog and how long they have been sitting there
- How early in the software development lifecycle you identify an accessibility issue
- The cost of reworking a feature because an accessibility issue has been identified
- The number of new accessibility issues your latest release introduced
- The number of customer complaints related specifically to accessibility since the last release
All these metrics are critical in informing you if you are on the right track. Notice that none of them will tell you directly how accessible you are, how compliant you are or how many checkboxes of the WCAG you check.
Of course, you might rely on some of the same tools you would use the track lagging indicators to measure these. The difference is, you now have control over what to do with those results right now. This is where you can change your process, your inputs, and affect the results next time you run through those measurements.
Leading indicators have an added benefit. Let's say you stick to measuring just lagging indicators, like website accessibility. You're likely to miss small, but important bits, such as costs associated with delivering an accessible website and longer delivery times because of costly redesigns and rework. By tracking these leading indicators you're likely to identify possibilities for reaching your goal faster and cheaper.
If you’re using lagging indicators without leading indicators, you're missing out on some big potential gains. You won't have a complete picture without measuring both either.