If your accessibility backlog dropped from 40 tickets to 15, we should all be ecstatic, right? Right?
I mean, that's what most people would call a job well done.
Not me. I'd have a few questions before calling it progress.
What did you close? Did you check the fixes? How? Are you still looking for problems as carefully as before or did you start ignoring them before your backlog is thinner now?
Closed tickets show you completed work. Not all tickets are equal though. Correcting or adding a missing label and making an appointment flow usable might each close a ticket, but their impact on patients could be very different.
The numbers also depend on how much you test and what you test with. If you've never tested with a screen reader before, starting to will suddenly lead to a report of more issues. I'd welcome that visibility, even though the chart looks worse.
It's not that the backlog count doesn't matter, but you need to put a short explanation right beside that number. If you closed eight issues around the booking flow and keyboard users can complete it now, but you haven't yet tested how cancelling a booking would work with a screen reader, say so. I'd much rather know what's improved and what's uncertain. It also gives us something useful to discuss when deciding what comes next.
We don't need an elaborate measurement programme to start this. Just a bit more context around the numbers will do fine. How much context? Just enough to avoid rewarding people for a number.
As much as I hate backlogs, I hate large backlogs even more. That's why a smaller backlog is encouraging for me, but only when we understand why and how it got smaller.
On its own, it only tells us how many tickets are still to do. It doesn't tell us how many people we've helped and how useful our efforts really are.