The Numbers Trap
do you track numbers?
- We have to get those defect counts down!
- We have to reduce those cycle times!
- We have to increase lines of defect-free code written for each release!
We forget that numbers are just a gauge and they don't tell the whole story.
The obsession with increasing/decreasing counts becomes your core purpose and product quality suffers as a result.
Instead of focusing on numbers, we should focus on our core purpose: creating the highest quality product possible.
Here are some things to think about when tracking QA stats:
- numbers are just a gauge
- the numbers may not be accurate for a variety of reasons (changes in tracking/reporting methods, team members with varying skill levels doing the reporting causing duplicate bug reports)
- because the defect descriptions & counts may be inaccurate, using them as the sole measurement of product quality may be misleading.
The most important questions to ask are those that deal with customer satisfaction:
- Are the user numbers increasing or decreasing?
- In a poll of customer satisfaction, are there more satisfied or dissatisfied users?
- Looking ahead, is product development addressing the features most in demand?
- Do we have the resources to create quality code to create those features?
- Do our development and QA teams work in harmony for the core purpose of creating a better product?
Spend time tracking numbers, but spend more time tracking product quality as viewed through your customer's eyes.