Bug metrics that don't lie

A QA lead puts a slide up: 412 bugs found this quarter, up from 380. The room nods. Nobody knows what to do with that.

Then someone asks how many of those 412 are still open, and how old the oldest one is. The room goes quiet, because that answer would have changed the plan.

Short version

• A good bug metric changes a decision. If nobody would act differently at 8 percent versus 18 percent, stop tracking it.
• Four metrics carry most of the value: escape rate, bug aging, time to fix, and detection efficiency.
• Report each one with severity split. Totals hide the bugs that matter.
• Bug counts per tester, bug counts per developer, and total bugs found are vanity metrics. They reward the wrong behaviour.
• Track trends over quarters. Single-sprint movement is mostly noise.
• Never attach a bug metric to an individual's performance review.

---

The test for a useful metric

Before adding any metric to a dashboard, answer three questions:

1. What decision does it change? If the answer is "none", drop it.
2. Can someone improve the number without improving the product? If yes, someone eventually will.
3. Does it survive being split by severity? Totals that mix critical and cosmetic bugs are almost always misleading.

The four metrics below pass all three. Most popular alternatives fail question two.

Metric 1: escape rate

What it measures: the share of bugs that users found instead of your team.

Formula: escaped bugs / (bugs found in testing + escaped bugs) x 100

An example. In Q2, testing found 240 bugs and users reported 31 within 30 days of each release. Escape rate = 31 / 271 = 11 percent.

How to read it: this is the single best summary of whether testing is happening in the right places. A rising escape rate with a flat bug count usually means testing effort moved away from where the risk is.

Traps:

• Fix your measurement window and never change it. Thirty days after release works for most web products.
• Count only bugs that were findable in your test environment. A payment provider outage is not an escape.
• Split by severity. An 11 percent total that hides 25 percent escape on critical bugs is worse than a flat 15 percent.

Target: most product teams sit between 5 and 15 percent. Improvement matters more than the absolute value.

Metric 2: bug aging

What it measures: how long open bugs have been sitting, right now.

This is a snapshot, not an average. Averages hide the worst cases, and the worst cases are the problem.

How to report it: a table of open bugs by age band and severity.

How to read it: read the bottom two rows first. Two high-severity bugs older than 90 days is a decision that was never made, not a queue that is moving slowly. Somebody looked at them, did not want to fix them, and did not want to close them either.

The 164 low-severity bugs over 90 days are also a decision that was never made. They will never be fixed. They cost triage time every time someone searches the backlog, and they make every other count look worse.

What to do with it: run an aging review each month. For every bug over 90 days, choose fix, schedule with a date, or close as won't fix. "Leave it open" is not an option.

Metric 3: time to fix

What it measures: how long a bug takes from confirmed to verified fixed.

Formula: use the median, not the mean, and split by severity. One bug that sat for 400 days will drag a mean into nonsense.

Define your start point and write it down. Common choices are the creation date and the date the bug was confirmed in triage. The confirmed date is usually fairer, because a bug filed on Friday night that nobody has read yet is not a slow fix.

How to read it: compare the critical row against your incident promises. If your support agreement says critical issues are addressed in 24 hours and your median is 1.2 days, half your critical bugs miss the commitment.

Trap: this metric pushes people to fix easy bugs first. Watch it alongside bug aging. A falling time to fix with a growing 90-day pile means the hard bugs are being skipped, not solved.

Metric 4: defect detection efficiency

What it measures: how well a specific stage catches bugs before passing work along. It is the same idea as escape rate, applied stage by stage.

Formula: bugs found in a stage / (bugs found in that stage + bugs found in all later stages) x 100

An example for one release:

How to read it: each row tells you what that stage is worth. Manual QA is catching nearly three quarters of what reaches it. Code review is catching under a quarter, which is normal but tells you not to rely on it as a safety net.

What to do with it: this is a budgeting metric. It tells you where another hour of effort pays back most. If automated tests catch 45 percent and take two days of maintenance a week, you can argue about that trade with numbers instead of opinions.

Trap: it requires you to record where each bug was found. Add a required "Found in stage" field, or you cannot calculate this at all.

The metrics to drop

Bugs found per tester

The most common vanity metric, and the most damaging. It rewards volume, so testers split one bug into four tickets, file cosmetic issues they would normally skip, and avoid deep exploratory work that produces few but serious findings.

A tester who spends three days finding one data-corruption bug looks terrible on this chart. That tester is your best tester.

Bugs caused per developer

This reliably produces two behaviours: developers argue about whether each bug is a real defect, and developers stop taking on risky work. It also punishes whoever owns the oldest, most fragile module, which is exactly the person you need engaged.

Total bugs found

A number with no denominator. Did 412 bugs mean a bad quarter, or a big release, or a new tester who is finding things everyone else missed? You cannot tell.

If you want a volume signal, use defect density — bugs per unit of work — so the number scales with what you shipped.

Test case pass rate

Reported as "98 percent of tests passed", it sounds like quality. It measures the tests, not the product. A suite of 500 shallow tests passes at 100 percent while the checkout flow is broken.

How to present these without starting a fight

• Show four numbers, not fourteen. A dashboard nobody reads is worth zero.
• Put the trend next to the value. "11 percent, down from 16 percent last quarter" is a story. "11 percent" is trivia.
• Name the release context in the same sentence. A rushed hotfix quarter will look worse, and saying so first keeps the conversation on the process.
• Attribute to modules and stages, never to people.
• Bring one proposed change per review. Reviews that produce six actions produce zero completed actions.

Frequently asked questions

How many bug metrics should a QA team track?

Three or four. Escape rate and bug aging cover most decisions. Add time to fix if you have response commitments, and detection efficiency when you are deciding where to invest effort.

Should I use mean or median for time to fix?

Median, always, and split by severity. A handful of very old bugs will pull a mean far away from the typical experience.

How often should these be reviewed?

Bug aging monthly, because it drives cleanup decisions. Escape rate and detection efficiency quarterly, because sprint-level movement is mostly noise.

Can bug metrics go in performance reviews?

No. Every bug metric can be improved by changing how bugs are counted, so tying them to reviews reliably corrupts the data you need.

What if our tracker does not record where a bug was found?

Add a single required dropdown with four or five values. Without it, detection efficiency cannot be calculated, and escape rate becomes guesswork.