Root cause analysis for QA: 5 whys and fishbone on real bugs

A customer is charged 108.00 USD for an order that should cost 107.99 USD. A developer finds the rounding bug in twenty minutes and ships a fix. Case closed.

Six weeks later the same class of bug appears in the refunds screen. Then in the invoice PDF. The fix worked. Nothing stopped it happening again, because nobody asked why it happened the first time.

Short version

• Root cause analysis (RCA) is the practice of finding why a defect was possible, not just what line of code was wrong.
• Run RCA on a small number of bugs — escaped, repeated, or expensive ones. Not on everything.
• 5 Whys works for a single defect with a clear chain. A fishbone diagram works when several causes combine.
• A good RCA ends with a change to a process, a test, or a tool. If it ends with a person's name, you are not finished.
• Every action needs an owner and a date, or the RCA becomes a document nobody reads.
• Measure whether the same cause returns. That is the only proof the RCA worked.

---

When to run an RCA

RCA takes 30 to 60 minutes of several people's time. Spend it where the payoff is real:

• The bug reached production and a customer noticed.
• The same kind of bug has now appeared three times.
• The fix took over a week, or the bug was reopened twice.
• The bug caused data loss, a wrong charge, or a security exposure.

Skip it for typos, one-off environment glitches, and anything one person can explain in a sentence.

Method 1: the 5 Whys

You state the problem, then ask "why" until you reach something you can change. Five is a rough guide, not a rule. Sometimes it is three, sometimes seven.

Two rules keep it honest:

Each answer must be a fact you can point at. Not "the developer was careless" but "the pricing helper rounds each line separately, and the totals endpoint sums the rounded values".

Stop when the answer is a process, not a person. "Because Marco forgot" is never a root cause. The real answer is that nothing in the process would have caught the forgetting.

Worked example 1: the rounding bug

Problem: Order 88214 was charged 108.00 USD. The correct total was 107.99 USD.

1. Why was the charge wrong? The totals endpoint rounded each line item to two decimals, then summed them. The correct order is sum first, round last.
2. Why did the code round in that order? The formatCurrency helper returns a rounded string, and the totals code reused it for arithmetic instead of formatting.
3. Why was a formatting helper used for arithmetic? It is the only currency utility in the codebase, and its name does not signal that it loses precision.
4. Why did no test catch it? All money tests use values like 10.00 and 25.50, which round cleanly. No test used a value that exposes the difference, such as three lines at 35.995 USD.
5. Why did the test data avoid those values? Test fixtures were written by copying an existing seed file. Nobody had ever specified which money edge cases must be covered.

Root cause: there was no agreed set of money edge cases for tests, and no separation between currency formatting and currency arithmetic.

Actions:

• Add a Money type that does arithmetic in integer cents. Rename formatCurrency to formatCurrencyForDisplay. Owner: Ana. Date: 22 July.
• Write a shared money fixture with the edge cases: values ending in .995, negative amounts, zero, and amounts over 1,000,000. Owner: Ana. Date: 22 July.
• Add a checklist line to the pull request template for any change touching totals: "Tested with non-clean rounding values." Owner: Dev lead. Date: 15 July.

Notice what is not in that list: "be more careful with rounding". That is advice, not a change.

Worked example 2: the escaped session bug

Problem: Users on the reports page were logged out after 15 minutes of reading, losing unsaved filters. Found by three customers, not by QA.

1. Why were users logged out? The access token expires after 15 minutes, and the silent refresh only runs when a request is sent. The reports page sends no requests while you read.
2. Why did the refresh depend on outgoing requests? The refresh logic lives in the API client interceptor, which only runs on a request. There is no timer.
3. Why was that design not questioned? The token lifetime was reduced from 60 minutes to 15 minutes in a security change three months ago. The refresh design was written when 60 minutes made idle expiry unlikely.
4. Why did QA not catch it? The regression suite for authentication signs in and performs an action within two minutes. No test sits idle.
5. Why is there no idle test? Test cases are written from user stories, and no story ever described a user reading a page for 20 minutes.

Root cause: a configuration change altered an assumption in unrelated code, and nothing in the process links config changes to the behaviour that depends on them.

Actions:

• Add a timer-based refresh, independent of request activity. Owner: Backend team. Date: 29 July.
• Add one long-idle test to the auth regression suite: sign in, wait past token lifetime, then act. Owner: QA. Date: 29 July.
• Add a rule to the change process: any change to timeouts, limits, or token lifetimes requires a listed set of affected behaviours in the ticket. Owner: Tech lead. Date: 18 July.

The third action is the one that prevents the next unrelated version of this bug.

Method 2: the fishbone diagram

5 Whys assumes one chain. Some bugs happen because four weak things lined up. A fishbone diagram, also called an Ishikawa diagram, sorts possible causes into categories so you can see the combination. These six categories work well for software:

Worked example 3: the CSV import that keeps failing

Problem: Customer CSV import has failed in production four times in two months, each time with a different symptom. One import silently created 4,000 duplicate contacts.

Causes found in each category:

• Requirements: the spec says "import a CSV of contacts". It never defines maximum file size, required columns, encoding, or duplicate handling.
• Code: the parser assumes UTF-8 and comma separators. Files from older spreadsheet tools arrive as UTF-16 with semicolons.
• Tests: one test file exists, with 12 clean rows. Nothing covers 50,000 rows, missing columns, or duplicate emails.
• Environment: the import runs in a web request with a 30-second gateway timeout. Large files fail halfway, after some rows are written.
• Data: real customer files contain trailing commas, quoted line breaks inside fields, and emails that differ only by capitalisation.
• Process: the feature shipped behind no flag, and support tickets were closed one by one without anyone grouping them.

No single cause explains four different failures. The failure mode is that an undefined feature met real data with no safety net.

Root cause: the import feature has no written contract for its input, and no test data resembling real customer files.

Actions:

• Write the input contract: required columns, accepted encodings and separators, maximum rows, duplicate handling rule. Owner: Product. Date: 25 July.
• Move the import to a background job with progress and a transaction, so a partial failure rolls back. Owner: Backend. Date: 8 August.
• Build a test data set from five anonymised real customer files. Owner: QA. Date: 1 August.

Keeping the analysis blameless

Two habits do most of the work.

Ask about the system, not the decision. Instead of "why did you merge that", ask "what would have shown this before merge". The first question makes someone defend themselves. The second produces a check.

Assume everyone acted reasonably with what they knew. People rarely make bad choices on purpose. They make ordinary choices in a system that did not give them the information they needed. If the answer is "they should have known", write down how they were supposed to find out. If there was no way, that gap is your finding.

State the rule at the start of every session: no action item may name a person as the fix. Owners are named. Causes are not.

Getting evidence good enough to analyse

RCA falls apart when the ticket says "checkout was broken for a customer yesterday". You cannot trace a chain from that.

Ask for the same minimum on every escaped bug: the exact URL, the account, the browser and version, the build number, the console errors, and the failing network request with its status code. Capture tools such as Crosscheck attach the screenshot, console logs, network requests, and environment details automatically when a bug is reported from the page, so the RCA starts with facts instead of recollection.

Making the actions stick

An RCA with no follow-through is worse than none. It burns the team's willingness to do the next one.

1. Cap it at three actions. More than three means none get done.
2. Give each action one named owner. Shared ownership is no ownership.
3. Put each action in the same backlog as feature work, with a date.
4. Review open RCA actions at the start of the next RCA session.
5. Six weeks later, check whether the same cause has produced another bug. That is your result.

If the same cause returns, the RCA found a symptom, not a root cause. Run it again and go one level deeper.

Frequently asked questions

How many bugs should get a full root cause analysis?

Very few. Most teams do well running RCA on escaped production bugs, repeat offenders, and anything expensive. Two or three sessions a month is a healthy rate.

When should I use 5 Whys instead of a fishbone diagram?

Use 5 Whys when one clear chain of events led to the defect. Use a fishbone when several weak spots combined and no single chain explains what happened.

How do I stop an RCA turning into blame?

Set the rule before you start: no action item may name a person as the fix. Ask what would have caught the problem, not why someone missed it.

Who should attend an RCA session?

The developer who fixed it, the tester who found or missed it, and someone who knows the requirement. Five people maximum, since larger groups produce discussion but not decisions.

How do I know the RCA worked?

Track whether the same cause produces another defect in the following quarter. Recurrence is the only honest measure, and it is more useful than counting how many RCAs you ran.