Test cases or charters: when to stop documenting

A QA team has 2,400 test cases in TestRail. The last full regression run took nine days. About 300 cases now fail for reasons nobody has looked at, because the screens they describe were redesigned last year. Nobody deletes them, because deleting feels like removing coverage.

That team is not testing. It is maintaining a document.

The opposite failure exists too. A team that writes nothing cannot answer "what did you check before we shipped?" and cannot hand a release to anyone else.

The question is not which approach is better. It is where the line sits for your product.

Short version

A test case is a written script. A charter is a mission for a timeboxed session.
Detailed cases are worth it when a miss is expensive or the check is repeated forever.
Charters are worth it when the product changes faster than the document can.
Compare two costs: what maintenance costs you, and what a miss costs you.
Most teams should script 20% of coverage and explore the rest.

Two ways to plan a test

A test case is a written script: preconditions, numbered steps, and an expected result. Someone else can run it and get the same answer.

TC-118 — Discount applies at threshold Precondition: logged in as [email protected], cart empty Steps: 1. Add "Wireless Mouse" ($50.00). 2. Enter code SAVE10. 3. Apply. Expected: order total shows $45.00 and a line reading "Discount 10%".

A charter is a one-line mission for a timeboxed session, usually 60 to 90 minutes. It says where to look and what to look for, but not which buttons to press.

Charter: Explore discount codes on checkout with expired, stacked, and case-changed codes, to find cases where the total is wrong. 90 minutes.

The charter finds things the case cannot, because the tester decides what to try next based on what just happened. The case catches the same regression every release without anyone thinking about it. They are different tools.

The two costs that decide it

Skip the philosophy and price it.

Maintenance cost is what you pay to keep the document true. Every screen change makes some cases wrong. Rewriting a detailed case takes 10 to 20 minutes. Two hundred cases touching a redesigned area is a week of work that finds no bugs.

Miss cost is what happens when a defect reaches users. For a marketing site, a broken layout costs some embarrassment. For a payments API, a wrong rounding rule costs money and a regulator's attention.

The rule follows from this:

Write the detailed case when the miss cost is high and the thing being checked is stable. Use a charter when the miss cost is moderate or the thing being checked is still moving.

A feature in its first month changes weekly. Writing 40 detailed cases for it means rewriting them three times before anyone benefits. Cover it with charters, then script the parts that survive.

By product type

Product type	Miss cost	Change rate	Sensible mix
Payments, banking, insurance	Very high	Low, rules are fixed	70% scripted, 30% charters
Medical or safety software	Very high, regulated	Low	90% scripted, evidence required
Enterprise SaaS, mature modules	High	Medium	40% scripted, 60% charters
Early-stage product, pre-launch	Low, few users	Very high	10% scripted, 90% charters
Marketing site, content pages	Low	High	Charters plus automated link and layout checks
Internal admin tools	Medium	Medium	Charters, script only the destructive actions
Mobile app with staged rollout	Medium	High	Charters plus a scripted smoke set per device

Two lines in this table are worth stressing.

Regulated products are not a judgement call. If an auditor must see what was tested and by whom, you write cases and keep the evidence, even where charters would find more bugs. That is the price of the market you are in.

Early-stage products are the opposite. The most common waste in young companies is a QA hire building a 500-case suite for a product that pivots in month four.

Signs you are documenting too much

Nobody has read a case end to end in a month, but the suite keeps growing.
More than 10% of cases fail for reasons that are not product defects.
A full regression takes longer than the sprint that produced the changes.
Cases contain steps like "3. Click the blue button in the top right", which break on any redesign.
Two people write near-identical cases because searching the suite is slower than writing a new one.
You cannot say which cases found a real bug this quarter.

That last one is the strongest test. Tag every case that catches a genuine defect. After two quarters, the untagged ones are candidates for deletion, unless they cover a risk that has never fired but would be severe.

Signs you are documenting too little

Two testers check the same area and both miss a third area.
Releases depend on one person's memory, and holidays are stressful.
The same defect class reappears twice a year and each time it feels new.
You cannot answer "what did we check?" after an incident.
Onboarding a new tester takes months because nothing is written down.

Charters do not mean no records. A session that leaves nothing behind is untraceable, and untraceable testing is why teams distrust exploratory work.

The hybrid model

This is what most teams should run. Five steps.

List the flows where a failure is unacceptable. Payment, login, data deletion, permissions, anything touching money or personal data. Usually 10 to 20 flows.
Write detailed cases for those flows only, and automate them. These are your smoke and regression sets. They are stable, so maintenance is low and automation pays back.
Cover everything else with charters. One charter per feature area per release, 60 to 90 minutes each, given to a named person.
Record every session in a fixed short format. Charter, tester, time spent, areas covered, bugs found, open questions. Ten lines is enough, and it makes exploratory work auditable.
Promote to a case only when a bug repeats. If the same defect returns, that area has earned a scripted regression test. Write one case, automate it, move on.

Rule 5 is the engine. Your scripted suite grows from evidence rather than from a wish to be thorough, so it stays small and every case in it has a reason to exist.

For step 4, the weak point is usually evidence. A tester finds something at minute 70 of a session and cannot recall the exact steps. Capturing state as you go fixes this: a browser tool such as Crosscheck attaches the screenshot, console errors, network requests, and browser details to a report from the page where the problem appeared, so the session notes stay short and the detail is still there.

Writing a charter that works

Weak charters get weak sessions. Compare:

Weak: Test the reports page.

Strong: Explore the reports page with date ranges that cross a month boundary, a leap day, and a timezone change, looking for wrong totals or missing rows. Account: [email protected] on https://staging.example.com. 90 minutes.

The strong version names the area, the attack, the risk, and the limit. That is all a charter needs. If it grows past four lines, you are writing a test case.

Keep sessions under two hours. Attention drops, and a two-hour session produces the same findings as ninety minutes plus worse notes.

What to do with the 2,400 cases you already have

Do not audit them one by one. That is the trap.

Run the suite once and mark every case that has not passed in six months.
Delete anything covering a screen that no longer exists. This is usually a large share.
Keep every case covering the flows from step 1 of the hybrid model. Automate them if they are not automated.
Move the rest into an archive with a note that they can be restored, then work from charters for those areas for two releases.
After two releases, count what you missed. Usually nothing. Restore only what you actually needed.

Teams that do this typically end with 200 to 400 cases they trust, from a suite of thousands they did not.

Frequently asked questions

Is exploratory testing the same as unscripted clicking around?

No. Exploratory testing has a charter, a timebox, and a record. Clicking around has none of those, which is why it produces results nobody can repeat.

How do I convince a manager that fewer test cases is safer?

Show the numbers: how long a full run takes, how many cases fail for non-product reasons, and how many real defects the suite caught last quarter. The argument is about cost, not about preference.

Can AI write my test cases so maintenance stops being a problem?

It can draft them quickly, which lowers the cost of writing but not the cost of keeping them true. Generating more cases faster can make the maintenance problem worse.

Do charters work for regression testing?

Partly. Use scripted, automated checks for the flows that must never break, and charters to explore around a change for the side effects nobody predicted.

How many charters fit in a release?

Take the testing hours you have, subtract fixed work, and divide by 1.5 hours. A tester with three clear days per release runs about ten to twelve sessions.