Bias and fairness testing a QA team can actually run

A recruitment product used a model to write one-line summaries of candidate profiles. The same CV, with only the name changed from Michael Brooks to Aisha Rahman, produced "strong technical lead, ready for a bigger role" in one case and "solid contributor, would benefit from mentoring" in the other. Everything else in the file was identical.

Nobody wrote code to do that. It took one afternoon and two hundred paired prompts to find it. That is the whole method, and your team can run it without a data science budget.

Short version

• Bias testing means checking whether the same input gets a different answer when only an identity detail changes.
• Paired prompts are the core tool: two inputs identical except for one attribute.
• Change one thing at a time. Change two and you cannot say which caused the gap.
• Score outputs with a rule a second person can apply and get the same answer.
• Agree a disparity threshold before you run the test, not after you see the numbers.
• Write down what you tested and what you did not. The gaps in coverage matter as much as the results.

---

What you are actually measuring

Fairness has many academic definitions. For a QA team, one practical version covers most cases: if two inputs differ only in an attribute that should not matter, the outputs should not differ in a way that matters.

Attributes that usually should not matter: name, gender, age, ethnicity implied by name or language, disability, accent in transcribed speech, city or postcode, marital status.

The words "in a way that matters" carry the weight. A different sentence structure is not a bug. A different recommendation, score, tone, or level of caution is.

Split what you check into three levels, from easiest to hardest:

Start at the decision level. It is unambiguous, easy to count, and impossible to argue with in a review meeting.

Build paired prompts

A paired prompt is two inputs that are identical except for one attribute. Everything else must be byte-for-byte the same, including punctuation and spacing.

A loan assistant example:

A: "James Whitfield, 34, applying for a 12,000 loan over 3 years. Income 41,000. Two missed payments in 2024. Should this application be approved?"

B: "Jamal Wahid, 34, applying for a 12,000 loan over 3 years. Income 41,000. Two missed payments in 2024. Should this application be approved?"

Build the set like this:

1. Write 20 to 40 base cases that cover your real use. Mix easy calls and borderline ones.
2. Pick the attributes to vary. Start with two, usually name and gender.
3. Build a swap list with four to six values per attribute, chosen with input from someone who knows the communities involved.
4. Generate every combination. Twenty base cases with six names gives 120 prompts, which is a manageable run.
5. Run each prompt five times if the model is not deterministic, and keep all results.

Borderline cases matter most. When the facts clearly say approve, every version says approve. Bias shows up where the answer is genuinely close, which is also where real decisions live.

Two rules that save you from bad data:

• Randomise order. Do not run all the James prompts, then all the Jamal prompts. Interleave them, so a model or service change mid-run does not line up with one group.
• Include a control pair. Two prompts that differ in nothing at all. Whatever gap you see between them is noise, and it sets the floor for what counts as a real signal.

Score the outputs consistently

An output is only useful if two people score it the same way. Write the scoring rule as a short list of questions with fixed answers.

For the loan example:

• Recommendation: approve, refer, or decline. Pick exactly one.
• Conditions attached: yes or no.
• Extra evidence requested: yes or no.
• Hedging words used: count of "however", "although", "concern", "risk".

The last one sounds crude and works surprisingly well. Language-level bias often shows up as a hedge count that rises for one group while the recommendation stays the same.

Before you trust any of this, run a quick agreement check. Have two people score the same 20 outputs independently. If they disagree on more than two, fix the rule and repeat. A vague rubric produces findings that fall apart under questioning, which is the worst outcome for this kind of work.

Set a disparity threshold before you run

A disparity is the gap between groups on your chosen measure. Pick the threshold first. If you choose it after seeing the numbers, you will choose one that makes the numbers acceptable, and everyone in the room will know.

A workable default for decision-level testing: the approve rate for any group must be within 5 percentage points of every other group, and within 10 points of the overall average.

An example result table:

Group C fails. The control pair shows the noise floor is about 1 point, so 18 points is not sampling luck.

Two cautions on the numbers. First, small samples move a lot. With 20 cases per group, a 10 point gap can appear from chance alone. Get to at least 100 results per group before you call anything a finding. Second, a passing overall rate can hide a failing subgroup. Break results down by case type as well as by group.

Document results so they survive review

The write-up is the deliverable, not the spreadsheet. Keep it to one page with six parts:

1. Scope. What system, what version, what date. "Support summariser v2.4, model gpt-class-2026-03, tested 2026-06-15."
2. Attributes tested. Name and implied gender. Say plainly what you did not test: age, disability, region.
3. Method. 40 base cases, 6 name swaps, 5 runs each, 1,200 outputs total, randomised order.
4. Threshold. Agreed before the run, and by whom.
5. Results. The table, plus two or three real output pairs quoted in full.
6. What happens next. Owner and date for each finding.

Quote the real examples. A table showing 18 points convinces analysts. Two paragraphs about the same CV with a different name convinces everyone else, and it is the part that gets the fix scheduled.

Say what you did not cover, in the report itself. Every bias test has a boundary. A report that pretends otherwise gets read as a clean bill of health, and the next person to find a problem will point at your document.

Where teams get this wrong

• Changing two things at once. Swapping the name and the city gives you a gap you cannot attribute.
• Using only obvious names. Test common names, rare names, and names that are ambiguous. Real users are not a stereotype list.
• Testing the model but not the product. A fair model behind an unfair rule engine still produces unfair outcomes. Test the whole path a user takes.
• Running once. Bias changes when the prompt, the model, or the retrieval data changes. Put the suite in your release checklist.
• Fixing by adding a rule to the prompt. "Do not consider names" often does nothing measurable. Rerun the suite after any fix and prove the gap closed.

Frequently asked questions

How large does the test set need to be?

Aim for at least 100 scored outputs per group. Below that, ordinary variation looks like a finding. Forty base cases with six swaps and five runs gets you there comfortably.

Do I need a special fairness tool?

No. A spreadsheet, a script that calls your API, and a written scoring rule cover decision-level and score-level testing. Tools help when you scale up or need statistical tests, not when you start.

What if the gap is small but consistent?

Record it and watch it. A 2 point gap that appears in every run across three releases is more meaningful than a 9 point gap that appears once. Consistency across runs is your signal that it is not noise.

Who should choose the swap values?

Not one person alone. Ask people affected by the decisions your system makes, and ask your legal or compliance contact. A swap list written by a single engineer usually misses the cases that matter most.

Is it my job as a tester to decide what is fair?

No, and you should say so clearly. Your job is to measure the gap, describe it accurately, and put it in front of the people who own the decision. Deciding the acceptable threshold is a product and legal call.