How many testers do you really need?

Where does the 5-tester rule come from?

The rule stems from research by Jakob Nielsen and Thomas Landauer (1993): the number of newly discovered usability problems follows the formula N(1−L)ⁿ — where L is the hit rate of a single tester, about 31% averaged across projects. This produces the famous curve: the first tester finds roughly a third of all problems, five testers together find about 85% — and from the sixth onwards, mostly known issues repeat.

This leads to Nielsen's actual recommendation, which is often left out: not “test once with 5”, but “test iteratively” — three rounds of 5 testers uncover more than one large study with 15, because repairs happen between rounds and round 2 reaches the problems that were hidden behind round-1 blockers.

Five testers uncover around 85% of usability problems on average; the first tester alone about a third. From tester six onwards, the novelty value drops rapidly.Nielsen/Landauer 1993; Nielsen Norman Group 2000

The honest caveat: 85% is an average, not a guarantee

The rule has received justified criticism — most importantly from Laura Faulkner (2003). She had 60 people test the same system and randomly drew many different sets of 5 from the results: some sets found 99% of the problems, others only 55%. Only with 10 testers did the guaranteed minimum rise to 80%, with 20 to 95%.

What does that mean in practice? Not “5 is wrong”, but: an individual round of 5 can be lucky or unlucky. That is exactly why the rounds logic beats the big study — bad luck in round 1 is corrected by round 2, and repairs have already happened in between. If you want to be safe, plan two to three small rounds from the start instead of one big one.

The numbers per method at a glance

“5 testers” applies to qualitative think-aloud tests. For other research questions, the Nielsen Norman Group gives different guidance:

Method / goal	Recommended participants	Why
Qualitative user test (find problems)	≈ 5 per round	Insight per head drops rapidly; iterating beats adding
Quantitative test (metrics, e.g. success rate)	≥ 20	Metrics only stabilise from ~20 onwards
Card sorting (navigation/structure)	≥ 15	Patterns in categorisation need more data points
Eyetracking (stable heatmaps)	≈ 39	Gaze data varies strongly between people
Several clearly distinct target groups	≈ 5 per group	New customers fail at different things than pros

And the big brother of the quantitative world: if you want to measure conversion differences (variant A versus B) rather than find problems, you no longer need a user test but an A/B test — and with it, quickly 30,000 visitors. For everyone else, the qualitative test with 5 testers remains the best insight-to-effort ratio.

What the 3×5 rhythm looks like in practice

Round 1 — the big blockers (5 testers)A realistic core task (“buy X”, “request a quote”). Result: the heavy lifting — unclear offer, hidden costs, broken forms.

Fix & round 2 — the second layer (5 testers)Same tasks after the fix. Now the problems appear that were invisible behind the blockers — and you verify your fixes along the way.

Round 3 — fine-tuning or a new question (5 testers)Remaining problems, a new flow or the second target group. After three rounds, your website is measurably a different one.

With Test it Baby, each round is a few minutes of setup: book 5 testers from the DACH panel or invite your own customers, write the task, and the results — recording, transcript, AI summary — usually arrive the same day. Billing is per answer, so a round with 5 testers stays in the double-digit euro range. That turns the research rule into a weekly rhythm.

Frequently asked questions

Are 5 testers really enough for a user test?

For qualitative tests: yes, per round. 5 testers find around 85% of usability problems on average (Nielsen/Landauer). The key qualifier is “per round”: the recommendation is to test repeatedly with 5 testers each — fix the problems after each round, then test again. Three rounds of 5 find more than one round of 15.

Why not simply use more testers at once?

Because the insight gained per additional tester drops rapidly: each further tester mostly finds problems that are already known. Testers 6 to 15 cost money but deliver mostly repetition. The budget is better invested in a second test round after fixing the first findings — that is where the problems appear that were hidden before.

When do I need more than 5 testers?

In three cases: (1) Clearly different target groups — then about 5 testers per group (e.g. new customers and existing customers). (2) Quantitative statements like “success rate 74%” — for those the Nielsen Norman Group recommends at least 20 participants. (3) Special methods: card sorting from 15, stable eyetracking heatmaps from 39 participants.

What does the criticism of the 5-tester rule say?

The most important objection comes from Laura Faulkner (2003): different sets of 5 found between 55% and 99% of the problems — the 85% average is no guarantee for an individual round. The practical answer is not “more testers per round” but “more rounds”: what round 1 misses, round 2 finds after the first fix cycle.

Sources

Jakob Nielsen: Why You Only Need to Test with 5 Users. Nielsen Norman Group, 2000 (based on Nielsen/Landauer, ACM INTERCHI ’93).
Jakob Nielsen: How Many Test Users in a Usability Study? Nielsen Norman Group, 2012.
Laura Faulkner: Beyond the Five-User Assumption: Benefits of Increased Sample Sizes in Usability Testing. Behavior Research Methods, Instruments, & Computers 35(3), 2003.
Nielsen Norman Group: Why 5 Participants Are Okay in a Qualitative Study, but Not in a Quantitative One.