How Many Runs Do You Actually Need for a Design of Experiments?

Part of our complete guide to DOE for biologists.

Ask ten people running a DOE how many runs they need and most will give you a number that came from a template, a colleague's old protocol, or "whatever fits the plate." It's rarely calculated — because the calculation usually gets presented in terms of alpha, beta, and effect size, none of which map cleanly onto a lab decision. The actual question is simpler than the statistics vocabulary makes it sound.

The only two numbers you actually need

How much your assay's result normally varies between identical repeat runs. You likely already know this, even if you've never written it down — it's the spread you see when you run the same sample twice and get two slightly different numbers.

The smallest improvement that would actually be worth acting on. Not the improvement you're hoping for — the smallest one that would change what you do next. If a 2% titer improvement wouldn't change your process, but a 10% one would, your target is 10%, not 2%.

Why both numbers matter together

A design's ability to detect an effect depends on how that effect compares to the noise around it, not on the effect size alone. A 10% improvement is easy to detect against 2% noise and nearly invisible against 15% noise. This ratio — how big your target improvement is relative to your typical noise — is what actually determines whether a given run count has a real chance of finding it.

What happens if you skip this step

Run too few, and a real effect gets lost in the noise — you conclude a factor "doesn't matter" when the honest answer is "we didn't have enough runs to tell." This is the most common way a good experiment produces a misleading result, and it's silent: nothing about the output looks wrong, it just quietly under-detects. Run far more than you need, and you've spent weeks and reagents on precision beyond what the decision actually required.

A worked example

Say your assay's result typically varies by about 2.5 units between repeat runs, and you've decided a 5-unit change is the smallest one worth acting on. That's a signal roughly twice the size of your background noise — a solidly detectable ratio, and the calculation will tell you the minimum run count that gets you to a reasonable (commonly ≥80%) chance of actually catching it at that ratio. Push your target down to a 2-unit change against the same 2.5-unit noise, and the required run count climbs fast — you're asking the design to find a signal barely bigger than the noise around it.

This isn't a one-time calculation

If you later split a design into blocks, or flag a variable as Hard-to-Change, you've spent some of your design's degrees of freedom on handling that constraint — which means the same run count may no longer hit the power you originally calculated. It's worth revisiting this after those decisions, not just before you generate the design in the first place.

BioDOE calls this the Reliability Check — plug in your two numbers and it tells you straight away whether your planned run count can do the job, or what it would take to get there. Try it free.