Sampling & Statistics
Confidence Interval
Definition
A range around a sample-based estimate that is likely to contain the true value, at a stated confidence level.
Why it matters
A recall or elusion estimate without an interval is only half a number.
In practice
Any estimate from a sample comes with uncertainty. A confidence interval states it as a range: at a 95% confidence level, the method used to build the interval captures the true value in 95% of repeated samples.
Intervals get narrower with larger samples and wider with higher confidence levels. Near 0% and 100% the simple normal approximation breaks down, which is why rare events like elusion are usually reported with an exact Clopper-Pearson interval, which is conservative, or a Wilson score interval.
Reporting recall as "92% (95% CI 88% to 95%)" says more, and invites fewer questions, than "92%".
Formula
Normal approximation: p ± z × √(p × (1 − p) ÷ n), with z = 1.96 for 95%
Worked example
A sample of 1,500 documents finds 180 responsive: 12%. The 95% interval is about 12% ± 1.6%, or 10.4% to 13.6%.
Illustrative numbers, not from a real matter.
Compare with
- Margin of Error: The margin of error is half the interval width, the "± 1.6%" part.
- Sampling Error: Sampling error is the uncertainty itself; the interval is how you report it.
Work it out
- Richness & Yield calculator · How rich is this collection, and what yield should reviewers expect?
- Prevalence Results calculator · How much responsive material is in this population, and how big is the review?
- Elusion & Recall calculator · What did we leave behind, and what recall can we defend?
- Control Recall calculator · Does a known control set confirm our recall?