Sampling & Statistics

Confidence Interval

Definition

A range around a sample-based estimate that is likely to contain the true value, at a stated confidence level.

Why it matters

A recall or elusion estimate without an interval is only half a number.

In practice

Any estimate from a sample comes with uncertainty. A confidence interval states it as a range: at a 95% confidence level, the method used to build the interval captures the true value in 95% of repeated samples.

Intervals get narrower with larger samples and wider with higher confidence levels. Near 0% and 100% the simple normal approximation breaks down, which is why rare events like elusion are usually reported with an exact Clopper-Pearson interval, which is conservative, or a Wilson score interval.

Reporting recall as "92% (95% CI 88% to 95%)" says more, and invites fewer questions, than "92%".

Formula

Normal approximation: p ± z × √(p × (1 − p) ÷ n), with z = 1.96 for 95%

Worked example

A sample of 1,500 documents finds 180 responsive: 12%. The 95% interval is about 12% ± 1.6%, or 10.4% to 13.6%.

Illustrative numbers, not from a real matter.

Compare with

  • Margin of Error: The margin of error is half the interval width, the "± 1.6%" part.
  • Sampling Error: Sampling error is the uncertainty itself; the interval is how you report it.

Work it out

← Back to glossary