Metrics

Sampling Error

Definition

The uncertainty introduced when a conclusion is based on a sample instead of the full population.

Why it matters

It is the reason confidence intervals exist.

In practice

Sampling error is the chance difference between what a random sample shows and what is true of the whole population. Two honest random samples from the same collection will rarely give exactly the same answer.

It is measured by the standard error and shrinks as samples grow. Confidence intervals and margins of error are ways of reporting it.

It is not the only kind of error. Coding mistakes, a sample that was not truly random, or documents left out of the sampled set are non-sampling errors, and a bigger sample does not fix them.

Formula

Standard error = √(p × (1 − p) ÷ n)

Worked example

If a collection is 12% responsive, the standard error for a 400-document sample is about 1.6 points, so one sample showing 10% and another showing 13% is entirely expected.

Illustrative numbers, not from a real matter.

Compare with

  • Margin of Error: The margin of error is the standard error scaled to a confidence level, about 1.96 standard errors at 95%.

Work it out

← Back to glossary