Core Concepts

Recall

Definition

The share of truly responsive documents that were found by the workflow.

Why it matters

Often the headline metric in defensibility discussions.

In practice

Recall answers the completeness question: of all the responsive documents in the collection, what share did the review find?

Nobody knows the true number of responsive documents, so recall is always estimated from a sample. The usual routes are an elusion sample of the documents set aside, a control set coded at the start, or a stratified sample of the whole collection. Each estimate should come with a confidence interval.

Protocols often name a recall target, and the debate tends to focus on how it was measured as much as on the number itself.

Formula

Recall = responsive documents found ÷ (responsive documents found + responsive documents missed)

Worked example

A review finds 18,400 responsive documents. An elusion sample of the 160,000 documents set aside finds 4 responsive documents in 1,500, which projects to about 430 missed. Estimated recall is 18,400 ÷ (18,400 + 430), or about 97.7%.

Illustrative numbers, not from a real matter.

Compare with

  • Precision: Precision asks how much of what you marked responsive really is. Recall asks how much of what is responsive you found.
  • Elusion Rate: Elusion rate is the share of the set-aside documents that are responsive. It is an input to the recall estimate, not recall itself.

Work it out

← Back to glossary