Core Concepts
Recall
Definition
The share of truly responsive documents that were found by the workflow.
Why it matters
Often the headline metric in defensibility discussions.
In practice
Recall answers the completeness question: of all the responsive documents in the collection, what share did the review find?
Nobody knows the true number of responsive documents, so recall is always estimated from a sample. The usual routes are an elusion sample of the documents set aside, a control set coded at the start, or a stratified sample of the whole collection. Each estimate should come with a confidence interval.
Protocols often name a recall target, and the debate tends to focus on how it was measured as much as on the number itself.
Formula
Recall = responsive documents found ÷ (responsive documents found + responsive documents missed)
Worked example
A review finds 18,400 responsive documents. An elusion sample of the 160,000 documents set aside finds 4 responsive documents in 1,500, which projects to about 430 missed. Estimated recall is 18,400 ÷ (18,400 + 430), or about 97.7%.
Illustrative numbers, not from a real matter.
Compare with
- Precision: Precision asks how much of what you marked responsive really is. Recall asks how much of what is responsive you found.
- Elusion Rate: Elusion rate is the share of the set-aside documents that are responsive. It is an input to the recall estimate, not recall itself.
Work it out
- Culling Impact calculator · Did culling remove noise without losing responsive documents?
- Precision, Recall & F1 calculator · How accurate are the model or reviewer calls?
- Elusion & Recall calculator · What did we leave behind, and what recall can we defend?
- Control Recall calculator · Does a known control set confirm our recall?