Workflow & Validation

Elusion

Definition

A sample-based estimate of what responsive material may remain in the unreviewed set.

Why it matters

It informs stopping decisions but never proves a perfect review.

In practice

Elusion is the check on what review left behind. Take a random sample from the documents that were not reviewed or were predicted non-responsive, review it, and count the responsive documents you find.

That count gives an elusion rate, which can be projected to an estimated number of missed documents and combined with what review found to estimate recall.

An elusion test can only see what is in the set it samples. Documents removed earlier, for example by search terms or date filters before TAR, are outside it unless they are sampled too.

Formula

Estimated recall = found ÷ (found + elusion rate × documents set aside)

Worked example

Review found 18,400 responsive documents. A sample of 1,500 from the 160,000 set aside finds 4 responsive: an elusion rate of 0.27%, or about 430 missed documents, for estimated recall of 97.7%. At the upper end of a 95% interval the sample allows roughly 1,100 missed documents, which would put recall near 94%.

Illustrative numbers, not from a real matter.

Compare with

  • Elusion Rate: Elusion is the test; the elusion rate is the number it produces.
  • Recall: Elusion measures what was left behind. Recall needs both that and what was found.

Work it out

← Back to glossary