Metrics

Depth for Recall

Definition

How far down a ranked list you must review to reach a target recall level.

Why it matters

It connects a recall goal to the actual review volume and cost it implies.

In practice

Depth for recall is how much of a ranked collection you have to review to reach a given recall. It turns a recall target into a review volume, and so into cost and time.

It depends on two things: how rich the collection is and how well the model ranks it. A perfect ranking reaches a target as soon as it has reviewed that many responsive documents; real rankings need to go deeper.

Formula

Depth = documents reviewed to reach the target recall ÷ documents in the collection

Worked example

A 200,000-document collection is 10% responsive (20,000 documents). Reaching 80% recall means finding 16,000 of them. A perfect ranking would get there at 16,000 documents, a depth of 8%; if the real model needs the top 40,000, depth for 80% recall is 20%.

Illustrative numbers, not from a real matter.

Compare with

  • Recall: Recall is the completeness goal; depth is what it costs to reach it.
← Back to glossary