Metrics
Depth for Recall
Definition
How far down a ranked list you must review to reach a target recall level.
Why it matters
It connects a recall goal to the actual review volume and cost it implies.
In practice
Depth for recall is how much of a ranked collection you have to review to reach a given recall. It turns a recall target into a review volume, and so into cost and time.
It depends on two things: how rich the collection is and how well the model ranks it. A perfect ranking reaches a target as soon as it has reviewed that many responsive documents; real rankings need to go deeper.
Formula
Depth = documents reviewed to reach the target recall ÷ documents in the collection
Worked example
A 200,000-document collection is 10% responsive (20,000 documents). Reaching 80% recall means finding 16,000 of them. A perfect ranking would get there at 16,000 documents, a depth of 8%; if the real model needs the top 40,000, depth for 80% recall is 20%.
Illustrative numbers, not from a real matter.
Compare with
- Recall: Recall is the completeness goal; depth is what it costs to reach it.