Selective identity retrieval

Retrieval ranks candidates. Many systems then need a second decision: is the top hit the item, or only the nearest neighbor? A wrong commit usually costs more than leaving the list alone.

Hypothesis: a true match stands apart on the list — it is not enough that it scores highest.

What “apart” looks like

Four real lists from the benchmark (not drawn by hand): isolated correct, isolated wrong, crowded correct, and a typical absent list. Green = queried identity on top.

Manipulate the gap

Confidence = how far #1 sits ahead of #2. Drag the threshold used at serve time. The histogram below is the empirical distribution of that gap on present vs absent lists (from the evaluation scores).

Open-set identification

Detection and identification rate against false-accept rate on the mixed present/absent protocol. Separability scores sit well above the top similarity alone.

Risk on present lists

Selective risk as coverage grows — errors among the most confident share of present queries.

At 1% false accepts

Share of present queries correctly identified (DIR). Higher is better.

Mean shift ≈ the gap

Clustering scores and accepting a singleton top mode looked structural. Empirically the critical bandwidth tracks the #1–#2 gap almost perfectly.

What to use

• No labels: how far #1 sits ahead of #2, or the distance ratio.
• Labels: a small model on the list features.
• Several channels: require agreement when false accepts are costly.
• Always: set the threshold on held-out data for a target false-accept rate.

A ranked list should stay a list until the evidence isolates one item.