keep only the k best-scoring RAW candidates per ANON record

Blocking on a key reduces the candidate table before any score is computed. This reduces it *after*: given a score table, it keeps each ANON record's k best candidates and drops the rest. That is useful as a second stage – a cheap score over a blocked candidate set, pruned to k, then an expensive score only on what survived – and it bounds the memory of everything downstream at `k * n_anon` rows regardless of how many candidates the first stage produced.

Usage

top_k_candidates(
  scores,
  k = 10,
  ties = c("keep", "random"),
  seed = NULL,
  tolerance = reid_tie_tolerance()
)

Arguments

scores

a score table (see [score_num()])

k

number of candidates to keep per ANON record

ties

`"keep"` (default) to keep every candidate tied with the k-th, or `"random"` to cut at exactly k, breaking ties at random

seed

integer seed used when `ties = "random"`, or NULL for the ambient RNG stream

tolerance

relative tolerance for deciding that two candidate scores are tied, default `sqrt(.Machine$double.eps)` (Issue #108, following #61), so that the same data in different units is pruned the same way. Pass `tolerance = 0` for the exact `<=` comparison used before #108; see `docs/default-changes.md`.

Value

the score table restricted to the kept pairs, carrying a `blocking` attribute (a "reid_blocking" record).

Details

**Ties are kept, not cut.** If the k-th and (k+1)-th candidates score equally there is no evidence to choose between them, and cutting on row order would drop true pairs for no reason – silently lowering the reported rate. So `ties = "keep"` (the default) returns *at least* k candidates per record, more where the score is flat. `ties = "random"` caps hard at k, and needs a `seed` for the same reason [match_greedy()] does.

"Equally" means *to within `tolerance`*, not bit-for-bit (Issue #108). Until then this was the last tie-deciding function in the package still comparing with a bare `<=`, and the promise above was not kept: re-expressing the same data in 1/10 units turns a tie into a strict ordering (Issue #61), the cut lands on the wrong side of it, and the reported recall moves. On the `docs/` #61 fixture – both ANON records at 42.3, RAW records at 41.2 and 43.4, so every ANON record has two candidates tied at 1.1 in real arithmetic – `k = 1` kept 2 pairs at recall 0.5 instead of 4 at recall 1, and multiplying every value by 10 changed the answer.

Recall is reported the same way as for [block_candidates()]: it is below 1 whenever the true RAW record was not among the k best, which is not rare – that is exactly the "not identified at rank 1 but identified at rank 7" case the top-k hit rate of [reid_evaluate()] measures.

**Candidate pairs are a set.** A repeated (ANON, RAW) pair is rejected here, as it is by [match_greedy()], [match_optimal()], [reid_confidence()] and [reid_evaluate()]. That guard belongs here in particular because this function *removes* the evidence: with `k = 1` and `ties = "random"` it returns one row per ANON record, so the duplicates are gone from its output and every downstream check passes – after the duplicates have taken extra shares of the draw that chose the survivor. Measured with each wrong pair listed three times, the true record survived 0.2542 of draws instead of 0.4833 on a 4-record fixture and 0.2408 instead of 0.4767 on a 20-record one; on the latter `reid_evaluate()` then reported 0.3500 against 0.5500, with `lift` pinned at 1 in both cases because the random baseline moved by the same factor. That is exactly the Issue #60 signature.

Examples

raw <- data.frame(ROW_NUMBER = 1:6, V = c(1, 2, 3, 4, 5, 6))
s <- score_num(join_raw_anon_data(raw, raw), "V")
pruned <- top_k_candidates(s, k = 2)
nrow(pruned)
#> [1] 16
attr(pruned, "blocking")$recall
#> [1] 1