run an attack over several tie-break seeds and summarise the spread of the success rate

When a target column has ties, a single run reports one draw from a distribution of possible outcomes, not a fixed property of the data. A point estimate on its own is therefore not interpretable: on a 50-person fixture with a low-cardinality column the rate ranges over [0.02, 0.14] depending only on which tied candidate is picked. This runs the same attack across `seeds` and reports the mean together with the standard deviation, so the uncertainty is visible in the result rather than hidden.

Usage

reid_stability(reid_fn, dat_raw_anon, target, seeds = 1:20, ...)

Arguments

reid_fn

an attack function (or its name) taking `(dat_raw_anon, target, ..., seed)` and returning one row per ANON record with a logical `RESULT` column. A score/assignment pair is wrapped in one line; see the examples.

dat_raw_anon

dataframe of raw_anon form

target

target column

seeds

integer vector of tie-break seeds (default 1:20)

...

further arguments passed on to `reid_fn`

Value

an object of class "reid_stability": a list with `per_seed` (a data frame of seed / success / trial / rate), and the summary fields `mean`, `sd`, `min`, `max`, `trial` and `n_seeds`

Details

A near-zero `sd` means the target column is effectively collision-free and the point estimate can be read directly; a large `sd` means the single-run number should not be quoted without it.

[reid_evaluate()] reports the same spread (`success_mean` / `success_sd` / `success_min` / `success_max`) straight from a score table, alongside the baselines and the per-record risk, and is the usual way in. This function is the narrower tool: it takes a whole *attack* – any function of `(dat_raw_anon, target, ..., seed)` returning a data frame with a logical `RESULT` column – so a caller can measure the spread of an attack the score/assignment split does not cover.

See also

[reid_evaluate()], which reports the same spread from a score table together with the baselines it has to be read against.

Examples

# every value is shared by two records, so which one the attack "wins"
# is decided by the tie-break: a single run is one draw, not the answer
raw <- data.frame(ROW_NUMBER = 1:6, V = c(1, 1, 2, 2, 3, 3))
d <- join_raw_anon_data(raw, raw)

# an attack is a score function plus an assignment rule
attack_num <- function(dat, target, seed) {
  match_greedy(score_num(dat, target), seed = seed)
}
reid_stability(attack_num, d, "V", seeds = 1:5)
#> reid stability over 5 tie-break seeds (trial = 6)
#>   success rate: mean 0.5000  sd 0.2357  range [0.3333, 0.8333]

# a collision-free column gives sd 0: there the point estimate can be
# quoted on its own
u <- data.frame(ROW_NUMBER = 1:6, V = c(10, 20, 30, 40, 50, 60))
reid_stability(attack_num, join_raw_anon_data(u, u), "V", seeds = 1:5)
#> reid stability over 5 tie-break seeds (trial = 6)
#>   success rate: mean 1.0000  sd 0.0000  range [1.0000, 1.0000]