Skip to contents

A candidate pair scores 0 when the two records carry the same value, and the rarity weight of that value when they do not. Rare values therefore discriminate strongly and near-universal ones barely at all.

Usage

score_idf(
  dat_raw_anon,
  target,
  row_number = "ROW_NUMBER",
  source = c("anon", "raw", "pooled"),
  weight = c("idf", "inv_log", "inv", "none"),
  generalized = c("stop", "warn", "ignore"),
  .fn_name = "score_idf"
)

Arguments

dat_raw_anon

dataframe of raw_anon form

target

target column

row_number

name of the row-number column *before* the RAW_/ANON_ prefixing done by [join_raw_anon_data()] (default: "ROW_NUMBER")

source

which side to count: `"anon"` (default), `"raw"` or `"pooled"`. The default is `"anon"` because the released table is available to any attacker – estimating rarity from it needs no knowledge the adversary does not already have, which is what makes this weighting cheap to justify.

weight

rarity weighting scheme; see [idf_weight()]. `"none"` gives plain unweighted exact matching.

generalized

what to do when `target` turns out to hold generalised values on the ANON side: `"stop"` (default), `"warn"` or `"ignore"`.

.fn_name

name used in error messages; a function that wraps this one passes its own name so the message points at the function the user actually called

Value

a "reid_scores" table whose SCORE is 0 on agreement and the value's rarity weight on disagreement (a distance: smaller is a better match)

Generalised columns are refused

Exact matching on a generalised column agrees with **nothing**: the raw value `37` is never the string `"[30,40)"`, so every candidate pair scores the same rarity weight and the column carries no signal at all. It is the quietest form of the Issue #40 failure – no coercion, no error, a perfectly well-formed score table, and a reported success rate a fifth of the one [score_containment()] measures on the same release (Issue #100, and `docs/lessons-learned.md` section 2). The frequency table underneath is wrong in the same direction: it counts how often each *region* was published, not how rare the underlying value is, so a region covering a tenth of the population is scored as if it identified somebody.

See also

[score_containment()] for generalised columns.

Examples

d <- create_dummy_qi_data(people = 30, seed = 1)
j <- join_raw_anon_data(d, d)
match_greedy(score_idf(j, "ZIP"))
#>    ANON_ROW_NUMBER RAW_ROW_NUMBER CONFIDENCE RESULT
#> 1                1             20          0  FALSE
#> 2                2              1          0  FALSE
#> 3                3              4          0  FALSE
#> 4                4             21          0  FALSE
#> 5                5              9          0  FALSE
#> 6                6             24          0  FALSE
#> 7                7             30          0  FALSE
#> 8                8             20          0  FALSE
#> 9                9              5          0  FALSE
#> 10              10              4          0  FALSE
#> 11              11             30          0  FALSE
#> 12              12              8          0  FALSE
#> 13              13             26          0  FALSE
#> 14              14             20          0  FALSE
#> 15              15             23          0  FALSE
#> 16              16             26          0  FALSE
#> 17              17             28          0  FALSE
#> 18              18             29          0  FALSE
#> 19              19             13          0  FALSE
#> 20              20             20          0   TRUE
#> 21              21             21          0   TRUE
#> 22              22             19          0  FALSE
#> 23              23             23          0   TRUE
#> 24              24             15          0  FALSE
#> 25              25             17          0  FALSE
#> 26              26             13          0  FALSE
#> 27              27             21          0  FALSE
#> 28              28             29          0  FALSE
#> 29              29             25          0  FALSE
#> 30              30              7          0  FALSE