score candidate pairs the way Narayanan & Shmatikov's Scoreboard does

The similarity between an attacker's fragmentary knowledge of a person and a released record: for every attribute the attacker knows, add the rarity weight of that attribute if the two records agree there, allowing near matches within `tolerance`.

Usage

score_scoreboard(
  dat_raw_anon,
  targets,
  row_number = "ROW_NUMBER",
  tolerance = 0,
  partial = c("step", "linear"),
  aux_side = c("raw", "anon"),
  source = c("anon", "raw", "pooled"),
  weight = c("inv_log", "idf", "inv", "none"),
  generalized = c("stop", "warn", "ignore"),
  .fn_name = "score_scoreboard"
)

Arguments

dat_raw_anon: dataframe of raw_anon form
targets: character vector of attribute columns, *before* the RAW_/ANON_ prefixing done by [join_raw_anon_data()]
row_number: name of the row-number column *before* the RAW_/ANON_ prefixing done by [join_raw_anon_data()] (default: "ROW_NUMBER")
tolerance: how close two values must be to count as agreeing. Either one number for all of `targets`, or a vector named by target. Numeric columns only; ignored for the rest, where agreement is exact. Default 0, i.e. exact matching.
partial: `"step"` (default: anything within `tolerance` is a full match) or `"linear"` (credit decaying to 0 at `tolerance`).
aux_side: which side carries the attacker's knowledge and therefore defines which attributes are scored: `"raw"` (default – RAW is the attacker's background knowledge in this package, matching the paper's `supp(aux)`) or `"anon"`.
source: which side the support counts are taken over (default `"anon"`, the released table, which any attacker can count for themselves).
weight: rarity weighting scheme, passed to [idf_weight()]. `"none"` gives unweighted overlap counting, the baseline the weighting has to beat.
generalized: what to do when one of `targets` turns out to hold generalised values on the ANON side: `"stop"` (default), `"warn"` or `"ignore"`. A raw value never equals the region that contains it, so every candidate pair scores 0 similarity and the attribute contributes nothing – silently (Issue #100). See [score_containment()].
.fn_name: name used in error messages; a function that wraps this one passes its own name so the message points at the function the user actually called

Value

a "reid_scores" table with `score_type` `"similarity"`: larger means a better match.

Details

This is the score half of Scoreboard-RH. The decision half – answer only when the winner is far enough clear of the runner-up – is [match_scoreboard_rh()].

Sparsity is the point

`NA` means "this record has no value for this attribute", and that is a first-class state here rather than an error. It is what makes the method work on transaction-shaped data, where any one person touches a tiny fraction of the possible items, and it is how partial attacker knowledge is expressed: set every attribute the attacker does *not* know to `NA` on the `aux_side`, and only the rest are scored.

Other `score_*()` functions in this package reject `NA` because there a missing value is a data problem that could otherwise be scored as a confident match. Here a missing value contributes exactly 0 and can never manufacture agreement, so admitting it is safe.

Weighting

Each attribute is weighted by the inverse of how many records have it at all – its support – so that an item almost nobody has counts for much more than one everybody has. The paper writes this weight as `1 / log(|supp(i)|)`, which is infinite for an attribute exactly one record has; the default `"inv_log"` uses the package's `1 / log(support + 1)` instead, for the reason already documented on [idf_weight()] – the singleton is the case the method exists for, so the formula must not diverge there.

Note that this weights an *attribute* by its support, whereas [score_idf()] weights a *value* by its frequency. Both are rarity weightings; they are answers to different questions, and on wide sparse data it is the attribute one that carries the signal.

References

Narayanan, A. and Shmatikov, V. (2008) Robust De-anonymization of Large Sparse Datasets. IEEE Symposium on Security and Privacy, 111-125.

Examples

## three people, four sparse items; the attacker knows two items each
anon <- data.frame(
  ROW_NUMBER = 1:3,
  I1 = c(5, NA, 1), I2 = c(NA, 2, 2), I3 = c(3, 4, NA), I4 = c(NA, 1, 5)
)
aux <- anon
aux$I3 <- NA
d <- join_raw_anon_data(aux, anon)
s <- score_scoreboard(d, c("I1", "I2", "I3", "I4"), tolerance = 1)
match_scoreboard_rh(s)
#>   ANON_ROW_NUMBER RAW_ROW_NUMBER CONFIDENCE RESULT
#> 1               1              1   1.732051   TRUE
#> 2               2              2   1.000000   TRUE
#> 3               3              3   1.309307   TRUE