Measures each score table on its own and tests whether it ranks the true RAW record above chance. An axis that does not is worse than useless in a combination: summed in at equal weight its scores act as noise, reorder candidates the informative axes had right, and pull the reported reidentification rate *down*. A safety assessment that moves down for that reason is an under-estimate, so the condition is worth detecting explicitly.
Value
a data frame of class "reid_axis_report", one row per axis, with
- axis
name of the score table
- n_anon
number of ANON records measured
- success
exact expected single-guess success rate of this axis alone
- baseline
success rate of random assignment on the same candidate sets
- lift
`success / baseline`
- mean_rank_pct
average position of the true record in the candidate ranking, as a fraction: 0.5 is chance, smaller is better for the attacker
- z
rank statistic in units of its null standard deviation
- p_value
one-sided p-value against the null
- informative
`p_value < alpha`
What the null is
Under "this axis carries no information about identity" the true RAW record is exchangeable with the other candidates of its ANON record, so its mid-rank is uniform over that record's rank multiset: mean `(n + 1) / 2`, and a variance that follows exactly from the multiset (and so accounts for ties without an approximation). The statistic is the total, over ANON records, of how far above the middle the true record was ranked, and its normal approximation is sound because it is a sum of many independent bounded terms. Nothing is sampled and there is no seed.
Why ranks and not the success rate
Comparing an axis's own single-guess success rate against the random-assignment baseline is the obvious test and is far too blunt: the baseline is `1 / n`, so the null expects about one hit in the whole table and no realistic number of records gives it any power. Measured on this package's fixtures it called five axes with real signal uninformative. The rank test separates the same fixtures with no overlap at all – dead axes at `z` between 0.4 and 0.8, live ones from 5.6 up. `success`, `baseline` and `lift` are still reported, because they are what a risk report is written in; they are not what `informative` is decided on.
This needs the answer key
A record's rank is only defined relative to the RAW record it actually came from, which the score layer reads off matching row numbers. Screening therefore describes an attacker who already knows which of their attributes work. Where no ANON record has a matching RAW record the question cannot be asked at all, and every column of the result is `NA`.
See also
[score_multi()], whose `screen` argument runs this over the declared columns, and [axis_report()], which reads the result back off a combined score.
Examples
d <- create_dummy_qi_data(people = 40, seed = 1)
j <- join_raw_anon_data(d, d)
axis_informativeness(list(AGE = score_num(j, "AGE"), SEX = score_char(j, "SEX")))
#> axis informativeness (2 axis/axes, alpha = 0.05)
#> AGE success 0.8000 baseline 0.0250 lift 32.00x rank 0.033 z 10.53 p = 0.0000 informative
#> SEX success 0.0500 baseline 0.0250 lift 2.00x rank 0.265 z 6.29 p = 0.0000 informative