Package index • reidentify

データ準備

RAW と ANON を突き合わせ可能な形にそろえる。join_raw_anon_data() が作る RAW_ / ANON_ 接頭辞つきの候補ペア表が、以降すべての入力になる。

join_raw_anon_data(): create reid-format data from raw and anon data frame
transform_transaction_to_master(): transform transaction -> master
create_dummy_qi_data(): create dummy master data with an explicit quasi-identifier structure
create_dummy_master_data(): create dummy master data
create_dummy_transaction_data(): create dummy transaction data

3 層 API の第 1 層。属性 1 つ (あるいは宣言した複数属性) を手がかりに、 RAW × ANON の各候補ペアへスコアを付ける。ここでは割当を決めない。

score_num(): score a numeric column by absolute difference
score_char(): score a character column by Levenshtein edit distance
score_num_rank(): rank a numeric column within RAW and within ANON, and score by rank gap
score_dist(): score a distribution column ("A:B:C") by quantile-vector distance
score_count(): score how much activity two records show
score_span(): score how long a record's activity stretches over
score_profile(): score the shape of an activity profile

score_idf(): score one column by rarity-weighted exact matching
score_idf_match(): score several columns at once by rarity-weighted exact matching
value_frequencies(): count how often each value of a column occurs
score_jaccard(): score a set-valued column ("A:B:C") by set overlap
score_minhash(): score a set-valued column by min-hash estimated Jaccard distance
score_mahalanobis(): score several numeric columns jointly by Mahalanobis distance
score_multi(): attack several attributes at once
score_scoreboard(): score candidate pairs the way Narayanan & Shmatikov's Scoreboard does

大規模データで RAW × ANON の全ペアを持たずに済ませる。いずれも削減率と再現率 (recall) を実測して blocking 属性に記録し、recall が 1 を下回れば警告する。

block_candidates(): build a reduced RAW/ANON candidate table by deterministic blocking
lsh_candidates(): build a reduced RAW/ANON candidate table by min-hash LSH blocking
top_k_candidates(): keep only the k best-scoring RAW candidates per ANON record
blocking_recall(): measure the reduction and the recall of any candidate set

3 層 API の第 2 層。尺度の違うスコア表を正規化してから加重和にする。軸の診断は「その属性がランダム割当より良いことを言えているか」を検定する。

normalize_scores(): rescale score tables so several attributes can be summed
combine_scores(): combine several score tables into one weighted score
axis_informativeness(): does an axis carry any information about identity?
axis_report(): read the per-axis report off a combined score

3 層 API の第 3 層。スコア表を「どの ANON をどの RAW に対応づけるか」に変える。同点の決着は乱数に依存するので、シードを指定する。

match_greedy(): assign each ANON record to its best-scoring RAW record, independently
match_optimal(): assign ANON records to RAW records under a global one-to-one constraint
match_scoreboard_rh(): the RH decision rule: answer only when the winner is clearly ahead

成功率を単独で出さない。ランダム割当のベースライン、シード違いのばらつき、レコードごとのリスクを併記して、測定が壊れていたときに気づけるようにする。

reid_evaluate(): evaluate a reidentification attack: baselines, precision-recall, top-k and per-record risk
reid_confidence(): per-ANON-record confidence in the best-scoring candidate
reid_stability(): run an attack over several tie-break seeds and summarise the spread of the success rate

攻撃を実行せずに「そもそも何人が一意に定まるか」を測る。上限としてのリスク。

unicity(): measure unicity as a function of the number of known attributes
unicity_fraction(): fraction of records that are unique on a given set of columns
spatiotemporal_unicity(): how many people are pinned down by k points of their trace
coarsen_place(): merge neighbouring location codes into a coarser grid
coarsen_time(): put timestamps onto a coarser grid

攻撃者が何を見られる想定かを明示する。W / M / S の 3 水準を横並びで比較でき、想定を書かない成功率の数字を残さないためのしくみ。

attacker_knowledge(): declare what a modelled attacker knows
dummy_qi_knowledge(): the attacker knowledge specification that matches create_dummy_qi_data()
score_by_knowledge(): score a RAW/ANON table using only the columns an attacker can see
reid_knowledge_curve(): compare reidentification risk across the W / M / S knowledge levels

「30代」「東京都」のように区画へ丸めた公開値を、階層に沿って元の値と突き合わせる。含意関係に基づく絞り込みと、k 匿名性の実測。

generalization_hierarchy(): build a generalisation hierarchy from a table of edges
read_generalization_hierarchy(): read a generalisation hierarchy from a CSV or YAML file
generalization_units(): units that may follow a number in a generalised value
generalize_value(): generalise raw values with a declared hierarchy
is_generalized_value(): does a value name a region rather than a single value?
containment_counts(): how far containment narrows the candidate set, per ANON record
score_containment(): score generalised (interval / categorical) attributes by containment

このパッケージが返すオブジェクトの表示。通常は明示的に呼ばない。