score the shape of an activity profile

Reads a collapsed column such as the `<col>_DIST` that [transform_transaction_to_master()] produces – a day-of-week column, an hour-of-day column, a category column – turns each record's values into a histogram over a fixed set of bins, and scores a candidate pair by the distance between the two histograms.

Usage

score_profile(
  dat_raw_anon,
  target,
  row_number = "ROW_NUMBER",
  split = ":",
  bins = NULL,
  shape_only = TRUE,
  metric = c("l1", "l2"),
  generalized = c("stop", "warn", "ignore"),
  .fn_name = "score_profile"
)

Arguments

dat_raw_anon: dataframe of raw_anon form
target: target column
row_number: name of the row-number column *before* the RAW_/ANON_ prefixing done by [join_raw_anon_data()] (default: "ROW_NUMBER")
split: literal separator between the elements of the collapsed column (default ":")
bins: character vector of bin labels, in order. `NULL` (the default) uses every value that occurs anywhere in either side, so a caller who knows the full support – all seven weekdays, all 24 hours – should pass it explicitly, otherwise a bin nobody happens to use simply does not exist.
shape_only: divide each histogram by its total, making the score independent of activity volume (default TRUE). A record with no value in any bin then has no shape at all and is an error rather than an all-zero profile, which would match every other empty record perfectly; pass `FALSE` to compare raw counts, where an empty record is a meaningful zero.
metric: `"l1"` (default; total variation when `shape_only = TRUE`) or `"l2"` (squared Euclidean)
generalized: what to do when `target` turns out to hold generalised values on the ANON side: `"stop"` (default), `"warn"` or `"ignore"`.
.fn_name: name used in error messages; a function that wraps this one passes its own name so the message points at the function the user actually called

Value

a "reid_scores" table (a distance: smaller is a better match)

Details

SCALE-FREE BY CONSTRUCTION. With the default `shape_only = TRUE` each histogram is divided by its own total, so a record whose every event is duplicated scores exactly as before. This is the point of splitting the activity profile into three functions: how *much* activity there is belongs to [score_count()], and if it leaked into this score as well, adding the two together in [score_multi()] would count the same evidence twice. It is also the property Issue #5 established that a distribution distance must have.

Unlike [score_dist()], nothing here is treated as a number: the bins are compared as labels, so an hour-of-day or day-of-week profile is scored correctly rather than being run through numeric quantiles that assume hour 23 and hour 0 are as far apart as possible.

Generalised columns are refused

The bins are labels, and a published region is a label like any other, so a generalised column bins cleanly into a histogram with one occupied bin per record and produces a complete, plausible score table – silently. Measured on a fully generalised age column it reported a fifth of the success rate [score_containment()] reports on the same data (Issue #100). Both this and [score_idf()] were missed by the Issue #40 fix because that fix was attached to a list of functions rather than to the act of reading a target column.

Examples