Thresholds

Classification thresholds for match probabilities.

Scores at a threshold and the choice of the threshold that maximises MCC or F1, on all record pairs of a split at once. The deep learning pipeline uses them to score its validation pairs after every epoch, to choose each stage’s best epoch and to tune the threshold of its final model.

neer_match_utilities.thresholds.non_match_weight(labels, match_share)[source]

The weight that gives the non-matches among labels the share of matches match_share: with it, the pairs count as if matches made up that share of them.

Paired record by record, a split of n records with one match each has a share of about 1/n, so that a small validation split holds a far larger share of matches than the training or test pairs. A threshold chosen on it then lets through more false positives than it seemed to.

Return type:: float

neer_match_utilities.thresholds.pair_labels(matches, n_left, n_right)[source]

Labels of all n_left * n_right record pairs, in the order DLMatchingModel.predict scores them: left record by left record.

matches holds the row positions of the matched pairs in its first two columns, left and right, as neer-match reads them.

Return type:: ndarray

neer_match_utilities.thresholds.scores_at(labels, probabilities, threshold, non_match_weight=1.0)[source]

TP, FP, TN, FN, Accuracy, Recall, Precision, F1 and MCC at one threshold. Precision, Recall, F1 and MCC are 0 where they are undefined. The rates count every non-match non_match_weight times; the counts are as they are.

Return type:: dict

neer_match_utilities.thresholds.tune_threshold(labels, probabilities, metric='mcc', thresholds=None, match_share=None)[source]

The threshold that maximises metric (‘mcc’ or ‘f1’), with the scores at it.

The thresholds tried are THRESHOLD_GRID, 0.01 to 0.99 in steps of 0.01, unless given. Of equally good thresholds the lowest wins, as in GradientBoostingModel’s tuning. With match_share, the non-matches are weighted as if matches made up that share of the pairs (see non_match_weight), both in choosing the threshold and in the rates returned; the counts are as they are.

Return type:: tuple[float, dict]