Baseline Training

class neer_match_utilities.baseline_training.BaselineTrainingPipe(model_name, similarity_map, training_data, testing_data, validation_data=None, id_left_col='id', id_right_col='id', matches_id_left='left', matches_id_right='right', matches_are_indices=True, model_kind='gb', mismatch_share_fit=1.0, random_state=42, shuffle_fit=True, threshold=0.5, tune_threshold=True, tune_metric='mcc', base_dir=None, export_model=True, export_stats=True, reload_sanity_check=True, overwrite=False, pairs_per_chunk=250000)[source]

Orchestrates training + evaluation + export for baseline (non-DL) models: - LogitMatchingModel (statsmodels) - ProbitMatchingModel (statsmodels) - GradientBoostingModel (sklearn)

Pipeline steps

  1. Set up the pairwise similarity features for train/val/test

  2. Subsample non-matches for fitting (optional), choosing the pairs before computing their features

  3. Fit selected baseline model

  4. Choose threshold (optional; recommended for GB)

  5. Save model via ModelBaseline.save(…)

  6. Evaluate on full train + test, together with the reloaded model for the sanity check

  7. Export performance.csv + similarity_map.dill via Training.performance_statistics_export(…)

Similarity features are computed in blocks of at most pairs_per_chunk pairs and discarded once scored, so memory does not grow with the number of pairs. Only the labels and predictions of all pairs are kept, a few bytes each. The fitted model does not depend on pairs_per_chunk. Logit and probit probabilities come from matrix products, whose rounding can differ in the last bits between block sizes, so a pair scored within about 1e-15 of the threshold could be classified differently.

__init__(model_name, similarity_map, training_data, testing_data, validation_data=None, id_left_col='id', id_right_col='id', matches_id_left='left', matches_id_right='right', matches_are_indices=True, model_kind='gb', mismatch_share_fit=1.0, random_state=42, shuffle_fit=True, threshold=0.5, tune_threshold=True, tune_metric='mcc', base_dir=None, export_model=True, export_stats=True, reload_sanity_check=True, overwrite=False, pairs_per_chunk=250000)