# Model Ledger snapshot — 2026-07-14

_Dated full ledger snapshot, including live metrics and offline experiments, generated by `python -m evaluation.ledger`. This report may lag the public dashboard. For current live results, use the Performance page's manifest-pinned accepted sync; both surfaces use `production.evaluation.ledger` math. Probabilities are P(player1 wins); ground truth is joined from `prediction_log.csv` by `match_uid`._

## Cohort tiers

- **GOLD** = settled & `snapshot_v2` & `exact_feature_snapshot` & features_complete & feature snapshot ID verified against persisted lineage (decision-grade; headline).
- **COMPLETE** = settled & features_complete (~65% legacy_backfilled; context).
- **\*_intersection** = restricted to match_uids where all of nn/xgb/rf/market predicted (apples-to-apples).
- **\*_all_model_intersection** = restricted to match_uids shared by nn/xgb/rf/market and every scored variant in the current `DEFAULT_SHADOW_MODEL_SPECS`; retired historical variants do not shrink the active comparison.
- **\*_market_timing** = the same settled match_uids at first observed and last valid pre-start prices; requires at least two distinct pre-start captures and never treats first observed as the sportsbook's true opener.
- **settled_market_timing** = every explicit valid winner with comparable market captures, independent of feature completeness; use this for the broad standalone market-open versus market-close diagnostic.
- **Shadow variants** = one deterministic opening observation per `(match_uid, model_version)`, joined to the operational opening feature snapshot; hourly repeats do not increase n.

## Verdict (live, settled)

On the **gold_intersection** cohort (n=111, all of nn/xgb/rf/market predicted), ranked by log loss:

- **market** — log_loss 0.5917, brier 0.2037, acc 0.6757, AUC 0.7456, ROI(flat) 0.0% over 0 bets, ROI(Kelly) 0.0%
- **xgb** — log_loss 0.5982, brier 0.2061, acc 0.6396, AUC 0.7416, ROI(flat) -7.7% over 73 bets, ROI(Kelly) 4.8%
- **nn** — log_loss 0.5994, brier 0.2066, acc 0.6847, AUC 0.7461, ROI(flat) -1.8% over 75 bets, ROI(Kelly) 4.5%
- **rf** — log_loss 0.6149, brier 0.2126, acc 0.6937, AUC 0.7364, ROI(flat) -9.1% over 79 bets, ROI(Kelly) -8.6%

**Best discriminator: `market`** (log_loss 0.5917) vs current betting model `nn` (log_loss 0.5994). Bets are currently staked entirely off `nn`.
No model beats the market on log loss on this cohort.
**ROI (GOLD, profitable models):** `shadow_performance_v1_xgb_depth4_slow_regularized__2026-04-25` 1.6% flat / 7.6% Kelly (68 bets, n=100). Beating the vig (>0%) is the bar; treat n<250 as a lead, not proof.

## Live model table

### gold_intersection (n=111)

| model | n | accuracy | auc | log_loss | brier | ece | cal_slope | roi_flat | roi_kelly | n_bets_flat |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| market | 111 | 0.6757 | 0.7456 | 0.5917 | 0.2037 | 0.0846 | 1.2395 | 0.0000 | 0.0000 | 0 |
| xgb | 111 | 0.6396 | 0.7416 | 0.5982 | 0.2061 | 0.0998 | 1.4306 | -0.0774 | 0.0483 | 73 |
| nn | 111 | 0.6847 | 0.7461 | 0.5994 | 0.2066 | 0.0621 | 1.4867 | -0.0180 | 0.0452 | 75 |
| rf | 111 | 0.6937 | 0.7364 | 0.6149 | 0.2126 | 0.0999 | 1.7677 | -0.0907 | -0.0863 | 79 |

### gold_all_model_intersection (n=100)

| model | n | accuracy | auc | log_loss | brier | ece | cal_slope | roi_flat | roi_kelly | n_bets_flat |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| market | 100 | 0.6700 | 0.7384 | 0.5976 | 0.2065 | 0.0790 | 1.1622 | 0.0000 | 0.0000 | 0 |
| nn | 100 | 0.6900 | 0.7528 | 0.5976 | 0.2054 | 0.0796 | 1.5550 | 0.0325 | 0.1092 | 68 |
| xgb | 100 | 0.6500 | 0.7408 | 0.6010 | 0.2071 | 0.0940 | 1.3931 | -0.0434 | 0.0838 | 68 |
| shadow_performance_v1_xgb_depth5_balanced_regularized__2026-04-25 | 100 | 0.7000 | 0.7616 | 0.6023 | 0.2069 | 0.0989 | 1.8672 | -0.0168 | 0.0915 | 67 |
| shadow_performance_v1_xgb_depth5_recency_hl_8y__2026-04-25 | 100 | 0.6900 | 0.7524 | 0.6037 | 0.2074 | 0.1038 | 1.8175 | -0.0154 | 0.0782 | 66 |
| shadow_performance_v1_xgb_depth4_slow_regularized__2026-04-25 | 100 | 0.7200 | 0.7656 | 0.6046 | 0.2075 | 0.1272 | 2.1206 | 0.0158 | 0.0762 | 68 |
| shadow_performance_v1_xgb_depth5_recency_hl_12y__2026-04-25 | 100 | 0.6700 | 0.7576 | 0.6046 | 0.2079 | 0.1059 | 1.8275 | -0.0312 | 0.1022 | 68 |
| shadow_performance_v1_cat_depth6_screening_one_hot__2026-04-25 | 100 | 0.6700 | 0.7468 | 0.6099 | 0.2103 | 0.0832 | 1.8506 | -0.0398 | 0.0388 | 71 |
| shadow_performance_v1_lgbm_leaves31_regularized_one_hot__2026-04-25 | 100 | 0.6700 | 0.7408 | 0.6116 | 0.2113 | 0.0966 | 1.7522 | -0.0588 | 0.0531 | 71 |
| rf | 100 | 0.7100 | 0.7488 | 0.6118 | 0.2110 | 0.1216 | 1.8303 | -0.0207 | -0.0186 | 71 |
| shadow_performance_v1_xgb_depth6_medium__2026-04-25 | 100 | 0.6700 | 0.7464 | 0.6141 | 0.2122 | 0.1144 | 1.7284 | -0.0517 | 0.0527 | 69 |
| shadow_performance_v1_nn_logits_128_64_robust__2026-04-25 | 100 | 0.6000 | 0.6976 | 0.6310 | 0.2208 | 0.1020 | 1.1079 | -0.1862 | -0.1070 | 69 |
| shadow_performance_v1_nn_logits_128_64_32_lowdrop__2026-04-25 | 100 | 0.6600 | 0.7156 | 0.6323 | 0.2212 | 0.0989 | 1.8406 | -0.1645 | -0.0374 | 71 |
| shadow_performance_v1_nn_logits_96_48_lowdrop__2026-04-25 | 100 | 0.6600 | 0.7148 | 0.6351 | 0.2228 | 0.0904 | 1.7478 | -0.1763 | -0.0833 | 77 |

### gold_market_timing (n=80)

| model | n | accuracy | auc | log_loss | brier | ece | cal_slope | roi_flat | roi_kelly | n_bets_flat |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| market_close | 80 | 0.7000 | 0.7558 | 0.5841 | 0.2004 | 0.1244 | 1.2552 | 0.0000 | 0.0000 | 0 |
| market_open | 80 | 0.7125 | 0.7558 | 0.5848 | 0.2005 | 0.1410 | 1.2514 | 0.0000 | 0.0000 | 0 |

### settled_market_timing (n=136)

| model | n | accuracy | auc | log_loss | brier | ece | cal_slope | roi_flat | roi_kelly | n_bets_flat |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| market_open | 136 | 0.7206 | 0.7598 | 0.5819 | 0.1989 | 0.0776 | 1.1491 | 0.0000 | 0.0000 | 0 |
| market_close | 136 | 0.7059 | 0.7594 | 0.5825 | 0.1993 | 0.0631 | 1.1434 | 0.0000 | 0.0000 | 0 |

### gold (n=111)

| model | n | accuracy | auc | log_loss | brier | ece | cal_slope | roi_flat | roi_kelly | n_bets_flat |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| market | 111 | 0.6757 | 0.7456 | 0.5917 | 0.2037 | 0.0846 | 1.2395 | 0.0000 | 0.0000 | 0 |
| xgb | 111 | 0.6396 | 0.7416 | 0.5982 | 0.2061 | 0.0998 | 1.4306 | -0.0774 | 0.0483 | 73 |
| nn | 111 | 0.6847 | 0.7461 | 0.5994 | 0.2066 | 0.0621 | 1.4867 | -0.0180 | 0.0452 | 75 |
| shadow_performance_v1_xgb_depth5_balanced_regularized__2026-04-25 | 100 | 0.7000 | 0.7616 | 0.6023 | 0.2069 | 0.0989 | 1.8672 | -0.0168 | 0.0915 | 67 |
| shadow_performance_v1_xgb_depth5_recency_hl_8y__2026-04-25 | 100 | 0.6900 | 0.7524 | 0.6037 | 0.2074 | 0.1038 | 1.8175 | -0.0154 | 0.0782 | 66 |
| shadow_performance_v1_xgb_depth4_slow_regularized__2026-04-25 | 100 | 0.7200 | 0.7656 | 0.6046 | 0.2075 | 0.1272 | 2.1206 | 0.0158 | 0.0762 | 68 |
| shadow_performance_v1_xgb_depth5_recency_hl_12y__2026-04-25 | 100 | 0.6700 | 0.7576 | 0.6046 | 0.2079 | 0.1059 | 1.8275 | -0.0312 | 0.1022 | 68 |
| shadow_performance_v1_cat_depth6_screening_one_hot__2026-04-25 | 100 | 0.6700 | 0.7468 | 0.6099 | 0.2103 | 0.0832 | 1.8506 | -0.0398 | 0.0388 | 71 |
| shadow_performance_v1_lgbm_leaves31_regularized_one_hot__2026-04-25 | 100 | 0.6700 | 0.7408 | 0.6116 | 0.2113 | 0.0966 | 1.7522 | -0.0588 | 0.0531 | 71 |
| shadow_performance_v1_xgb_depth6_medium__2026-04-25 | 100 | 0.6700 | 0.7464 | 0.6141 | 0.2122 | 0.1144 | 1.7284 | -0.0517 | 0.0527 | 69 |
| rf | 111 | 0.6937 | 0.7364 | 0.6149 | 0.2126 | 0.0999 | 1.7677 | -0.0907 | -0.0863 | 79 |
| shadow_performance_v1_nn_logits_128_64_robust__2026-04-25 | 100 | 0.6000 | 0.6976 | 0.6310 | 0.2208 | 0.1020 | 1.1079 | -0.1862 | -0.1070 | 69 |
| shadow_performance_v1_nn_logits_128_64_32_lowdrop__2026-04-25 | 100 | 0.6600 | 0.7156 | 0.6323 | 0.2212 | 0.0989 | 1.8406 | -0.1645 | -0.0374 | 71 |
| shadow_performance_v1_nn_logits_96_48_lowdrop__2026-04-25 | 100 | 0.6600 | 0.7148 | 0.6351 | 0.2228 | 0.0904 | 1.7478 | -0.1763 | -0.0833 | 77 |

### complete_intersection (n=1066)

| model | n | accuracy | auc | log_loss | brier | ece | cal_slope | roi_flat | roi_kelly | n_bets_flat |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| market | 1066 | 0.6651 | 0.7340 | 0.6074 | 0.2097 | 0.0306 | 1.1475 | 0.0000 | 0.0000 | 0 |
| xgb | 1066 | 0.6492 | 0.7022 | 0.6305 | 0.2197 | 0.0216 | 1.0207 | -0.0904 | -0.0560 | 656 |
| rf | 1066 | 0.6398 | 0.6950 | 0.6369 | 0.2228 | 0.0411 | 1.2217 | -0.1122 | -0.1103 | 656 |
| nn | 1066 | 0.6144 | 0.6482 | 0.7245 | 0.2416 | 0.0491 | 0.3060 | -0.1332 | -0.0845 | 694 |

### complete_all_model_intersection (n=119)

| model | n | accuracy | auc | log_loss | brier | ece | cal_slope | roi_flat | roi_kelly | n_bets_flat |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| market | 119 | 0.6807 | 0.7328 | 0.6010 | 0.2080 | 0.0753 | 1.0167 | 0.0000 | 0.0000 | 0 |
| shadow_performance_v1_xgb_depth5_recency_hl_8y__2026-04-25 | 119 | 0.6891 | 0.7383 | 0.6031 | 0.2080 | 0.0676 | 1.5130 | 0.0632 | 0.1283 | 78 |
| shadow_performance_v1_xgb_depth5_recency_hl_12y__2026-04-25 | 119 | 0.6723 | 0.7434 | 0.6045 | 0.2087 | 0.0756 | 1.4685 | 0.0277 | 0.1397 | 80 |
| shadow_performance_v1_xgb_depth5_balanced_regularized__2026-04-25 | 119 | 0.6975 | 0.7411 | 0.6082 | 0.2101 | 0.0786 | 1.4577 | 0.0187 | 0.0991 | 79 |
| shadow_performance_v1_xgb_depth4_slow_regularized__2026-04-25 | 119 | 0.7143 | 0.7419 | 0.6106 | 0.2108 | 0.1035 | 1.5714 | 0.0291 | 0.0838 | 83 |
| nn | 119 | 0.6807 | 0.7270 | 0.6132 | 0.2125 | 0.0545 | 1.2820 | 0.0282 | 0.0572 | 82 |
| shadow_performance_v1_lgbm_leaves31_regularized_one_hot__2026-04-25 | 119 | 0.6723 | 0.7278 | 0.6133 | 0.2126 | 0.0615 | 1.4270 | -0.0070 | 0.0950 | 82 |
| xgb | 119 | 0.6387 | 0.7216 | 0.6135 | 0.2129 | 0.0719 | 1.1834 | -0.0242 | 0.0650 | 84 |
| shadow_performance_v1_cat_depth6_screening_one_hot__2026-04-25 | 119 | 0.6723 | 0.7321 | 0.6139 | 0.2124 | 0.0577 | 1.4267 | -0.0098 | 0.0521 | 85 |
| shadow_performance_v1_xgb_depth6_medium__2026-04-25 | 119 | 0.6723 | 0.7298 | 0.6159 | 0.2136 | 0.0906 | 1.3861 | 0.0004 | 0.0939 | 80 |
| rf | 119 | 0.6975 | 0.7318 | 0.6193 | 0.2144 | 0.1117 | 1.5529 | 0.0497 | -0.0269 | 85 |
| shadow_performance_v1_nn_logits_128_64_32_lowdrop__2026-04-25 | 119 | 0.6639 | 0.7044 | 0.6305 | 0.2212 | 0.0784 | 1.3982 | -0.1285 | -0.0059 | 82 |
| shadow_performance_v1_nn_logits_96_48_lowdrop__2026-04-25 | 119 | 0.6555 | 0.7035 | 0.6323 | 0.2221 | 0.0720 | 1.3227 | -0.0937 | -0.0477 | 89 |
| shadow_performance_v1_nn_logits_128_64_robust__2026-04-25 | 119 | 0.6134 | 0.6865 | 0.6382 | 0.2242 | 0.1084 | 0.9024 | -0.1354 | -0.1099 | 81 |

### complete_market_timing (n=88)

| model | n | accuracy | auc | log_loss | brier | ece | cal_slope | roi_flat | roi_kelly | n_bets_flat |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| market_open | 88 | 0.7159 | 0.7581 | 0.5818 | 0.1993 | 0.1072 | 1.2673 | 0.0000 | 0.0000 | 0 |
| market_close | 88 | 0.7045 | 0.7563 | 0.5825 | 0.1999 | 0.1108 | 1.2562 | 0.0000 | 0.0000 | 0 |

### complete (n=1247)

| model | n | accuracy | auc | log_loss | brier | ece | cal_slope | roi_flat | roi_kelly | n_bets_flat |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| shadow_performance_v1_xgb_depth5_recency_hl_8y__2026-04-25 | 119 | 0.6891 | 0.7383 | 0.6031 | 0.2080 | 0.0676 | 1.5130 | 0.0632 | 0.1283 | 78 |
| shadow_performance_v1_xgb_depth5_recency_hl_12y__2026-04-25 | 119 | 0.6723 | 0.7434 | 0.6045 | 0.2087 | 0.0756 | 1.4685 | 0.0277 | 0.1397 | 80 |
| market | 1245 | 0.6635 | 0.7354 | 0.6066 | 0.2094 | 0.0341 | 1.1496 | -0.2167 | 0.2221 | 3 |
| shadow_performance_v1_xgb_depth5_balanced_regularized__2026-04-25 | 119 | 0.6975 | 0.7411 | 0.6082 | 0.2101 | 0.0786 | 1.4577 | 0.0187 | 0.0991 | 79 |
| shadow_performance_v1_xgb_depth4_slow_regularized__2026-04-25 | 119 | 0.7143 | 0.7419 | 0.6106 | 0.2108 | 0.1035 | 1.5714 | 0.0291 | 0.0838 | 83 |
| shadow_performance_v1_lgbm_leaves31_regularized_one_hot__2026-04-25 | 119 | 0.6723 | 0.7278 | 0.6133 | 0.2126 | 0.0615 | 1.4270 | -0.0070 | 0.0950 | 82 |
| shadow_performance_v1_cat_depth6_screening_one_hot__2026-04-25 | 119 | 0.6723 | 0.7321 | 0.6139 | 0.2124 | 0.0577 | 1.4267 | -0.0098 | 0.0521 | 85 |
| shadow_performance_v1_xgb_depth6_medium__2026-04-25 | 119 | 0.6723 | 0.7298 | 0.6159 | 0.2136 | 0.0906 | 1.3861 | 0.0004 | 0.0939 | 80 |
| shadow_performance_v1_nn_logits_128_64_32_lowdrop__2026-04-25 | 119 | 0.6639 | 0.7044 | 0.6305 | 0.2212 | 0.0784 | 1.3982 | -0.1285 | -0.0059 | 82 |
| xgb | 1066 | 0.6492 | 0.7022 | 0.6305 | 0.2197 | 0.0216 | 1.0207 | -0.0904 | -0.0560 | 656 |
| shadow_performance_v1_nn_logits_96_48_lowdrop__2026-04-25 | 119 | 0.6555 | 0.7035 | 0.6323 | 0.2221 | 0.0720 | 1.3227 | -0.0937 | -0.0477 | 89 |
| rf | 1066 | 0.6398 | 0.6950 | 0.6369 | 0.2228 | 0.0411 | 1.2217 | -0.1122 | -0.1103 | 656 |
| shadow_performance_v1_nn_logits_128_64_robust__2026-04-25 | 119 | 0.6134 | 0.6865 | 0.6382 | 0.2242 | 0.1084 | 0.9024 | -0.1354 | -0.1099 | 81 |
| nn | 1247 | 0.6151 | 0.6504 | 0.7140 | 0.2398 | 0.0436 | 0.3472 | -0.1188 | -0.0903 | 790 |

## Offline experiments (backtest, not live)

_Different `split` labels use different test eras and are NOT comparable across splits. Compare within a split only._

| family | experiment | split | accuracy | auc | log_loss | brier | ece |
| --- | --- | --- | --- | --- | --- | --- | --- |
| xgboost | xgb_depth5_recency_hl_12y | fixed_2022_val_2023plus_test | 0.6712 | 0.7374 | 0.6008 | 0.2073 | 0.0073 |
| xgboost | xgb_depth5_recency_hl_8y | fixed_2022_val_2023plus_test | 0.6713 | 0.7371 | 0.6010 | 0.2074 | 0.0063 |
| xgboost | xgb_depth6_medium | fixed_2022_val_2023plus_test | 0.6701 | 0.7365 | 0.6015 | 0.2076 | 0.0072 |
| lightgbm | lgbm_leaves31_regularized | fixed_2022_val_2023plus_test | 0.6708 | 0.7362 | 0.6017 | 0.2077 | 0.0067 |
| xgboost | xgb_depth5_balanced_regularized | fixed_2022_val_2023plus_test | 0.6705 | 0.7361 | 0.6017 | 0.2077 | 0.0062 |
| lightgbm | lgbm_leaves31_regularized | fixed_2022_val_2023plus_test | 0.6705 | 0.7359 | 0.6018 | 0.2078 | 0.0065 |
| catboost | cat_depth6_screening | fixed_2022_val_2023plus_test | 0.6704 | 0.7357 | 0.6020 | 0.2079 | 0.0047 |
| catboost | cat_depth6_screening | fixed_2022_val_2023plus_test | 0.6702 | 0.7353 | 0.6023 | 0.2080 | 0.0042 |
| xgboost | xgb_depth4_slow_regularized | fixed_2022_val_2023plus_test | 0.6691 | 0.7351 | 0.6024 | 0.2080 | 0.0047 |
| nn | nn_logits_128_64_32_lowdrop | fixed_2022_val_2023plus_test | 0.6674 | 0.7330 | 0.6039 | 0.2088 | 0.0092 |
| nn | nn_logits_96_48_lowdrop | fixed_2022_val_2023plus_test | 0.6677 | 0.7330 | 0.6047 | 0.2089 | 0.0119 |
| xgboost | xgb_depth5_recency_hl_8y | fixed_2022_val_2023plus_test | 0.6672 | 0.7318 | 0.6051 | 0.2092 | 0.0095 |
| catboost | cat_depth6_regularized | fixed_2022_val_2023plus_test | 0.6667 | 0.7314 | 0.6051 | 0.2093 | 0.0069 |
| xgboost | xgb_depth5_recency_hl_12y | fixed_2022_val_2023plus_test | 0.6663 | 0.7315 | 0.6053 | 0.2093 | 0.0082 |
| lightgbm | lgbm_leaves31_regularized | fixed_2022_val_2023plus_test | 0.6669 | 0.7310 | 0.6056 | 0.2094 | 0.0100 |
| xgboost | xgb_depth5_recency_hl_5y | fixed_2022_val_2023plus_test | 0.6671 | 0.7314 | 0.6057 | 0.2094 | 0.0121 |
| catboost | cat_depth6_screening | fixed_2022_val_2023plus_test | 0.6657 | 0.7306 | 0.6057 | 0.2095 | 0.0081 |
| xgboost | xgb_depth5_balanced_regularized | fixed_2022_val_2023plus_test | 0.6650 | 0.7307 | 0.6058 | 0.2095 | 0.0081 |
| lightgbm | lgbm_leaves31_regularized | fixed_2022_val_2023plus_test | 0.6661 | 0.7308 | 0.6059 | 0.2095 | 0.0083 |
| nn | nn_logits_128_64_robust | fixed_2022_val_2023plus_test | 0.6662 | 0.7319 | 0.6059 | 0.2095 | 0.0190 |
| xgboost | xgb_depth6_medium | fixed_2022_val_2023plus_test | 0.6664 | 0.7305 | 0.6061 | 0.2096 | 0.0086 |
| catboost | cat_depth6_screening | fixed_2022_val_2023plus_test | 0.6657 | 0.7302 | 0.6061 | 0.2097 | 0.0061 |
| xgboost | xgb_depth4_slow_regularized | fixed_2022_val_2023plus_test | 0.6648 | 0.7300 | 0.6062 | 0.2097 | 0.0054 |
| xgboost | xgb_depth5_recency_hl_3y | fixed_2022_val_2023plus_test | 0.6651 | 0.7304 | 0.6064 | 0.2097 | 0.0126 |
| nn | nn_logits_128_64_robust | fixed_2022_val_2023plus_test | 0.6627 | 0.7268 | 0.6090 | 0.2109 | 0.0132 |
| nn | nn_logits_128_64_32_lowdrop | fixed_2022_val_2023plus_test | 0.6616 | 0.7262 | 0.6092 | 0.2111 | 0.0162 |
| nn | nn_logits_96_48_lowdrop | fixed_2022_val_2023plus_test | 0.6631 | 0.7272 | 0.6097 | 0.2111 | 0.0183 |
| xgboost | xgb_depth5_recency_hl_12y | train_2010_2015__val_2016_2016__test_2017_2017 | 0.6909 | 0.7617 | 0.5791 | 0.1982 | 0.0063 |
| xgboost | xgb_depth5_balanced_regularized | train_2010_2015__val_2016_2016__test_2017_2017 | 0.6871 | 0.7589 | 0.5814 | 0.1992 | 0.0065 |
| xgboost | xgb_depth5_recency_hl_12y | train_2010_2015__val_2016_2016__test_2017_2017 | 0.6872 | 0.7591 | 0.5814 | 0.1992 | 0.0054 |
| xgboost | xgb_depth5_recency_hl_8y | train_2010_2015__val_2016_2016__test_2017_2017 | 0.6867 | 0.7587 | 0.5817 | 0.1993 | 0.0066 |
| catboost | cat_depth6_screening | train_2010_2015__val_2016_2016__test_2017_2017 | 0.6887 | 0.7586 | 0.5820 | 0.1994 | 0.0043 |
| lightgbm | lgbm_leaves31_regularized | train_2010_2015__val_2016_2016__test_2017_2017 | 0.6874 | 0.7584 | 0.5821 | 0.1995 | 0.0048 |
| nn | nn_logits_128_64_32_lowdrop | train_2010_2015__val_2016_2016__test_2017_2017 | 0.6814 | 0.7534 | 0.5864 | 0.2013 | 0.0060 |
| xgboost | xgb_depth5_recency_hl_12y | train_2012_2017__val_2018_2018__test_2019_2019 | 0.6625 | 0.7262 | 0.6088 | 0.2110 | 0.0089 |
| lightgbm | lgbm_leaves31_regularized | train_2012_2017__val_2018_2018__test_2019_2019 | 0.6587 | 0.7219 | 0.6118 | 0.2123 | 0.0073 |
| xgboost | xgb_depth5_recency_hl_12y | train_2012_2017__val_2018_2018__test_2019_2019 | 0.6561 | 0.7214 | 0.6121 | 0.2125 | 0.0079 |
| xgboost | xgb_depth5_balanced_regularized | train_2012_2017__val_2018_2018__test_2019_2019 | 0.6580 | 0.7215 | 0.6122 | 0.2125 | 0.0078 |
| xgboost | xgb_depth5_recency_hl_8y | train_2012_2017__val_2018_2018__test_2019_2019 | 0.6586 | 0.7215 | 0.6122 | 0.2125 | 0.0091 |
| catboost | cat_depth6_screening | train_2012_2017__val_2018_2018__test_2019_2019 | 0.6571 | 0.7189 | 0.6138 | 0.2133 | 0.0067 |
| nn | nn_logits_128_64_32_lowdrop | train_2012_2017__val_2018_2018__test_2019_2019 | 0.6572 | 0.7173 | 0.6154 | 0.2139 | 0.0088 |
| xgboost | xgb_depth5_recency_hl_12y | train_2014_2019__val_2020_2020__test_2021_2021 | 0.6576 | 0.7206 | 0.6133 | 0.2130 | 0.0136 |
| xgboost | xgb_depth5_balanced_regularized | train_2014_2019__val_2020_2020__test_2021_2021 | 0.6556 | 0.7150 | 0.6189 | 0.2151 | 0.0150 |
| xgboost | xgb_depth5_recency_hl_8y | train_2014_2019__val_2020_2020__test_2021_2021 | 0.6523 | 0.7149 | 0.6189 | 0.2151 | 0.0172 |
| xgboost | xgb_depth5_recency_hl_12y | train_2014_2019__val_2020_2020__test_2021_2021 | 0.6540 | 0.7146 | 0.6190 | 0.2152 | 0.0153 |
| catboost | cat_depth6_screening | train_2014_2019__val_2020_2020__test_2021_2021 | 0.6569 | 0.7126 | 0.6203 | 0.2158 | 0.0160 |
| lightgbm | lgbm_leaves31_regularized | train_2014_2019__val_2020_2020__test_2021_2021 | 0.6550 | 0.7134 | 0.6204 | 0.2158 | 0.0156 |
| nn | nn_logits_128_64_32_lowdrop | train_2014_2019__val_2020_2020__test_2021_2021 | 0.6506 | 0.7068 | 0.6273 | 0.2184 | 0.0249 |
| xgboost | xgb_depth5_recency_hl_12y | train_2016_2021__val_2022_2022__test_2023_2023 | 0.6681 | 0.7332 | 0.6041 | 0.2087 | 0.0095 |
| catboost | cat_depth6_screening | train_2016_2021__val_2022_2022__test_2023_2023 | 0.6644 | 0.7273 | 0.6083 | 0.2107 | 0.0093 |
| xgboost | xgb_depth5_recency_hl_12y | train_2016_2021__val_2022_2022__test_2023_2023 | 0.6649 | 0.7278 | 0.6085 | 0.2106 | 0.0126 |
| xgboost | xgb_depth5_recency_hl_8y | train_2016_2021__val_2022_2022__test_2023_2023 | 0.6647 | 0.7273 | 0.6085 | 0.2107 | 0.0139 |
| lightgbm | lgbm_leaves31_regularized | train_2016_2021__val_2022_2022__test_2023_2023 | 0.6640 | 0.7273 | 0.6086 | 0.2108 | 0.0134 |
| xgboost | xgb_depth5_balanced_regularized | train_2016_2021__val_2022_2022__test_2023_2023 | 0.6641 | 0.7273 | 0.6088 | 0.2108 | 0.0153 |
| nn | nn_logits_128_64_32_lowdrop | train_2016_2021__val_2022_2022__test_2023_2023 | 0.6564 | 0.7199 | 0.6130 | 0.2129 | 0.0076 |