Recovering a hidden signal
A bin-boundary condition was collapsing 0/1 binary features into a single bin and discarding them with an IV of 0. Once the boundary was fixed:
| Bondora | 0.640 → 0.709 |
|---|---|
| Mortgage | 0.591 → 0.676 |
| GMSC (cv) | 0.783 → 0.850 |
Research · Validation
We don’t claim.
We measure.
The HACS engine learns from public credit data with recorded default and delinquency outcomes, and is tested on data it never trained on. Every figure carries its measurement date and method.
4,538,938
17 core datasets with outcome labels
40,048,548
52 public credit datasets collected and hash-checked
4
Datasets with holdout AUC of 0.70 or above
0.806
Headline holdout AUC (HELOC)
The data
The world’s credit data, in one place
We collected 52 public credit datasets from UCI, Kaggle, PKDD, the US Federal Reserve SCF, the World Bank Findex and others, and checked them against a sha256 catalog. The 17 with outcome labels are the core of the discrimination measurements; data suspected of being duplicated or synthetic is kept out of every figure.
17
Core — with default and delinquency outcome labels
8
Reference datasets
16
Auxiliary, feature and derived data
11
Suspected duplicate or synthetic — excluded from all figures
Discrimination · 2026-09-25
Every dataset, down to thin-file
Holdout AUC is measured on the 20% not used in training. Telco is tested one month out of time (OOT); Bondora, LendingClub, L&T and Prosper are split into 10 time quantiles and tested on the latest. The dots on the right are measured only on people with thin credit histories (thin-file).
Holdout AUC · Thin-file AUC
Bar = headline AUC, dot = thin-file AUC, dashed line = 0.70 target
KPI Final Results Report 6.3 · re-measured 2026-09-25
| Dataset | Rows | Default rate | Headline AUC | With external/platform scores | Thin-file AUC |
|---|---|---|---|---|---|
| HELOC (FICO) | 10,459 | 52.2% | 0.806 | 0.809 | 0.773 |
| Telco microcredit | 168,919 | 15.5% | 0.769 OOT | — | 0.687 |
| Taiwan | 30,000 | 22.1% | 0.768 | — | 0.717 |
| Bondora | 99,999 | 52.5% | 0.709 | 0.705 | 0.551 |
| Mortgage | 148,670 | 24.6% | 0.676 | — | — |
| Prosper | 38,960 | 31.2% | 0.670 | 0.707 | 0.670 |
| Home Credit | 307,511 | 8.1% | 0.662 | 0.737 | 0.670 |
| LendingClub | 336,776 | 14.8% | 0.629 | 0.691 | 0.570 |
| L&T | 233,154 | 21.7% | 0.567 | 0.588 | 0.578 |
| GMSC | 150,000 | 6.7% | cv 0.850 | — | 0.737 |
Beyond the target (0.70 on 3 or more public datasets, 0.75 on 1 headline dataset): 4 datasets reach 0.70 or above, with a headline of 0.806. That some datasets are hard to score from application data alone is exactly why an alternative-data axis is needed.
How the AUC went up
Three things that raised the AUC
1,152 candidates, pre-registered rules
We searched 1,152 candidate production points tables, but recorded the selection rule first and confirmed the effect of each adjustment with 95% confidence intervals from resampling the same sample 200 times (bootstrap).
| Taiwan | GMSC | Home Credit | |
|---|---|---|---|
| v1 | 0.728 | 0.676 | 0.599 |
| v2 | 0.737 | 0.813 | 0.600 |
| v3 Level 1 | 0.742 | 0.819 | 0.666 |
Reading the rejected, too
Training only on approved applicants misses the risk of those who were rejected. On Home Credit, treating the bottom 30% as rejected and re-scoring them with parceling raised discrimination.
0.639 → 0.656
Beyond AUC
Not just the ranking — the probabilities are right, too
KS is the maximum separation between the good and bad distributions; ECE is the gap between predicted probability and actual default rate (calibration); Brier is the squared error of the probability forecast.
| Dataset | KS | ECE | Brier |
|---|---|---|---|
| Telco | 0.558 | — | — |
| GMSC | 0.545 | — | — |
| HELOC | 0.454 | 0.041 | 0.181 |
| Taiwan | 0.393 | 0.020 | 0.139 |
| Home Credit | 0.233 | 0.002 | 0.071 |
| LendingClub | 0.194 | 0.014 | — |
Model governance
No pass, no production
Every statistical model goes through a DRAFT → CHALLENGER → CHAMPION → RETIRED lifecycle. Even after it clears every promotion gate, a human presses the final button.
Blend
Statistical weight λ ≤ 0.70
λ = min(share of observed features, 0.70). Blending with the expert rules in logit space happens only when a validated champion exists, with a margin of conservatism (MoC ≤ 1.25 logit).
Refresh
Every 100 new labels
Retraining once 100 or more new outcome labels accumulate, recalibration at 50 rows or more. Drift (PSI) and fairness (AIR 4/5) are monitored alongside.
Provenance
3,000 decisions, 30/30 reproduced
Model version · policy version · feature snapshot · hash are bound together in a chain. We recomputed recorded decisions and confirmed a bit-for-bit match.
Methodology
The mathematics of a bank-standard scorecard
01 · WOE
WOE = ln(%good / %bad)
Weight of evidence per bin. Laplace +0.5 for empty bins.
02 · IV
IV = Σ(%good − %bad)·WOE
Variables without predictive power are dropped, and adjacent bins whose risk direction reverses are merged monotonically.
03 · Logistic
logOdds = β₀ + Σβⱼ·WOEⱼ
IRLS with ridge regularization. Feature selection also runs inside the cross-validation folds to prevent information leakage.
04 · Scaling
PD = 1 / (1 + 50·2^((s−600)/20))
Siddiqi scaling: base score 600, PDO 20, base odds 50:1.
Engineering quality
The engine is proven in code
2,432
Backend tests · 0 failures
287
App tests passed
216
Admin console tests passed
26/28
External review findings addressed (2026-09-07)