import matplotlib
matplotlib.use("Agg") # headless: render figures to buffers, never pop a window
import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
rng = np.random.default_rng(42) # one pinned generator drives the whole notebook
import matplotlib.pyplot as plt
plt.rcParams["figure.figsize"] = (9, 5)
plt.rcParams["axes.grid"] = True
plt.rcParams["grid.alpha"] = 0.3
print("numpy ", np.__version__)
print("pandas ", pd.__version__)
import statsmodels
print("statsmodels", statsmodels.__version__)nb6.4 — Mortgage-Disparity Decomposition: How Much of Maya’s Gap Is Real Risk?
Week 6, Chapter 6.4. Companion notebook to the Reader’s Guide on Bartlett et al. (2022) and Bhutta, Hizmo & Ringo.
Maya has the disparity. On a stack of mortgage applications, minority applicants are denied far more often than white applicants with — at first glance — similar files. Mentor Session 4 drilled the one sentence she now cannot unhear: a disparity is not a finding. The raw gap between “minority borrowers are denied more” and “lenders discriminate against minority borrowers” is the entire intellectual content of fair lending, and no list of controls closes it by itself.
This notebook builds the machinery to take that raw gap apart. We will:
- Plant a world where we know the truth — a seeded, synthetic, HMDA-like loan-level dataset with (a) a genuine risk-based component and (b) a tunable direct-discrimination knob. Because we built it, we know exactly how much discrimination is in it, and we can check whether our methods recover it.
- Show the raw gap in denial — the alarming “before” picture.
- Decompose it Blinder–Oaxaca-style into an explained part (legitimate risk: income, LTV, DTI, credit score) and an unexplained residual — and watch the residual shrink toward the planted truth as the controls go in.
- Demonstrate the omitted-creditworthiness problem — drop the credit score and watch the residual inflate. This is Week 2’s omitted-variable bias (OVB), finally measured rather than feared, and it is the spine of the whole HMDA literature: HMDA does not contain the credit score.
- Spring the over-controlling trap — condition on a variable that sits on the discrimination pathway (the offered interest rate / a steering channel) and watch the measured discrimination wrongly vanish. Controlling for more is not always controlling for better.
- Compare a “human” rule to an “algorithmic / race-blind” rule — the Bartlett-vs-BHR question: does removing the human in the loop make the disparity disappear?
The discipline is Lab 4’s slogan, applied to cross-sectional decomposition instead of DiD: when you are unsure whether a tool does what it claims, build a universe where you know the truth and check.
1. Build a universe where we know the truth
We simulate N mortgage applications. Each applicant carries a protected-group indicator minority \in\{0,1\} and four observable risk factors a fair lender is allowed to use:
- income (annual, in $000s),
- LTV (loan-to-value ratio, in percent — higher is riskier),
- DTI (debt-to-income ratio, in percent — higher is riskier),
- credit score (300–850 FICO-like — higher is safer).
The honesty of a synthetic study lives entirely in its data-generating process (DGP), so we state ours in full. Two design choices are deliberate and load-bearing.
(a) The risk factors genuinely differ by group, for reasons upstream of the lender. Decades of unequal wealth, income, and credit access mean the minority group in our world has, on average, modestly lower income and credit scores and higher LTV/DTI. This is real, legitimate risk that a non-discriminating lender would price — and it is exactly what makes the raw gap an unreliable measure of discrimination. We are not asserting these patterns are fair in any deep sense (a credit score shaped by historical inequity is itself contestable, as Ch 6.4 §6 warns); we are asserting only that a lender who used them identically across groups would still produce a raw gap.
(b) The denial decision has a true latent risk index plus a tunable discrimination term. An applicant is denied when a latent “risk score” crosses a threshold. That latent index is built from the true coefficients on the risk factors plus a direct penalty DISCRIM_TRUE applied to minority applicants for no risk reason at all — pure differential treatment. We will turn that knob and watch our estimates chase it.
The one-line spec (CONVENTIONS §4): outcome = loan denial (0/1); key regressor = minority; controls = income, LTV, DTI, credit score; fixed effects = none; standard errors = classical/robust (this is a measurement exercise, not an inference showdown); sample = synthetic applications; identifying assumption = the residual minority coefficient equals differential treatment only if the controls capture all legitimate risk and nothing on the discrimination pathway.
N = 60_000 # applications
DISCRIM_TRUE = 0.80 # TUNABLE: direct discrimination, in latent-index units (the knob)
# True coefficients of the latent risk index. Sign convention: HIGHER latent risk -> MORE likely denied.
B_INCOME = -0.018 # more income -> safer (lower risk)
B_LTV = 0.045 # higher LTV -> riskier
B_DTI = 0.055 # higher DTI -> riskier
B_SCORE = -0.020 # higher score -> safer
B0 = -1.10 # base level of the latent index (sets the overall denial rate)
def simulate_hmda(n, discrim, rng):
"""Seeded synthetic HMDA-like loan-level data with (a) real risk and (b) tunable discrimination.
Returns a fresh DataFrame (never a view): minority, the four risk factors, the OFFERED RATE
(a downstream pricing/steering channel), the latent risk index, and the denial outcome.
"""
minority = rng.integers(0, 2, size=n) # protected-group indicator
# (a) Risk factors differ by group for UPSTREAM reasons (wealth/credit-access gaps), not lender choice.
income = rng.normal(85 - 12 * minority, 25).clip(15, 400) # $000s; minority lower on average
ltv = rng.normal(78 + 4.0 * minority, 12).clip(20, 100) # %; minority higher on average
dti = rng.normal(34 + 3.5 * minority, 8).clip(5, 65) # %; minority higher on average
score = rng.normal(720 - 30 * minority, 55).clip(300, 850) # FICO-like; minority lower on average
# (b) Latent risk index = true risk pricing + DIRECT discrimination penalty on minority + noise.
latent = (B0
+ B_INCOME * (income - 85)
+ B_LTV * (ltv - 78)
+ B_DTI * (dti - 34)
+ B_SCORE * (score - 720)
+ discrim * minority # <-- pure differential treatment
+ rng.normal(0.0, 1.0, size=n))
denied = (latent > 0).astype(int) # denied if latent risk crosses 0
# The OFFERED RATE is set DOWNSTREAM of the same latent risk AND of group (a steering channel):
# it partly encodes the discrimination, so it is a POST-TREATMENT variable, not a clean control.
offered_rate = (4.0
+ 0.55 * latent # tracks the latent index...
+ 0.45 * minority # ...PLUS a direct steering markup
+ rng.normal(0.0, 0.30, size=n)).clip(2.0, 12.0)
return pd.DataFrame({
"minority": minority, "income": income, "ltv": ltv, "dti": dti,
"score": score, "offered_rate": offered_rate, "latent": latent, "denied": denied,
})
loans = simulate_hmda(N, DISCRIM_TRUE, rng)
print(f"Simulated {len(loans):,} applications. Overall denial rate: {loans['denied'].mean():.3f}")
print(f"Minority share: {loans['minority'].mean():.3f} | planted discrimination knob = {DISCRIM_TRUE}")
loans.drop(columns="latent").head()2. The raw gap — the alarming “before” picture
Read it the way a journalist would: just compare denial rates across groups, no controls. This is the number that makes a headline. It is also, by construction in our world, a mixture of two very different things — legitimate risk that genuinely differs across groups, and the direct discrimination we planted — and the raw gap cannot tell them apart.
rate_min = loans.loc[loans["minority"] == 1, "denied"].mean()
rate_maj = loans.loc[loans["minority"] == 0, "denied"].mean()
raw_gap_pp = 100.0 * (rate_min - rate_maj)
print(f"Denial rate, minority group : {rate_min:.3f}")
print(f"Denial rate, majority group : {rate_maj:.3f}")
print(f"RAW GAP : {raw_gap_pp:+.2f} percentage points")
# Same number as a one-variable linear-probability regression coefficient (no controls):
raw_lpm = smf.ols("denied ~ minority", data=loans).fit()
print(f"\nRaw gap as an OLS coefficient on `minority`: {raw_lpm.params['minority']*100:+.2f} pp"
f" (identical to the difference in means)")3. Add the risk controls — watch the gap march
Now we do the BHR move (Ch 6.4 §4): add the legitimate risk factors one block at a time and watch the minority coefficient march across columns. Each control we add is a piece of risk that genuinely differs across groups; once we hold it fixed, it can no longer masquerade as discrimination, so the coefficient shrinks. What is left over after we have conditioned on all the legitimate risk is the unexplained residual — our estimate of differential treatment.
We use a linear probability model (OLS with a 0/1 outcome) throughout. It is not the most efficient model for a binary outcome, but its coefficients are differences in probabilities you can read directly in percentage points — exactly the currency a fair-lending decomposition trades in, and far easier for a 17-year-old to reason about than a log-odds. We report the minority coefficient \times
100 so every number below is “percentage points of denial-probability gap attributable to being in the minority group, holding the listed controls fixed.”
Watch the bottom-row N stay fixed (Ch 6.4 §4: be sure the sample is not silently changing as controls enter).
specs = {
"(1) raw, no controls" : "denied ~ minority",
"(2) + income" : "denied ~ minority + income",
"(3) + income, LTV, DTI" : "denied ~ minority + income + ltv + dti",
"(4) + ALL risk incl. SCORE" : "denied ~ minority + income + ltv + dti + score",
}
print(f"{'specification':<30}{'minority coef (pp)':>20}{'N':>10}")
print("-" * 60)
march = {}
for label, formula in specs.items():
m = smf.ols(formula, data=loans).fit()
coef_pp = 100.0 * m.params["minority"]
march[label] = coef_pp
print(f"{label:<30}{coef_pp:>+18.2f} {int(m.nobs):>10,}")
print(f"\nThe raw {march['(1) raw, no controls']:+.2f} pp gap shrinks to "
f"{march['(4) + ALL risk incl. SCORE']:+.2f} pp once all legitimate risk is held fixed.")
print("That residual is what survives as our estimate of differential treatment.")4. The Blinder–Oaxaca decomposition: explained vs. unexplained, in one number
The march-across-columns table shows the residual shrinking, but a referee wants the split stated as an accounting identity: of the raw gap, how many percentage points are explained by group differences in legitimate risk, and how many are the unexplained residual?
That is the Blinder–Oaxaca decomposition. Here is the whole idea in one breath. Fit a denial model separately for each group, giving each its own coefficients (\hat\beta^{maj}, \hat\beta^{min}) and its own average risk profile (\bar X^{maj}, \bar X^{min}). The mean denial gap then splits exactly:
\underbrace{\bar Y^{min}-\bar Y^{maj}}_{\text{raw gap}} =\underbrace{(\bar X^{min}-\bar X^{maj})\,\hat\beta^{maj}}_{\text{EXPLAINED: groups differ in } X} +\underbrace{\bar X^{min}\,(\hat\beta^{min}-\hat\beta^{maj})}_{\text{UNEXPLAINED: same } X \text{ priced differently}}.
Read the two pieces in plain English. The explained part asks: if minority applicants were priced by the majority’s own rule, how much gap would their riskier average profile alone produce? The unexplained part asks: holding the profile at the minority average, how much extra gap comes from the two groups being scored by different rules? Differential treatment lives in the second piece — it is the same applicant getting a different answer because of group, which is precisely discrimination.
(There is a well-known ambiguity — you could equally weight the explained part by \hat\beta^{min} — called the index-number problem; we use the majority coefficients as the non-discriminatory benchmark, the standard choice, and note that the qualitative split is robust to it.)
def oaxaca(df, controls):
"""Blinder-Oaxaca decomposition of the minority-majority denial gap into
EXPLAINED (group differences in X, priced by the majority rule) and UNEXPLAINED (different rule).
Returns (raw_gap_pp, explained_pp, unexplained_pp), all in percentage points."""
rhs = " + ".join(controls)
maj = df[df["minority"] == 0].copy()
minr = df[df["minority"] == 1].copy()
m_maj = smf.ols(f"denied ~ {rhs}", data=maj).fit()
m_min = smf.ols(f"denied ~ {rhs}", data=minr).fit()
cols = ["Intercept"] + controls
Xbar_maj = pd.Series({"Intercept": 1.0, **{c: maj[c].mean() for c in controls}})[cols]
Xbar_min = pd.Series({"Intercept": 1.0, **{c: minr[c].mean() for c in controls}})[cols]
b_maj = m_maj.params[cols]
b_min = m_min.params[cols]
raw = minr["denied"].mean() - maj["denied"].mean()
explained = float((Xbar_min - Xbar_maj) @ b_maj) # endowments, at majority prices
unexplained = float(Xbar_min @ (b_min - b_maj)) # coefficients, at minority endowments
return 100*raw, 100*explained, 100*unexplained
risk_controls = ["income", "ltv", "dti", "score"]
raw, expl, unexpl = oaxaca(loans, risk_controls)
print("Blinder-Oaxaca decomposition (full risk set, score INCLUDED):")
print(f" raw gap : {raw:+.2f} pp")
print(f" EXPLAINED by legitimate risk : {expl:+.2f} pp ({100*expl/raw:.0f}% of the raw gap)")
print(f" UNEXPLAINED residual : {unexpl:+.2f} pp ({100*unexpl/raw:.0f}% of the raw gap)")
print(f" check (expl + unexpl == raw) : {expl + unexpl:+.2f} pp")
print(f"\nThe unexplained residual ({unexpl:+.2f} pp) is our measured differential treatment.")5. The omitted-creditworthiness problem — drop the score and watch the residual inflate
Here is the single most important fact about real HMDA, and the spine of the entire literature in Ch 6.4: HMDA does not contain the applicant’s credit score. Sun & Gao (2019, PNAS) had to reach for a research design precisely because the score was missing; BHR’s whole contribution was to get the score and measure how much of the residual it explained.
We can show exactly what that missing variable does, because in our world we control it. Re-run the decomposition with the score omitted from the controls — simulating a HMDA-only analyst — and compare the residual to the full-information version.
This is Week 2 OVB made concrete. The omitted score is (i) a real driver of denial and (ii) correlated with minority (our minority group has lower average scores). Both conditions for OVB hold, so leaving it out biases the residual minority coefficient. And the direction is the dangerous one: because the score is protective and the minority group has less of it, omitting it dumps real risk into the residual and inflates the apparent discrimination. A HMDA-only analyst would over-state differential treatment — not from malice, but from a missing column.
controls_with_score = ["income", "ltv", "dti", "score"]
controls_no_score = ["income", "ltv", "dti"] # the HMDA-only analyst's world
raw_w, expl_w, unexpl_w = oaxaca(loans, controls_with_score)
raw_n, expl_n, unexpl_n = oaxaca(loans, controls_no_score)
print(f"{'analyst':<28}{'unexplained residual (pp)':>28}")
print("-" * 56)
print(f"{'full info (score IN)':<28}{unexpl_w:>+26.2f}")
print(f"{'HMDA-only (score OUT)':<28}{unexpl_n:>+26.2f}")
print(f"\nDropping the credit score INFLATES the measured discrimination by "
f"{unexpl_n - unexpl_w:+.2f} pp ({100*(unexpl_n-unexpl_w)/unexpl_w:+.0f}%).")
print("Same data, same truth -- one missing column moves the verdict. That is Week-2 OVB, measured.")
# Confirm the OVB conditions explicitly: score predicts denial AND differs by group.
score_gap = loans.loc[loans.minority==1,"score"].mean() - loans.loc[loans.minority==0,"score"].mean()
print(f"\nOVB check -- mean score gap (min - maj): {score_gap:+.1f} FICO points (correlated with group);"
f" true B_SCORE = {B_SCORE} (<0, protective).")6. The over-controlling trap — condition on the rate and watch discrimination vanish
OVB warns: too few controls inflates the residual. The over-controlling trap is the mirror image: the wrong control collapses it. Mentor 4’s warm-up 3 stated the rule — never condition on a variable that lies on the discrimination pathway, because doing so regresses away the very effect you are hunting (Ch 6.4 §6).
In our world, the offered interest rate is exactly such a variable. Look back at the DGP: the rate is set downstream of the latent risk index and carries a direct minority steering markup. It is a post-treatment / mediator variable — part of the discrimination is channeled through it. A naive analyst, reaching for “more controls = more rigor,” might add the rate to the denial regression to “hold loan terms fixed.” Watch what happens.
good_controls = "income + ltv + dti + score"
m_good = smf.ols(f"denied ~ minority + {good_controls}", data=loans).fit()
m_over = smf.ols(f"denied ~ minority + {good_controls} + offered_rate", data=loans).fit()
resid_good = 100 * m_good.params["minority"]
resid_over = 100 * m_over.params["minority"]
print(f"{'specification':<46}{'minority coef (pp)':>18}")
print("-" * 64)
print(f"{'(4) legitimate risk controls only':<46}{resid_good:>+18.2f}")
print(f"{'(5) ALSO control for offered_rate (a MEDIATOR)':<46}{resid_over:>+18.2f}")
print(f"\nAdding the offered rate collapses the measured discrimination from "
f"{resid_good:+.2f} pp to {resid_over:+.2f} pp.")
print("The discrimination did NOT go away -- we controlled it away. The rate is ON the pathway:")
print("part of the bias FLOWS THROUGH the markup, so conditioning on it absorbs the very effect we want.")
print("Rule (Mentor 4 / Ch 6.4 §6): is this control a CONFOUNDER to hold fixed, or a CHANNEL to leave alone?")7. One picture: the gap decomposed, and the two traps
Put the whole story in a single figure. From left to right: the raw gap; the residual after legitimate controls (our honest estimate, which should land near the planted truth translated into percentage points); the inflated residual when the score is dropped (OVB); and the collapsed residual when we over-control on the rate. The horizontal line marks the residual we should recover with the right controls — the target both traps miss in opposite directions.
labels = ["raw gap\n(no controls)", "residual\n(risk controls)",
"residual\n(score DROPPED)\nOVB inflates", "residual\n(rate CONTROLLED)\nover-control collapses"]
values = [raw_w, unexpl_w, unexpl_n, resid_over]
colors = ["C3", "C0", "C1", "C2"]
fig, ax = plt.subplots(figsize=(10, 5.5))
bars = ax.bar(labels, values, color=colors, alpha=0.85)
ax.axhline(0, color="0.4", lw=0.8)
ax.axhline(unexpl_w, color="C0", ls="--", lw=1.2, alpha=0.7,
label=f"honest residual = {unexpl_w:+.1f} pp")
for b, v in zip(bars, values):
ax.text(b.get_x() + b.get_width()/2, v + (0.4 if v >= 0 else -0.4),
f"{v:+.1f}", ha="center", va="bottom" if v >= 0 else "top", fontweight="bold")
ax.set_ylabel("minority denial gap (percentage points)")
ax.set_title("Decomposing Maya's gap: legitimate risk explains most of it;\n"
"dropping the score inflates the residual, over-controlling collapses it")
ax.legend(loc="upper right")
fig.tight_layout()
fig.savefig("nb6.4-disparity-decomposition.png", dpi=120)
print("Saved nb6.4-disparity-decomposition.png")
plt.close(fig)8. Human vs. algorithmic / race-blind lending — does removing the human cure it?
Now the Week-6 question that the algorithmic era did not retire (Ch 6.4): if a computer makes the call instead of a loan officer, does the discrimination go away? Bartlett et al. and BHR take that hope seriously and refuse to flatter it.
We can stage the contrast directly. Build two decision rules and apply each to the same applicants:
- Human rule. The denial decision we already simulated — the latent index that includes the
DISCRIM_TRUEpenalty on minority applicants. A human with discretion, applying a different rule by group. - Race-blind / algorithmic rule. Re-decide every application using a latent index built from the risk factors only — the discrimination term forced to zero. This is BHR’s race-blind automated underwriting system (AUS): a mechanical counterfactual for what a rule that cannot see group would decide. Crucially, we hold the noise draw fixed across the two rules, so the only difference between them is whether the group penalty is switched on.
Compare the residual minority gap (after full legitimate controls) under each rule. The race-blind rule should drive the residual toward zero — it literally cannot apply a different rule by group. That is the good news the algorithm offers. The sobering caveat, which the two papers spend their pages on and which our clean toy understates: a real algorithm can still discriminate by pricing on a proxy for race (geography, a steered product), so “race-blind in the code” is not “race-blind in effect.” Our synthetic AUS is blind by construction; real ones must be audited, not trusted.
(One honest technical aside: the race-blind residual will land near zero but not exactly zero — a fraction of a percentage point survives. That leftover is not hidden discrimination; it is the linear probability model approximating a genuinely nonlinear threshold rule on two groups whose risk-factor distributions differ. It is the price of reading coefficients in tidy percentage points, and it is tiny next to the human residual — which is exactly the contrast that matters.)
def relatent_and_denial(df, discrim, rng_noise):
"""Rebuild the latent index and denial decision for the SAME applicants under a chosen
discrimination level, reusing a FIXED noise draw so the only change is the group penalty."""
latent = (B0
+ B_INCOME * (df["income"] - 85)
+ B_LTV * (df["ltv"] - 78)
+ B_DTI * (df["dti"] - 34)
+ B_SCORE * (df["score"] - 720)
+ discrim * df["minority"]
+ rng_noise)
return (latent > 0).astype(int)
# One fixed noise draw shared by both rules => the ONLY difference is the discrimination term.
noise = rng.normal(0.0, 1.0, size=len(loans))
human = loans.copy()
human["denied"] = relatent_and_denial(human, DISCRIM_TRUE, noise) # human rule: discrimination ON
algo = loans.copy()
algo["denied"] = relatent_and_denial(algo, 0.0, noise) # race-blind AUS: discrimination OFF
rhs = "minority + income + ltv + dti + score"
resid_human = 100 * smf.ols(f"denied ~ {rhs}", data=human).fit().params["minority"]
resid_algo = 100 * smf.ols(f"denied ~ {rhs}", data=algo).fit().params["minority"]
gap_human = 100 * (human.loc[human.minority==1,"denied"].mean() - human.loc[human.minority==0,"denied"].mean())
gap_algo = 100 * (algo.loc[algo.minority==1,"denied"].mean() - algo.loc[algo.minority==0,"denied"].mean())
print(f"{'rule':<26}{'raw gap (pp)':>16}{'residual after risk controls (pp)':>36}")
print("-" * 78)
print(f"{'human (discretion ON)':<26}{gap_human:>+16.2f}{resid_human:>+36.2f}")
print(f"{'race-blind AUS (OFF)':<26}{gap_algo:>+16.2f}{resid_algo:>+36.2f}")
print(f"\nThe race-blind rule cuts the residual from {resid_human:+.2f} pp to {resid_algo:+.2f} pp.")
print("Note the raw gap does NOT vanish under the algorithm: legitimate risk still differs by group.")
print("Removing the human removes the DISCRETION channel -- not the underlying risk disparity.")9. Verification — did our methods recover the planted truth?
Lab 4’s discipline: before we trust this code on real HMDA (where we can never see the truth), prove on synthetic data — where we can — that it behaves. We assert four things this notebook claimed:
- Adding legitimate controls shrinks the raw gap toward the residual.
- Dropping the score inflates the residual (OVB, in the discrimination-flattering direction).
- Over-controlling on the rate collapses the residual below the honest estimate.
- The race-blind rule drives the residual toward zero, while the human rule leaves it large.
If any assertion fails, the lesson the notebook teaches is not actually in the numbers — so we make the machine check us.
checks = {
"controls shrink the raw gap" : abs(unexpl_w) < abs(raw_w),
"residual is positive (real discrim)" : unexpl_w > 0.5,
"dropping score INFLATES residual" : unexpl_n > unexpl_w + 0.5,
"over-control COLLAPSES residual" : resid_over < unexpl_w - 1.0,
"race-blind rule shrinks residual >85%": abs(resid_algo) < 0.15 * abs(resid_human),
"race-blind residual near zero (<2pp)" : abs(resid_algo) < 2.0,
"human rule keeps a large residual" : resid_human > 3.0,
"explained + unexplained == raw" : abs((expl_w + unexpl_w) - raw_w) < 1e-6,
}
for name, ok in checks.items():
print(f" [{'PASS' if ok else 'FAIL'}] {name}")
assert all(checks.values()), "A verification check failed -- the planted lesson is not in the numbers."
print("\nALL CHECKS PASSED: controls shrink the gap toward the planted truth; dropping the score")
print("inflates the residual; over-controlling masks the effect; the race-blind rule neutralizes it.")10. The real HMDA pull — the recipe you would run (NOT executed here)
Everything above runs offline on a universe we built. This is how you would point the same decomposition at real, public HMDA — the recipe for Capstone 1 (Fair Lending on HMDA). HMDA LAR is free and public (FFIEC/CFPB), but it is enormous (tens of millions of rows per year) and, most importantly for this chapter, it does not contain the credit score. The cell below is illustrative — it is not executed (no network, no pinned vintage in this notebook), exactly as Lab 4’s Path A. Read it for the query pattern and the schema, and pin a vintage before you cite a single number from it.
The decisive caveat is the one this whole notebook dramatized in §5: on real HMDA you are permanently the “score-dropped” analyst from the OVB demo. Your residual is therefore an upper bound on differential treatment, not a clean estimate — which is precisely why Sun & Gao (2019) leaned on a design, and why BHR’s contribution was getting the score at all.
# ---------------------------------------------------------------------------
# NON-EXECUTED illustrative recipe. Run on the camp container against a PINNED
# vintage; confirm parameter names against the live API docs for that vintage. [CHECK]
# ---------------------------------------------------------------------------
import requests
# CFPB HMDA Data Browser -- public, no key required. We pull only the columns we need,
# filtered by state and year, so we never move the multi-GB national microdata.
HMDA = "https://ffiec.cfpb.gov/v2/data-browser-api/view/csv"
def fetch_hmda(states, year):
# Filtered HMDA LAR pull: applicant demographics, action taken, and the risk factors
# HMDA DOES carry. NOTE: HMDA has NO credit score -- that is the OVB you cannot fix here.
params = {
"states": ",".join(states), # e.g. ["VA", "MD", "DC"]
"years": year, # PIN this vintage and record it [CHECK]
"actions_taken": "1,3", # 1 = originated, 3 = denied
}
r = requests.get(HMDA, params=params, timeout=120)
r.raise_for_status()
from io import StringIO
return pd.read_csv(StringIO(r.text))
# raw = fetch_hmda(["VA", "MD", "DC"], year=2022) # run on the container, pinned vintage
# Build the decomposition inputs from the real fields HMDA provides:
# minority <- derived_race / derived_ethnicity (define your protected group explicitly)
# denied <- (action_taken == 3)
# income <- income (HMDA: applicant income, $000s)
# ltv <- loan_to_value_ratio (post-2018 field)
# dti <- debt_to_income_ratio (post-2018, BUCKETED -- not continuous)
# rate_spread <- rate_spread (post-2018 PRICING field; a MEDIATOR -- do NOT control on it)
# score <- *** NOT IN HMDA *** (the permanent omitted variable; your residual is an UPPER BOUND)
#
# Then run the IDENTICAL oaxaca(...) decomposition from Section 4 -- with the explicit caveat that
# the missing score means you are stuck in the Section-5 "score-dropped" regime by construction.11. Where this connects — Sun & Gao (2019) and Capstone 1
Step back and see what you have built. You took a raw disparity, split it Blinder–Oaxaca-style into legitimate risk and an unexplained residual, watched the residual shrink toward a planted truth as the right controls entered, and then sprung both traps that ambush this literature: omitted creditworthiness inflated the residual (the missing-score problem that defines HMDA work), and over-controlling on a mediator collapsed it. Finally you watched a race-blind rule neutralize the discretion channel without erasing the underlying risk disparity — Bartlett-vs-BHR in miniature.
This is exactly why Sun & Gao (2019, PNAS) reached for a design rather than a kitchen-sink regression in their same-sex-borrower work: with the credit score permanently missing from HMDA, no pile of controls could close the gap between “we see a disparity” and “we measured discrimination,” so a clean comparison had to carry the argument all the way to Congressional testimony. That paper is the camp’s anchor, and Bartlett et al. / BHR are its algorithmic-era sequel: same question — are two otherwise-identical applicants treated differently? — new defendant.
It is also the launch point for Capstone 1 (Fair Lending on HMDA). The honest residual you can report on real HMDA is an upper bound on differential treatment; your job in the capstone is to shrink the gap between that bound and the truth — by finding more legitimate controls (carefully!), by a matched comparison (Ch 6.4 §7 Exercise 2), or, best of all, by a clean policy design in the spirit of Lab 4. The referee’s first question will always be the one this notebook trained you to ask: what is in the controls — and is the score in there?
Your Turn — vary the true discrimination and the controls
You now have a God’s-eye decomposition lab. Each extension is a small change to the code above; predict the answer before you run it.
1. Sweep the discrimination knob and confirm the method tracks it. Loop DISCRIM_TRUE over, say, [0.0, 0.4, 0.8, 1.2], regenerate the data each time, and record the honest residual (full risk controls, score in). Plot residual vs. knob. Predict: a clean, monotone rising line — more planted discrimination, more measured residual. At knob =0 the residual lands small but not exactly zero (a few pp): that floor is the same linear-probability-vs-threshold misspecification artifact §8 flagged, not hidden bias. The placebo lesson still holds in the slope — the residual moves one-for-one with the truth — but it teaches a second, humbler point: even a correct method carries a small modeling floor, so read the change across the sweep, not any single number against an idealized zero.
resids = []
knobs = [0.0, 0.4, 0.8, 1.2]
for k in knobs:
d = simulate_hmda(N, k, np.random.default_rng(42)) # fresh rng each time for a clean sweep
_, _, u = oaxaca(d, ["income", "ltv", "dti", "score"])
resids.append(u)
# plot resids vs knobs -- monotone rising; small nonzero floor at knob=0 (LPM-vs-threshold artifact)2. Make the OVB worse, then better. In simulate_hmda, widen the group score gap (change 720 - 30*minority to 720 - 60*minority) and re-run Section 5. Predict: the score-dropped residual inflates more, because the omitted variable is now more strongly correlated with group — OVB scales with that correlation (Week 2). Then shrink the gap toward zero and watch the two residuals converge.
3. Add a second legitimate control that HMDA also lacks. Invent an assets (reserves) factor that lowers risk and differs by group, add it to the DGP and to the full-info control set but not to the HMDA-only set. Predict: the full-info residual falls further toward the truth while the HMDA-only residual barely moves — every unobserved legitimate factor is one more reason the HMDA residual is only an upper bound.
4. Build the over-controlling trap on a different channel. Replace the offered rate with a steered flag (a product the applicant was pushed into, set as a function of latent and minority), and condition on it. Predict: the same collapse as §6 — any variable on the discrimination pathway, not just price, will regress away the effect. Then ask the Mentor-4 question for your new variable: confounder to hold fixed, or channel to leave alone?
When you can predict all four before running them, you understand the difference between controlling for risk and controlling away discrimination — which is the whole of fair-lending measurement.