nb6.1 — Building a Patent-Value Panel: Reading a Patent’s Worth Off the Stock Price (KPSS-style)

Week 6, Chapter 6.1. Companion notebook.

Leah has been circling one question since Week 1: how much is an invention actually worth? Counting patents treats a throwaway design tweak and the transistor as one unit each. Counting forward citations is better but slow (citations accrue over a decade) and truncated (recent patents look uncited only because the future has not happened yet). Neither is denominated in the unit anyone cares about — dollars.

Kogan, Papanikolaou, Seru & Stoffman (2017) — KPSS — found the trick: let the stock market value the patent for you. A patent grant is news. If the patent is valuable, the firm’s stock should jump when the grant becomes public; if it is worthless, the stock should not move. So KPSS read the abnormal stock return in a short window around the grant date — the part of the return the market’s normal movement does not explain — as the market’s real-time, dollar-denominated estimate of the patent’s value. That is exactly the event-study logic from Week 4, with the “event” being a patent grant.

This notebook builds the measure from the ground up on a seeded synthetic dataset so it runs offline on your laptop. Because we simulate the data, we get to play God: we bake in a known true patent value, then see whether the event-study reconstruction recovers it. We will:

Simulate firms with daily returns and patents with grant dates, injecting a true patent value as an abnormal return in a short window — plus noise and confounding window news.
Reconstruct the KPSS-style value per patent via a market-model event study (Week 4), then aggregate to a firm-year innovation-value panel.
Validate the measure: show it recovers the planted value (correlation) and predicts a planted future outcome (next-year profitability).
Break it: turn up the confounding window news and watch the measure — and its predictive power — degrade.

A clearly-marked, non-executed cell shows the real path: the USPTO PatentsView bulk API pattern and the public KPSS dataset pointer. We never fabricate real numbers.

import matplotlib
matplotlib.use("Agg")  # headless: render to buffers, never to a screen

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import statsmodels.formula.api as smf

rng = np.random.default_rng(42)  # one generator, pinned, drives the whole notebook

plt.rcParams["figure.figsize"] = (9, 5)
plt.rcParams["axes.grid"] = True
plt.rcParams["grid.alpha"] = 0.3

print("numpy      ", np.__version__)
print("pandas     ", pd.__version__)
import statsmodels
print("statsmodels", statsmodels.__version__)

1. Simulate the world: firms, daily returns, and patents with a known value

We build a small universe of firms observed over several years of trading days. Each firm i has a daily return generated by the market model — the same one-factor return decomposition you met in Week 4 and Week 5:

R_{it} = a_i + b_i\,R_{mt} + \varepsilon_{it},

where R_{mt} is the market return that day (common to all firms), b_i is the firm’s market beta, and \varepsilon_{it} is the idiosyncratic (firm-specific) surprise. The whole KPSS trick lives in \varepsilon_{it}: on most days it is just noise, but on the days right after a patent grant we will quietly add the patent’s true value into it.

Here is the data-generating process (DGP), written forward so you can see exactly what we plant:

R_{it} = a_i + b_i R_{mt} + \underbrace{v_{j}\big/\text{mktcap}_i}_{\text{patent's true \% value, in window}} + \underbrace{c_{j}}_{\text{confounding news, in window}} + \varepsilon_{it}.

A patent j granted to firm i is worth v_j dollars. The market impounds that value as a return over the grant window, so the percentage bump is v_j / \text{mktcap}_i — divide dollars by firm size to get a percent. Run the event study in reverse — measure the abnormal return, multiply by market cap — and you should get v_j back. That round trip is the entire measure. The confound c_j (earnings news that happens to land in the window) is off for now; we switch it on in Section 5 to break things.

# ---- Universe size and calendar -------------------------------------------------------
N_FIRMS   = 60
YEARS     = [2018, 2019, 2020, 2021]
TDAYS_YR  = 252                          # trading days per year (approx)
N_DAYS    = len(YEARS) * TDAYS_YR        # total trading-day index 0..N_DAYS-1

# ---- Event-study windows (in trading days) --------------------------------------------
EST_WINDOW   = 120        # length of the pre-event estimation window for the market model
EST_GAP      = 10         # gap between estimation window and event window (avoid leakage)
EVENT_WIN    = (0, 2)     # grant-window: days t0..t0+2 inclusive -> a 3-day window (KPSS-style)

# ---- Firm-level fundamentals ----------------------------------------------------------
# Each firm gets a market cap (heavily right-skewed: a few giants, many minnows) and a
# beta and alpha for its market model. log-normal mktcap mirrors real cross-sections.
firm_id   = np.arange(N_FIRMS)
log_mktcap = rng.normal(np.log(2_000), 1.1, N_FIRMS)     # in $ millions, on log scale
mktcap_m  = np.exp(log_mktcap)                            # market cap in $ millions
beta_i    = rng.normal(1.0, 0.25, N_FIRMS).clip(0.3, 1.8)
alpha_i   = rng.normal(0.0, 0.0002, N_FIRMS)             # tiny daily drift
sigma_i   = rng.uniform(0.010, 0.020, N_FIRMS)           # idiosyncratic daily vol (1.0%-2.0%)

firms = pd.DataFrame({"firm": firm_id, "mktcap_m": mktcap_m,
                      "beta": beta_i, "alpha": alpha_i, "sigma": sigma_i})
print(f"{N_FIRMS} firms over {len(YEARS)} years = {N_DAYS} trading days.")
print("Market cap ($m) is right-skewed by construction:")
print(firms["mktcap_m"].describe()[["min", "50%", "mean", "max"]].round(1).to_string())

1.1 Patents, grant dates, and the planted dollar value

Now the patents. We hand each firm a random number of patents per year, give each a grant date (a trading-day index), and — crucially — draw each patent’s true dollar value v_j from a right-skewed distribution. This skew is the whole motivation for KPSS: most patents are nearly worthless, a handful are blockbusters, and counting them equally throws away exactly the information that matters. We use a log-normal draw so a few patents are worth hundreds of millions while the median is modest.

We also plant a firm-level trait, innovation quality q_firm, that scales how valuable a firm’s patents tend to be. We will reuse q_firm in Section 4 to manufacture a future outcome the measure should predict — so the validation has something real to find.

# Firm-level latent innovation quality: drives BOTH patent value and (later) future profit.
q_firm = rng.normal(0.0, 1.0, N_FIRMS)
firms["q_firm"] = q_firm

patent_rows = []
pj = 0
for i in range(N_FIRMS):
    for y_idx, year in enumerate(YEARS):
        n_pat = rng.poisson(2.5)                      # ~2-3 patents/firm/year, sometimes 0
        for _ in range(n_pat):
            # grant date: a trading day inside this year, kept clear of the very start so the
            # estimation window fits before it.
            day_in_yr = rng.integers(EST_WINDOW + EST_GAP + 5, TDAYS_YR - EVENT_WIN[1] - 2)
            t0 = y_idx * TDAYS_YR + day_in_yr
            # true value: log-normal $ value, shifted up by the firm's innovation quality.
            log_v = rng.normal(2.4 + 0.8 * q_firm[i], 1.0)        # mean/scale on log($m)
            v_true_m = np.exp(log_v)                              # true value in $ millions
            patent_rows.append({"patent": pj, "firm": i, "year": year,
                                 "grant_t": int(t0), "v_true_m": v_true_m})
            pj += 1

patents = pd.DataFrame(patent_rows)
print(f"{len(patents)} patents granted across {N_FIRMS} firms x {len(YEARS)} years.")
print("True per-patent value ($m) is wildly right-skewed (note mean >> median):")
print(patents["v_true_m"].describe()[["min", "50%", "mean", "max"]].round(2).to_string())

1.2 Look before you trust: the value distribution

KPSS’s first lesson is to look at the distribution before you believe any single number. Plot the true per-patent dollar values and you should see the textbook shape: a dense pile of small patents and a long right tail of blockbusters. On a linear axis the tail squashes everything; on a log axis the shape is legible. This skew is precisely why “count the patents” is the wrong measure — it weights the transistor and a paperclip redesign equally.

fig, ax = plt.subplots(1, 2, figsize=(11, 4))
ax[0].hist(patents["v_true_m"], bins=40, color="steelblue", edgecolor="white")
ax[0].set_title("Per-patent value ($m), linear axis\n(the tail squashes everything)")
ax[0].set_xlabel("true value ($m)"); ax[0].set_ylabel("# patents")

ax[1].hist(np.log10(patents["v_true_m"]), bins=40, color="darkorange", edgecolor="white")
ax[1].set_title("log10 per-patent value\n(skew becomes legible)")
ax[1].set_xlabel("log10 value ($m)"); ax[1].set_ylabel("# patents")
fig.tight_layout()

top_share = patents["v_true_m"].sort_values(ascending=False).head(int(0.1*len(patents))).sum() \
            / patents["v_true_m"].sum()
print(f"Top 10% of patents hold {top_share:.0%} of total planted value -- the blockbuster phenomenon.")

2. Generate daily returns and inject the value into the grant window

Now we spin up the daily return panel. First a common market return path R_{mt}, then each firm’s return from its market model. Finally we walk through every patent and add its true percentage value v_j/\text{mktcap}_i into the firm’s idiosyncratic return, spread across the 3-day grant window. That is the signal the event study will try to recover.

A subtlety worth pausing on: we add the value as a return, and a return is a percentage. A $50m patent moves a $200m minnow by 25% but a $20{,}000m giant by only 0.25%. So the same dollar value produces a tiny, easily-buried blip for a large firm and a huge, obvious one for a small firm — a preview of why the measure is noisier for big firms, and why dividing the abnormal return by market cap (to get back to dollars) is the right move.

# Market return path: a single common factor, mild drift, realistic daily vol.
mkt_ret = rng.normal(0.0003, 0.010, N_DAYS)

# Firm x day idiosyncratic shocks, then assemble returns via the market model.
# rows = firms, cols = days. Each firm has its own sigma.
eps = rng.normal(0.0, 1.0, (N_FIRMS, N_DAYS)) * sigma_i[:, None]
ret = alpha_i[:, None] + beta_i[:, None] * mkt_ret[None, :] + eps   # R_it = a + b R_mt + e

def inject_value(ret_matrix, patents_df, confound_sd=0.0, confound_frac=0.0, seed=7):
    # Add each patent's TRUE value into its firm's grant-window returns (as a % = $value/mktcap).
    #   confound_sd  : sd of a confounding window-news shock (e.g. an earnings surprise) in DAILY
    #                  return units, added to the SAME window. 0 => clean (Sections 2-4).
    #   confound_frac: fraction of patents whose window is contaminated. 0 => clean.
    # Returns a COPY of ret_matrix with the planted signal (+ optional confound) added, plus a
    # boolean array flagging which patents were contaminated.
    R = ret_matrix.copy()                         # never mutate the caller's array
    gen = np.random.default_rng(seed)             # local generator -> reproducible per call
    win_days = EVENT_WIN[1] - EVENT_WIN[0] + 1    # 3 days
    contaminated = np.zeros(len(patents_df), dtype=bool)
    for row in patents_df.itertuples():
        i, t0 = row.firm, row.grant_t
        cap = mktcap_m[i]
        pct = row.v_true_m / cap                  # dollars -> percent, total over the window
        per_day = pct / win_days                  # spread evenly across the 3 window days
        sl = slice(t0 + EVENT_WIN[0], t0 + EVENT_WIN[1] + 1)
        R[i, sl] += per_day
        if confound_sd > 0 and gen.random() < confound_frac:
            # one big surprise on the FIRST window day: mimics earnings landing in the window
            R[i, t0 + EVENT_WIN[0]] += gen.normal(0.0, confound_sd)
            contaminated[row.Index] = True
    return R, contaminated

ret_clean, _ = inject_value(ret, patents, confound_sd=0.0, confound_frac=0.0)
print("Daily return panel:", ret_clean.shape, "(firms x trading days)")
print(f"Grant window = {EVENT_WIN[1]-EVENT_WIN[0]+1} trading days; "
      f"estimation window = {EST_WINDOW} days with a {EST_GAP}-day gap.")

3. Reconstruct the value: a market-model event study, one patent at a time

Here is the Week-4 machine, applied to a patent grant. For each patent j of firm i with grant day t_0:

Estimation window. Take the EST_WINDOW trading days ending a EST_GAP-day gap before t_0. Regress the firm’s daily return on the market return there: R_{it} = \hat a_i + \hat b_i R_{mt} + \varepsilon_{it}. This learns the firm’s normal relationship to the market, untouched by the grant.
Predict the normal return in the grant window using \hat a_i, \hat b_i and the realized market returns those days. This is the counterfactual: what the stock would have done absent the patent.
Abnormal return AR_t = R_{it} - (\hat a_i + \hat b_i R_{mt}) on each window day. The cumulative abnormal return (CAR) is their sum over the 3-day window — the firm-specific surprise.
Dollarize. Multiply CAR (a percent) by market cap to convert to dollars: \widehat{v}_j = \text{CAR} \times \text{mktcap}_i. That is the KPSS-style value estimate.

We deliberately keep the estimation window before the grant (with a gap) so the patent’s own signal never leaks into the betas we estimate — the same hygiene Week 4 drilled.

def estimate_patent_value(ret_matrix, firm, grant_t, mktcap, mkt=mkt_ret,
                          est_window=EST_WINDOW, est_gap=EST_GAP, event_win=EVENT_WIN):
    # KPSS-style value for ONE patent via a market-model event study.
    # Returns (v_hat_m, car): estimated value in $millions and cumulative abnormal return over
    # the grant window. Returns (nan, nan) if the estimation window does not fit -- a real
    # selection issue: no usable pre-history -> no value.
    est_end   = grant_t - est_gap                  # last day of estimation window
    est_start = est_end - est_window
    if est_start < 0:
        return np.nan, np.nan                      # not enough pre-history: patent gets NO value
    # 1) Fit the market model on the estimation window (OLS via closed form, no statsmodels overhead).
    y = ret_matrix[firm, est_start:est_end]
    x = mkt[est_start:est_end]
    X = np.column_stack([np.ones_like(x), x])      # [intercept, market]
    ahat, bhat = np.linalg.lstsq(X, y, rcond=None)[0]
    # 2-3) Abnormal returns over the grant window, then cumulate.
    sl = slice(grant_t + event_win[0], grant_t + event_win[1] + 1)
    r_win   = ret_matrix[firm, sl]
    m_win   = mkt[sl]
    ar      = r_win - (ahat + bhat * m_win)        # AR_t = R_it - normal return
    car     = ar.sum()                             # cumulative abnormal return over the window
    # 4) Dollarize: CAR (percent) x market cap -> dollars.
    v_hat_m = car * mktcap
    return v_hat_m, car

# Demo on a single patent: a high-value one so the signal is visible above the noise.
demo = patents.sort_values("v_true_m", ascending=False).iloc[0]
v_hat, car = estimate_patent_value(ret_clean, int(demo.firm), int(demo.grant_t),
                                   mktcap_m[int(demo.firm)])
print(f"Patent {int(demo.patent)} (firm {int(demo.firm)}, mktcap ${mktcap_m[int(demo.firm)]:.0f}m):")
print(f"   true value   = ${demo.v_true_m:8.2f}m")
print(f"   window CAR   = {car:+.4f}  ({car*100:+.2f}%)")
print(f"   estimated v  = ${v_hat:8.2f}m   <- CAR x mktcap")

3.1 Run it on every patent and check the round trip

One patent is an anecdote. Run the event study on all of them, attach the estimate \widehat{v}_j back to the patent table, and ask the central question: does the reconstructed value track the planted truth? If the trick works, a scatter of \widehat{v}_j against v_j should hug the 45-degree line, and their correlation should be high. It will not be perfect — daily idiosyncratic noise \varepsilon sits on top of every window — but it should be unmistakably positive.

def value_all_patents(ret_matrix, patents_df):
    # Estimate KPSS-style value for every patent; return a NEW patents frame with v_hat_m, car.
    out = patents_df.copy()
    vh, cr = [], []
    for row in out.itertuples():
        v, c = estimate_patent_value(ret_matrix, row.firm, row.grant_t, mktcap_m[row.firm])
        vh.append(v); cr.append(c)
    out["v_hat_m"] = vh
    out["car"]     = cr
    return out

pat_clean = value_all_patents(ret_clean, patents)
n_valued  = pat_clean["v_hat_m"].notna().sum()
corr_lvl  = pat_clean[["v_true_m", "v_hat_m"]].corr().iloc[0, 1]
# correlation in logs too (skew-robust): only where v_hat is positive (a CAR can come out negative)
mask = (pat_clean["v_hat_m"] > 0) & pat_clean["v_hat_m"].notna()
corr_log = np.corrcoef(np.log(pat_clean.loc[mask, "v_true_m"]),
                       np.log(pat_clean.loc[mask, "v_hat_m"]))[0, 1]

fig, ax = plt.subplots(figsize=(6.5, 6))
ax.scatter(pat_clean["v_true_m"], pat_clean["v_hat_m"], s=14, alpha=0.5, color="steelblue")
lim = [0, pat_clean["v_true_m"].max() * 1.05]
ax.plot(lim, lim, "r--", lw=1, label="45 degrees (perfect recovery)")
ax.set_xlim(lim); ax.set_xlabel("true value $v_j$ ($m)")
ax.set_ylabel(r"estimated value $\hat{v}_j$ ($m)"); ax.legend()
ax.set_title("Reconstructed vs. planted patent value")
fig.tight_layout()

print(f"{n_valued} of {len(pat_clean)} patents got a value (rest lacked pre-history).")
print(f"corr(true, est)      level = {corr_lvl:.3f}")
print(f"corr(true, est)      log   = {corr_log:.3f}  (skew-robust)")

4. Aggregate to the firm-year innovation-value panel

The per-patent estimate is the raw material; the firm-year panel is the data object the rest of KPSS (and your capstone) is built on. Sum the estimated patent values within each firm and year. That gives one row per firm-year with the firm’s total innovation value — the flow of valuable innovation, in dollars, that year.

We sum \widehat{v}_j (clipping negative per-patent estimates to zero, since a granted patent cannot have negative value — a negative CAR there is just window noise). We also carry the true firm-year total alongside, so we can keep scoring the reconstruction at the panel level.

def firm_year_panel(pat_df):
    # Aggregate per-patent values to a firm-year panel. Negative est. patent values clipped to 0.
    d = pat_df.copy()
    d["v_hat_pos"] = d["v_hat_m"].clip(lower=0)        # granted patent: value >= 0
    g = (d.groupby(["firm", "year"], as_index=False)
           .agg(n_patents=("patent", "size"),
                inno_value_hat=("v_hat_pos", "sum"),
                inno_value_true=("v_true_m", "sum")))
    return g

fy_clean = firm_year_panel(pat_clean)
print(f"Firm-year panel: {fy_clean.shape[0]} rows (firm x year).")
print("\nTop 8 firm-years by ESTIMATED innovation value ($m):")
top = fy_clean.sort_values("inno_value_hat", ascending=False).head(8)
print(top.to_string(index=False,
      formatters={"inno_value_hat": "{:.1f}".format, "inno_value_true": "{:.1f}".format}))

panel_corr = fy_clean[["inno_value_true", "inno_value_hat"]].corr().iloc[0, 1]
print(f"\ncorr(true firm-year value, estimated) = {panel_corr:.3f}  -- aggregation averages out noise.")

5. Validate the measure: does it predict a future outcome?

A constructed measure earns trust by predicting things it should predict. KPSS’s strongest validation is that a patent the market valued highly at grant is followed by higher future profitability for the owning firm. We can reproduce the spirit of that here because we control the truth: we plant a firm’s next-year profitability as a noisy function of its innovation quality q_firm — the same latent trait that drove its patent values back in Section 1.1. So a genuine link exists in the world; the question is whether the estimated measure, built from 3-day stock blips, can see it.

We regress next-year profitability on this year’s estimated innovation value (log, to tame the skew), with firm-size and year controls. The one-line spec, in our Week-1 discipline:

outcome = firm next-year profitability · key regressor = log estimated innovation value · controls = log market cap · fixed effects = year · sample = firm-years with at least one valued patent · identifying assumption — this is predictive, not causal: we ask only whether the measure forecasts the outcome, exactly as KPSS’s validation does.

# Plant next-year profitability: driven by innovation quality + size, plus noise.
# (One row per firm-year; "next year" profitability attached to the year the innovation was measured.)
def plant_future_profit(fyc, seed=123):
    # Fixed-seed profit draw so the SAME firm-year always gets the SAME outcome, regardless of
    # which pipeline run asks for it -> the Section-6 sweep isolates the CONFOUND, not run noise.
    d = fyc.copy()
    d["log_mktcap"] = np.log(d["firm"].map(dict(zip(firms["firm"], firms["mktcap_m"]))))
    d["q_firm"]     = d["firm"].map(dict(zip(firms["firm"], firms["q_firm"])))
    gen = np.random.default_rng(seed)                 # local generator -> reproducible
    # future profitability: real signal from q_firm (the latent quality), with size + noise.
    d["future_profit"] = (0.05
                          + 0.020 * d["q_firm"]
                          + 0.010 * (d["log_mktcap"] - d["log_mktcap"].mean())
                          + gen.normal(0.0, 0.015, len(d)))
    return d

fy = plant_future_profit(fy_clean)

# Regress future profit on log estimated innovation value (+ controls), predictive spec.
reg = fy[fy["inno_value_hat"] > 0].copy()
reg["log_inno_hat"] = np.log(reg["inno_value_hat"])
m_hat = smf.ols("future_profit ~ log_inno_hat + log_mktcap + C(year)", data=reg).fit()

# Benchmark: the SAME regression using the TRUE value -- the best the measure could possibly do.
reg["log_inno_true"] = np.log(reg["inno_value_true"])
m_true = smf.ols("future_profit ~ log_inno_true + log_mktcap + C(year)", data=reg).fit()

print("VALIDATION -- next-year profitability on innovation value (predictive):")
print(f"   estimated value:  coef = {m_hat.params['log_inno_hat']:+.4f}  "
      f"t = {m_hat.tvalues['log_inno_hat']:+.2f}  p = {m_hat.pvalues['log_inno_hat']:.4f}")
print(f"   TRUE value (bm):  coef = {m_true.params['log_inno_true']:+.4f}  "
      f"t = {m_true.tvalues['log_inno_true']:+.2f}  p = {m_true.pvalues['log_inno_true']:.4f}")
print("\nThe estimated measure -- built from 3-day stock blips -- forecasts future profits, just like KPSS.")

6. Break it: confounding window news adds noise — and eats the validation

The whole design assumes the only value-relevant news in the grant window is the patent. But firms release earnings, announce deals, and get analyst revisions on ordinary days too. If such news lands in the window, the abnormal return mixes the patent’s value with the other news, and the measure is contaminated (Reader’s Guide §6, the second vulnerability).

We now turn the confound knob on: a fraction of grant windows get hit with a large earnings-surprise shock on the first window day. Watch two things degrade together: the recovery correlation (estimate vs. truth) and the validation t-statistic (does the measure still predict future profit?). The short window and aggregation are the defenses KPSS leans on — but they only go so far.

def run_pipeline(confound_sd, confound_frac, seed=7):
    # Full pipeline under a given confound setting -> recovery corr + validation t-stat.
    R, contam = inject_value(ret, patents, confound_sd=confound_sd,
                             confound_frac=confound_frac, seed=seed)
    pat = value_all_patents(R, patents)
    m = (pat["v_hat_m"] > 0) & pat["v_hat_m"].notna()
    rec_corr = np.corrcoef(np.log(pat.loc[m, "v_true_m"]),
                           np.log(pat.loc[m, "v_hat_m"]))[0, 1]
    fyc = plant_future_profit(firm_year_panel(pat))   # same fixed-seed outcome every run
    r = fyc[fyc["inno_value_hat"] > 0].copy()
    r["log_inno_hat"] = np.log(r["inno_value_hat"])
    mm = smf.ols("future_profit ~ log_inno_hat + log_mktcap + C(year)", data=r).fit()
    return rec_corr, mm.tvalues["log_inno_hat"], contam.mean()

# Sweep the share of contaminated windows, holding the surprise size fixed and large.
# We AVERAGE each setting over several confound seeds: WHICH patents get hit is itself random,
# so one draw is noisy -- averaging isolates the effect of the contamination SHARE (and quietly
# re-teaches the lesson that averaging over many events tames idiosyncratic luck).
CONF_SD = 0.06     # 6% one-day earnings surprise -- big relative to the patent's window signal
fracs = [0.0, 0.15, 0.30, 0.50, 0.75]
SEEDS = [7, 11, 19, 23, 31]
rows = []
for f in fracs:
    rcs = [run_pipeline(CONF_SD, f, seed=s) for s in SEEDS]
    rows.append({"contam_frac": f,
                 "recovery_corr": np.mean([x[0] for x in rcs]),
                 "validation_t":  np.mean([x[1] for x in rcs])})
deg = pd.DataFrame(rows)
print("Confounding window news degrades the measure (surprise sd = {:.0%}):".format(CONF_SD))
print(deg.to_string(index=False,
      formatters={"contam_frac": "{:.0%}".format,
                  "recovery_corr": "{:+.3f}".format, "validation_t": "{:+.2f}".format}))

6.1 The degradation, in one picture

Plot the recovery correlation and the validation t-statistic against the share of contaminated windows. Both slope down: as more grant windows are polluted by earnings news, the event study confuses that news for patent value, the reconstruction drifts from the truth, and the measure’s power to forecast future profitability erodes. This is the §6 vulnerability made into a number on your own screen — and it is exactly why KPSS use a tight window (less time for confounds to land) and aggregate over many patents (idiosyncratic confounds partly wash out). Systematic contamination, though, would not wash out — a caveat to carry into your capstone.

fig, ax1 = plt.subplots(figsize=(8.5, 5))
x = deg["contam_frac"] * 100
ax1.plot(x, deg["recovery_corr"], "o-", color="steelblue", label="recovery corr (left)")
ax1.set_xlabel("% of grant windows contaminated by earnings news")
ax1.set_ylabel("recovery correlation (log)", color="steelblue")
ax1.tick_params(axis="y", labelcolor="steelblue")

ax2 = ax1.twinx()
ax2.plot(x, deg["validation_t"], "s--", color="darkorange", label="validation t (right)")
ax2.set_ylabel("validation t-stat (future profit)", color="darkorange")
ax2.tick_params(axis="y", labelcolor="darkorange")
ax2.grid(False)
ax1.set_title("Confounding window news degrades both recovery AND prediction")
fig.tight_layout()

print(f"Clean (0% contam):   recovery corr = {deg.iloc[0]['recovery_corr']:+.3f}, "
      f"validation t = {deg.iloc[0]['validation_t']:+.2f}")
print(f"Polluted (75%):      recovery corr = {deg.iloc[-1]['recovery_corr']:+.3f}, "
      f"validation t = {deg.iloc[-1]['validation_t']:+.2f}")

7. The real path (do not run): PatentsView + the public KPSS dataset

Everything above ran on synthetic data so it works offline and we know the truth. On real data you would stitch three sources, exactly as the Reader’s Guide §3 describes. The cell below is illustrative and intentionally not executed (RUN_REAL = False) — it shows the pattern, not fabricated numbers.

Patents + grant dates come from USPTO PatentsView — free and public, but heavy. PatentsView migrated to the USPTO Open Data Portal (ODP) on March 20, 2026; the current home is https://api.uspto.gov with an X-API-KEY header (key from USPTO_ODP_API_KEY). The ODP bulk PatentsView tables (TSV/parquet) serve grant dates, assignees, and technology classes; the targeted-query patent-search endpoint path is being finalized post-cutover — see the uspto-patentsview data card and nb7.2 for the verified call.
Daily stock returns come from CRSP — licensed; it stays read-only on GMU infrastructure (Hopper / WRDS Cloud) per Conventions §5. You pin a snapshot date and never copy it to your laptop.
The public KPSS patent-value dataset is the shortcut: KPSS released their patent-level dollar-value estimates publicly (and keep extending them), so most users load the measure off the shelf rather than rebuild it. That is the “dataset as infrastructure” point of §5.

RUN_REAL = False   # <-- intentionally OFF. This cell documents the real pipeline; it does not run it.

if RUN_REAL:
    import os, requests

    # (1) USPTO PatentsView via the USPTO Open Data Portal (ODP).
    #     PatentsView migrated to ODP on March 20, 2026; the legacy api.patentsview.org host
    #     was discontinued May 2025, and the interim search.patentsview.org/api/v1 PatentSearch
    #     API was paused at the ODP cutover. The current home is api.uspto.gov, the header is
    #     X-API-KEY (lowercase), and the key is read from the env var USPTO_ODP_API_KEY.
    #     Docs: https://data.uspto.gov/apis/getting-started
    #     Transition guide: https://data.uspto.gov/support/transition-guide/patentsview
    key  = os.environ["USPTO_ODP_API_KEY"]   # env only; CONVENTIONS §5
    base = "https://api.uspto.gov"           # pin in config.py
    headers = {"X-API-KEY": key}             # lowercase header name per ODP docs
    # [CHECK] the exact post-cutover patent-search endpoint path under ${base}/api/v1/...
    # against data.uspto.gov when you run; nb7.2 holds the verified, current call. For
    # panel-scale work, prefer the ODP "PatentsView Bulk Downloads" TSV/parquet tables
    # (flat patents/inventors/assignees/citations) over millions of API calls.
    # resp = requests.get(f"{base}/api/v1/patent/applications",
    #                     headers=headers,
    #                     params={"q": "patentDate:[2015-01-01 TO 2015-12-31]"},
    #                     timeout=60)
    # patents = resp.json()  # shape: paginated; loop pages, respect rate limits.

    # (2) CRSP daily returns -- LICENSED, read-only on GMU infra (Hopper/WRDS). Pin a snapshot.
    #     CRSP_SNAPSHOT = "2025-12-31"   # record the exact extract date in your data card.
    #     import wrds; db = wrds.Connection(wrds_username="<gmu_user>")
    #     dsf = db.raw_sql("select permno, date, ret, prc, shrout from crsp.dsf "
    #                      "where date between '2015-01-01' and '2015-12-31'")
    #     # mktcap = abs(prc) * shrout; match PatentsView assignee -> CRSP permno (name match).

    # (3) Public KPSS patent-value dataset -- load the measure OFF THE SHELF.
    #     Hosted at: https://github.com/KPSS2017/ (patent-level value estimates, periodically updated).
    #     kpss = pd.read_csv("KPSS_2020_public.csv")   # patent_num, issue_date, xi_nominal, xi_real, ...
    #     # then aggregate to firm-year exactly as in Section 4 above.
    pass

print("Real-path cell is documentation only (RUN_REAL = False). No network calls were made.")

Your Turn — make the measure degrade with your own hands

You now have a God’s-eye patent-value lab. Three extensions, each a small change to the machinery above.

1. Vary the window length and watch the trade-off. Change EVENT_WIN to a 1-day window (0, 0), then a 5-day window (0, 4), re-run Sections 3–3.1, and record the recovery correlation each time. You should find a tension: a shorter window catches less confounding news (good) but also captures less of the patent’s reaction if the market is slow to impound it (it under-recovers); a longer window catches more of the reaction but lets in more confounds. There is no free lunch — KPSS’s 3-day choice is a compromise. Plot recovery corr vs. window length.

2. Turn up the noise until the measure dies. In Section 3, the firms’ idiosyncratic vol sigma_i sets the noise floor every window fights through. Re-draw sigma_i from a larger range (say rng.uniform(0.04, 0.07)), rebuild returns, and re-run the recovery scatter and the validation regression. Predict first: as noise rises, the per-patent CAR is buried, recovery corr falls, and the validation t-stat shrinks toward insignificance — even though aggregation to firm-year still helps. Where does the validation stop being significant?

3. Make the confound systematic instead of random. Section 6 contaminated windows at random, so aggregation washed it out. Now make earnings news land preferentially on the highest-value patents (e.g. contaminate a window with probability rising in v_true_m). This is the dangerous case §6 warns about: the bias correlates with the signal and does not average away. Re-run the sweep and compare the degradation curve to the random-confound case from Section 6.

Check your understanding. (a) In Section 3, why do we put a EST_GAP between the estimation window and the grant day — what would leak if the gap were zero? (b) In Section 4, why is the firm-year recovery correlation higher than the per-patent one? (c) In Section 6, the recovery corr fell but stayed positive even at 75% contamination — what feature of the design kept the measure from collapsing entirely, and when would that defense fail?