Ch 7.5 — The Identification Memo

There is one page in your capstone that matters more than any other, and it is not the abstract, not the headline table, not the conclusion. It is a page you write before you run the regression that the paper is about — a page that, if you write it honestly, will tell you in advance whether the project is worth finishing. It is called the identification memo, and it has exactly two parts: one paragraph stating your identifying assumption, and one table — the threats-and-responses table — listing every reason a skeptic might not believe you and what you intend to do about each one.

This chapter teaches you to write that page. It is short on new machinery, because there is none: you already have every design (Weeks 3–4), you already know how to read a paper’s identification section against a checklist (Week 5), and you already filed a pre-analysis plan in Chapter 7.3 that pins your primary specification. The identification memo is the bridge between the PAP and the paper. It is where you stop saying “I will run a difference-in-differences” and start saying “my estimate is the causal effect as long as this one thing is true, and here is how I will convince you it is.” Reveal-the-trick, one last time, but now the trick is yours and the audience is a referee who has read a thousand of these and is not in a charitable mood.

Here is the move the whole chapter turns on, so you have it before the details. The identifying assumption is the single sentence on which your entire causal claim rests. Everything else in the paper — the data work, the controls, the robustness checks — is in service of making that one sentence believable. If the sentence is false, no amount of clustering or winsorizing saves you. If you cannot state the sentence, you do not yet have a design; you have a regression and a hope. The memo forces you to write the sentence down, in public, before the data can seduce you into believing whatever the first significant coefficient happens to say.

1. The identifying-assumption sentence

Start with the sentence, because everything else hangs from it. The form is fixed, and you should write it in this template literally, filling the three blanks:

Our estimate of [the effect] is causal / credible as long as [the assumption], which we defend by [the design feature or evidence].

Three blanks, three jobs. The first names what you are estimating — not the regression, the effect. “The effect of a state fair-lending regulation on the mortgage-denial gap between minority and white applicants,” not “the coefficient on the treatment dummy.” The second names the condition under which the estimate equals that effect — and here is the rule that separates a real memo from a hand-wave: name the specific threat, not the word “endogeneity.” This is CONVENTIONS §4 stated for your own paper. “As long as there is no endogeneity” is not an identifying assumption; it is an admission you have not thought about which endogeneity. “As long as treated and control states would have followed parallel denial-gap trends absent the regulation” is an identifying assumption, because it names exactly the counterfactual that has to hold and exactly the way it could fail. The third blank names the design or evidence that makes the second blank believable — fixed effects that absorb the obvious confounder, a placebo test that would have caught the obvious failure, an institutional fact that rules out the obvious alternative story.

Notice the asymmetry baked into “causal / credible.” Some designs let you say causal with a fairly straight face when the assumption holds — a clean RD at a cutoff nobody can game, a randomized instrument. Most observational designs earn the weaker word credible: the assumption is not testable, you have argued for it as hard as you can, and an honest reader grants you a conditional belief, not a proof. Choose the word that matches how strong your assumption really is. Overclaiming in this sentence is the tell of an amateur; a referee who reads “causal” where “credible, conditional on parallel trends” was warranted will distrust everything that follows.

A worked contrast makes the discipline concrete. Here is Maya, drafting her sentence three times, getting worse to better:

Draft 1 (useless): “We estimate the causal effect of fair-lending regulation on denial rates, controlling for endogeneity.” No. No threat is named; “controlling for endogeneity” is the exact phrase the Conventions forbid. This sentence could front any paper on any topic. It commits to nothing, so it can be attacked from every side at once.
Draft 2 (better): “Our estimate of the regulation’s effect on the minority–white denial gap is credible as long as treated and control states had parallel trends.” Closer — it names the design’s assumption — but it still does the third blank no work. Defended how? A reader cannot tell whether parallel trends is asserted or earned.
Draft 3 (a real memo sentence): “Our estimate of the effect of state fair-lending examinations on the county-level minority–white mortgage-denial gap is credible as long as, absent the examinations, adopting and never-adopting states’ denial gaps would have moved in parallel — which we defend with a flat pre-trend in the event-study leads, a Callaway–Sant’Anna comparison against never- and not-yet-treated states only, and the institutional fact that adoption timing was driven by legislative calendars unrelated to local lending conditions.” That sentence is a contract. It tells the referee exactly what to attack (the parallel-trends counterfactual), exactly what evidence you are betting on (flat leads, clean controls, exogenous timing), and exactly what would falsify you (tilted leads, or a story tying adoption timing to local lending).

Draft 3 is long, and that is correct: the identifying-assumption sentence is allowed to be the longest sentence in your paper, because it is the most important one. The paragraph that contains it is the spine of the whole project.

2. The standard menu of threats, by design

You are not inventing threats from scratch. Each design from Weeks 3–4 comes with a standard menu of named failures — the ones a referee will reach for first, because they are the ones that bite that design. Knowing the menu is half the battle: it tells you, before you have written a word, which alternative explanations you must pre-empt. Here is the menu, design by design, each threat stated as the specific way the identifying assumption fails. These are the recurring villains of Weeks 3–4; the memo is where you finally name them out loud against your own data.

OLS on observables (selection-on-observables / conditional independence). The assumption is that, conditional on the controls, treatment is as-good-as-random — there is no confounder you left out. The standard threats are omitted-variable bias (a variable that drives both treatment and outcome and is not in your controls — Week 2’s ghost, and the reason “we control for X” is a claim, not a spell) and selection (units choose into treatment on something you cannot see). This is the weakest design on the credibility ladder precisely because its assumption — “I have measured every confounder” — is the hardest to defend and the easiest to attack: a referee needs only to name one plausible omitted variable to wound you.

Instrumental variables (exclusion / weak instrument). Two assumptions, and Week 3 drilled their asymmetry: relevance (the instrument moves the treatment) is testable — you report the first-stage F — and exclusion (the instrument affects the outcome only through the treatment) is arguable — there is no test, no p-value, only an argument from institutions. The two named threats are therefore the weak-instrument problem (first-stage F too small, so 2SLS drifts back toward OLS bias and its confidence interval lies about its own width) and the exclusion violation (the instrument has its own back-channel to the outcome). When a referee attacks an IV paper, they almost always attack exclusion, because it is the assumption the data can never defend.

Difference-in-differences (parallel trends / anticipation). The assumption is parallel trends: absent treatment, treated and control units’ outcomes would have moved together. The named threats are a pre-existing differential trend (the treated group was already diverging before treatment, so a slope gets mislabeled an effect), anticipation (units respond before the official treatment date — they saw it coming — so the “pre” period is already contaminated), and, in staggered designs, the forbidden comparison of Chapter 4.2 (already-treated units used as controls, injecting the negative-weights bias when effects are dynamic). Remember the one sentence underlined three times in Chapter 4.1: parallel trends is a claim about a counterfactual, so a pre-trend can refute it but a flat pre-trend can never confirm it.

Regression discontinuity (manipulation / compound treatment). The assumption is continuity of potential outcomes at the cutoff. The named threats are manipulation/sorting (units precisely control which side of the cutoff they land on, so the just-above and just-below groups are no longer comparable — the McCrary fingerprint is a jump in the density), covariate imbalance at the cutoff (something predetermined jumps where only treatment should), and the most insidious, compound treatment (more than one thing switches at the same threshold, so your jump is the bundled effect of all of them, not the clean treatment you wanted). Manipulation and imbalance leave fingerprints you can test for; compound treatment passes every statistical test and must be ruled out by institutional argument alone.

Synthetic control (donor pool / anticipation). The assumption is that a weighted average of untreated donor units reproduces the treated unit’s untreated path. The named threats are a bad donor pool (no convex combination of donors actually matches the treated unit’s pre-period, or the donors are themselves contaminated by spillovers from the treatment), anticipation (the treated unit reacts before the event, fouling the pre-period fit), and interpolation/overfitting (the weights fit pre-period noise and so mispredict the counterfactual). With an effective sample size of one, inference is not a t-test but a placebo permutation — assign the fake treatment to each donor in turn and ask how extreme the real effect looks against that distribution.

You will use exactly one of these designs for your primary specification — the one you committed to in your PAP. Pull its row off this menu and you have your starting list of threats. Then add any that are specific to your data (a survivorship filter, a look-ahead leak from Chapter 7.4, a measurement-error problem from Week 1). The menu guarantees you do not forget the obvious ones; your own data supplies the rest.

3. The threats-and-responses table

Now build the table. It is the operational heart of the memo, and it has exactly four columns. Write it as a literal Markdown table in your paper, because its rigid shape is what forces honesty: every threat must have a response, and every response must end in a residual concern, so you cannot quietly drop the threats you have no answer for.

Column	What goes in it
Threat	The named failure of the identifying assumption — from the menu above, stated in one phrase. Not “endogeneity.”
Why it’s plausible	One sentence on the concrete story under which this threat is real in your setting. If you cannot tell a plausible story, the threat is a strawman and does not belong in the table.
What we do about it	The specific test, design choice, or robustness check that addresses it — with the expected result that would reassure a reader.
Residual concern	What still worries you after you have done everything in column 3. The honest leftover. If this column is blank, you are not being honest.

The fourth column is the one students want to leave empty, and it is the one a good mentor reads first. A threats table whose Residual-concern column is all “none” is not a stronger paper — it is a less credible one, because every real design has residual concerns and a reader who sees none assumes you have not looked hard enough. The point of the table is not to pretend the threats are vanquished; it is to show you have seen them, engaged them, and bounded what remains. That is what intellectual honesty looks like on a page, and it is the single trait that most distinguishes a paper a referee trusts from one they do not.

A practical note on ordering: put your rows in descending order of danger. The first row should be the threat that, if true, would most cleanly kill the paper — the one a seminar audience asks about in the first ninety seconds. Burying your most serious threat in row four reads as evasion; leading with it reads as confidence. You are signaling that you are not afraid of your own hardest question.

4. Threats you can test vs. threats you can only argue

Here is the distinction that organizes the entire third and fourth columns of your table, and it is the most important conceptual point in the chapter. Some threats leave fingerprints in the data, and some do not. The ones that do, you test. The ones that do not, you argue. Confusing the two — running a test for something that is fundamentally untestable, or arguing your way around something you could simply have checked — is the most common error in a weak identification memo.

Threats you can test have a checkable empirical signature, so column 3 is a statistic and column 4 is “what the test can and cannot rule out.” The canonical four, one per design:

Pre-trends (DiD). Estimate the event study and read the leads. Flat, insignificant leads are consistent with parallel trends. But remember the asymmetry: a flat pre-trend cannot confirm the counterfactual — it can only fail to refute it — so even a clean test leaves a residual concern, and you must say so.
First-stage F (IV). Report it. An F comfortably above the modern weak-instrument thresholds (and an Anderson–Rubin interval that does not blow up) reassures on relevance. It says nothing about exclusion.
Balance (OLS / RD). A balance table — covariate means by treatment status, or covariates run as outcomes in an RD — shows whether the groups look alike on observables. Balance on observables is reassuring but, by construction, silent on unobservables: the very confounder you fear is the one not in the table.
McCrary density (RD). Test for a jump in the density of the running variable at the cutoff. A smooth density is consistent with no manipulation. A jump is often fatal.

Threats you can only argue have no fingerprint — the data are logically incapable of revealing them — so column 3 is an institutional fact or an economic argument, and column 4 is “this rests on a claim the data cannot confirm.” The canonical members:

The exclusion restriction (IV). Untestable, full stop. You argue it from how the instrument works, and you defend it against every alternative path a critic proposes. Week 3’s slogan: relevance is testable, exclusion is arguable.
Compound treatment (RD). Passes every statistical test and still breaks the design. Only institutional knowledge — “I have read the rule and only my treatment switches at this cutoff” — can rule it out.
The parallel-trends counterfactual (DiD), beyond the pre-period. The pre-trend test checks the periods you observe; whether trends would have stayed parallel in the post-period absent treatment is a counterfactual no test can reach. You argue it from the exogeneity of treatment timing.
Anticipation, when it occurs silently. If units reacted before any date you can observe, no lead coefficient catches it; you argue from institutions whether the treatment was anticipable.

The discipline is simple to state and hard to live by: for every threat in your table, decide first whether it is testable or arguable, and put the right kind of thing in column 3. A memo that runs a clever test against an untestable threat (e.g., claiming a balance table “shows the exclusion restriction holds” — it cannot) advertises that the author does not understand their own design. A memo that argues its way past a threat it could have simply tested looks lazy. Match the response to the nature of the threat, every time.

5. Intellectual honesty: state the residual, don’t oversell

The temptation, as the data come in and a coefficient turns up significant with three stars, is to write the memo backward — to start from the result you want and reverse-engineer an identifying assumption that delivers it. This is the spec-search and the p-hacking you were warned about in Week 1, wearing the costume of a research design. The PAP you filed in Chapter 7.3 exists precisely to stop this: it pinned your primary specification before you peeked, and your identification memo must defend that spec, not whichever one the data later flatters. The memo and the PAP are written in the same spirit — commit first, then test — and a memo that quietly drifts to a different design than the one you registered has broken the contract.

Three honesty rules, which double as the things a grader looks for:

State the residual concern, every time. The Residual-concern column is mandatory and must be substantive. “We cannot rule out a contemporaneous policy shock that hit treated states in the same year” is worth more to your credibility than a triumphant “no concerns remain.” Referees reward candor because candor is rare and easy to verify: they will think of the residual concern whether or not you write it, so writing it first robs the attack of its force.
Match the verb to the evidence. If your assumption is untestable and merely argued, your sentence says credible, not causal, and your prose says “consistent with” and “we cannot reject,” never “proves” or “establishes.” Calibrate every claim to the strength of its support. A two-star result on a design with a serious untestable threat is “suggestive evidence,” not “we show that.”
Do not bury the hard question. Lead the table with your most dangerous threat. The instinct to hide it is exactly backward: the hidden threat is the one the referee finds anyway, and finding it themselves is worse for you than your having flagged it.

The deep point is that the identification memo is not a defensive document you write to survive review. It is a thinking tool you write to find out whether your project is real. If, while filling the table, you discover a threat for which column 3 is empty and column 4 is “this could be the entire effect,” you have learned something invaluable before sinking Week 8 into a paper that cannot be saved — and the right response is to change the design, not to soften the language. The memo’s first reader is you, and its first job is to talk you out of a project that will not survive contact with a skeptic.

6. A worked identification memo: Maya’s HMDA fair-lending DiD

Let us write the whole thing, end to end, for the project Maya has carried since Week 1 and built in Lab 4: a staggered difference-in-differences on HMDA, asking whether state fair-lending examination programs reduced the minority–white gap in mortgage-denial rates. This is the exact design of Chapter 4.2 (staggered adoption, dynamic effects, Callaway–Sant’Anna against clean controls), pointed at the dataset Prof. Gao and his coauthor used to ask whether same-sex couples were treated differently when they applied to borrow.¹

The specification, in CONVENTIONS §4 form. Outcome: county-year minority–white mortgage-denial gap (denial rate for minority applicants minus denial rate for white applicants), from the HMDA LAR. Treatment: indicator for whether the applicant’s state had an active fair-lending examination program in that year (staggered adoption, cohort G_i = year the state’s program took effect; never-adopters as clean controls). Controls: county-year applicant-composition covariates (income, loan-to-value, requested-loan-amount distributions) entering via covariate-conditional parallel trends. Fixed effects: county and year, absorbed inside the group-time estimator. Clustering: by state (the level at which treatment varies). Sample: HMDA LAR, all U.S. counties, years spanning the adoption waves, pinned to a fixed data vintage. Identifying assumption (one sentence): absent the examination programs, adopting and never-adopting states’ county-level minority–white denial gaps would have followed parallel trends.

The identifying-assumption paragraph. Our estimate of the effect of state fair-lending examination programs on the county-level minority–white mortgage-denial gap is credible as long as, absent the programs, the denial gaps in adopting and never-adopting states would have evolved in parallel — a per-cohort parallel-trends assumption that we defend three ways: with an event study whose pre-treatment leads are flat and statistically indistinguishable from zero; with a Callaway–Sant’Anna estimator that forms only clean comparisons against never- and not-yet-treated states, never the already-treated states that contaminate pooled TWFE under dynamic effects; and with the institutional fact that the timing of each state’s program was set by its legislative and budget calendar, which we argue is unrelated to contemporaneous shocks to local mortgage markets. We report a single overall ATT and an event-time profile, with state-clustered inference and a placebo check that randomly reassigns adoption years. That paragraph is the contract. Everything in the paper defends it.

The threats-and-responses table.

Threat	Why it’s plausible	What we do about it	Residual concern
Differential pre-trend (parallel trends already failing before treatment)	States that chose to adopt examinations may have been on a different denial-gap trajectory — e.g., already-improving states legislate improvement.	Testable. Event-study leads, normalized at k=-1; report and plot them. Reassuring result: leads flat and insignificant.	A flat pre-trend cannot confirm the post-period counterfactual; it only fails to refute it (Ch 4.1).
Forbidden comparison / negative weights (staggered TWFE using already-treated states as controls)	Adoption is staggered (Ch 4.2) and the effect plausibly builds as examiners ramp up, the exact recipe for the bias.	Design choice. Use Callaway–Sant’Anna group-time ATTs against never- and not-yet-treated controls only; run a Goodman–Bacon decomposition to show how much weight pooled TWFE put on forbidden comparisons.	Clean estimator needs a sufficient never-treated pool; if few states never adopt, late cohorts’ effects are thinly identified.
Anticipation (lenders tighten or loosen before the program’s official start)	Programs are debated publicly for months; lenders could adjust ahead of the effective date, contaminating the “pre” period.	Partly testable. Inspect leads in the quarters just before G_i; optionally exclude the year before adoption and re-estimate.	Anticipation that began before any observed lead is invisible to the leads; rests on the institutional-timing argument.
Compound policy shock (another state policy moves with examination adoption)	States that strengthen fair-lending examinations may simultaneously change CRA enforcement, licensing, or down-payment-assistance rules.	Arguable. Document each adopting state’s contemporaneous lending-policy changes from legislative records; argue no co-timed policy plausibly moves the denial gap.	No statistical test can detect a perfectly co-timed policy; this rests on the completeness of the institutional review.
Outcome is a composite measure (the denial gap mixes applicant composition with treatment)	The raw gap can move if the mix of who applies changes, not because anyone was treated differently — a Week 1 measurement issue.	Testable / design. Covariate-conditional (doubly-robust) DiD on income/LTV/loan-size distributions; show the gap moves for composition-matched applicants.	Conditioning only on observed applicant characteristics; unobserved creditworthiness differences could still shift the gap.
Few-treated-cluster inference (state-clustered SEs strained when adopters are few)	Cluster-robust SEs need many treated clusters (Ch 4.1); a handful of adopting states strains the t-approximation and the BDM correction.	Inference choice. Report state-clustered SEs and a placebo permutation test (randomly reassign adoption years, locate the real ATT in that distribution).	With genuinely few adopters, the uncertainty is large and hard to pin precisely; the placebo distribution is the honest summary.

Read that table the way a referee would. The first row is the hardest question — parallel trends — and it leads, as it should. Every row’s column 3 is the right kind of response: a test where the threat is testable (pre-trends, composition, placebo) and an institutional argument where it is not (compound policy). And no Residual-concern cell is empty, because none honestly could be. This is a memo that does not oversell: the sentence says credible, not causal, and every leftover worry is on the page where a reader can see Maya already saw it. A paper built on this spine can be attacked — every paper can — but it cannot be ambushed.

7. The bridge to Week 8

You have now closed the design half of your project. Chapters 7.1–7.4 took you from a question to a feasible dataset to a filed pre-analysis plan to a clean, documented analysis dataset; Chapter 7.5 distilled the design into the one page the whole paper defends. This memo is what Assessment 7 grades — the PAP and the identification memo together, on the research-design rubric — and it is the spine of the Week-8 paper.

Here is how the spine becomes the paper. The identifying-assumption paragraph becomes your empirical strategy section, nearly verbatim. The threats-and-responses table becomes the skeleton of your robustness section: Chapter 8.2 (execution and robustness) is, almost literally, the instruction “go run column 3 for every row of your table” — the placebos, the alternative standard errors, the bandwidth and sample perturbations that turn each promised defense into a real result. Chapter 8.3 (writing the paper) will have you turn the memo’s prose into an introduction that promises exactly what the design can deliver and no more — the calibrated-verb discipline from §5, applied to the abstract. And Chapter 8.4 (peer review) will hand your memo to another student, who will read it the way you learned to read Fama–French in Week 5: tables first, identifying assumption second, prose last and skeptically — and who will attack the row you hoped to bury. Write the memo so that there is no such row.

The thread that runs from Week 5’s reading discipline through this memo to Week 8’s defense is a single idea, stated once more: the skeptic who reads a stranger’s table and the author who builds an honest one are the same person. You spent Weeks 5–6 learning to find the load-bearing assumption in someone else’s paper. The identification memo is you doing it to yourself, on purpose, before anyone else gets the chance.

Your Turn

Open nb7.5 — “First-look regressions (frozen until the PAP is filed).” This notebook runs the first estimate of your primary specification — the headline number your memo defends. It is frozen for a reason that is the whole ethic of this week: you do not run it until your pre-analysis plan (Chapter 7.3) and this identification memo are filed as a tagged commit (Lab 7). The freeze is the technological enforcement of §5’s honesty rules: with the spec and the threats committed in advance, the first-look regression can inform you but can no longer seduce you into rewriting the design to fit the result. Run it, compare the point estimate to what your PAP predicted, and — before you touch anything — re-read your threats table and ask which row the first look just made more or less worrying.

Three reflection prompts to write up alongside the notebook:

Find your most dangerous threat. Of the rows in your own threats table, which one, if it were true, would most cleanly kill your paper? Is it testable or only arguable? If it is arguable, write the two strongest sentences of institutional or economic argument you have — and be honest about whether they are enough.
Audit your verb. Reread your identifying-assumption sentence. Did you write causal or credible? Defend the choice against the actual strength of your evidence. If you wrote causal, name the testable fingerprint that earns it; if you cannot, downgrade the verb and adjust every claim in the memo to match.
The empty cell test. Is any Residual-concern cell in your table blank or “none”? If so, you have either a genuinely bulletproof threat (rare — defend it) or a threat you have not looked at hard enough (likely — go look). Conversely, is there a threat whose residual concern is “this could be the entire effect”? If so, the memo just did its most important job: it found the project-killer before Week 8 did. Decide now whether to redesign or to proceed with the limitation stated plainly.

Sun, H., & Gao, L. (2019). Lending practices to same-sex borrowers. Proceedings of the National Academy of Sciences, 116(19), 9293–9302.↩︎

# Ch 7.5 — The Identification Memo There is one page in your capstone that matters more than any other, and it is not the abstract, not the headline table, not the conclusion. It is a page you write *before* you run the regression that the paper is about — a page that, if you write it honestly, will tell you in advance whether the project is worth finishing. It is called the **identification memo**, and it has exactly two parts: one paragraph stating your **identifying assumption**, and one table — the **threats-and-responses table** — listing every reason a skeptic might not believe you and what you intend to do about each one. This chapter teaches you to write that page. It is short on new machinery, because there is none: you already have every design (Weeks 3–4), you already know how to read a paper's identification section against a checklist (Week 5), and you already filed a pre-analysis plan in Chapter 7.3 that pins your primary specification. The identification memo is the bridge between the PAP and the paper. It is where you stop saying "I will run a difference-in-differences" and start saying "my estimate is the causal effect *as long as* this one thing is true, and here is how I will convince you it is." Reveal-the-trick, one last time, but now the trick is yours and the audience is a referee who has read a thousand of these and is not in a charitable mood. Here is the move the whole chapter turns on, so you have it before the details. **The identifying assumption is the single sentence on which your entire causal claim rests.** Everything else in the paper — the data work, the controls, the robustness checks — is in service of making that one sentence believable. If the sentence is false, no amount of clustering or winsorizing saves you. If you cannot *state* the sentence, you do not yet have a design; you have a regression and a hope. The memo forces you to write the sentence down, in public, before the data can seduce you into believing whatever the first significant coefficient happens to say. --- ## 1. The identifying-assumption sentence Start with the sentence, because everything else hangs from it. The form is fixed, and you should write it in this template literally, filling the three blanks: > Our estimate of **[the effect]** is **causal / credible** as long as **[the assumption]**, which we defend by **[the design feature or evidence]**. Three blanks, three jobs. The first names *what you are estimating* — not the regression, the *effect*. "The effect of a state fair-lending regulation on the mortgage-denial gap between minority and white applicants," not "the coefficient on the treatment dummy." The second names *the condition under which the estimate equals that effect* — and here is the rule that separates a real memo from a hand-wave: **name the specific threat, not the word "endogeneity."** This is CONVENTIONS §4 stated for your own paper. "As long as there is no endogeneity" is not an identifying assumption; it is an admission you have not thought about which endogeneity. "As long as treated and control states would have followed parallel denial-gap trends absent the regulation" *is* an identifying assumption, because it names exactly the counterfactual that has to hold and exactly the way it could fail. The third blank names *the design or evidence* that makes the second blank believable — fixed effects that absorb the obvious confounder, a placebo test that would have caught the obvious failure, an institutional fact that rules out the obvious alternative story. Notice the asymmetry baked into "causal / credible." Some designs let you say *causal* with a fairly straight face when the assumption holds — a clean RD at a cutoff nobody can game, a randomized instrument. Most observational designs earn the weaker word *credible*: the assumption is not testable, you have argued for it as hard as you can, and an honest reader grants you a conditional belief, not a proof. Choose the word that matches how strong your assumption really is. Overclaiming in this sentence is the tell of an amateur; a referee who reads "causal" where "credible, conditional on parallel trends" was warranted will distrust everything that follows. A worked contrast makes the discipline concrete. Here is Maya, drafting her sentence three times, getting worse to better: - **Draft 1 (useless):** "We estimate the causal effect of fair-lending regulation on denial rates, controlling for endogeneity." *No.* No threat is named; "controlling for endogeneity" is the exact phrase the Conventions forbid. This sentence could front any paper on any topic. It commits to nothing, so it can be attacked from every side at once. - **Draft 2 (better):** "Our estimate of the regulation's effect on the minority–white denial gap is credible as long as treated and control states had parallel trends." *Closer* — it names the design's assumption — but it still does the third blank no work. *Defended how?* A reader cannot tell whether parallel trends is asserted or earned. - **Draft 3 (a real memo sentence):** "Our estimate of the effect of state fair-lending examinations on the county-level minority–white mortgage-denial gap is credible as long as, absent the examinations, adopting and never-adopting states' denial gaps would have moved in parallel — which we defend with a flat pre-trend in the event-study leads, a Callaway–Sant'Anna comparison against never- and not-yet-treated states only, and the institutional fact that adoption timing was driven by legislative calendars unrelated to local lending conditions." That sentence is a contract. It tells the referee exactly what to attack (the parallel-trends counterfactual), exactly what evidence you are betting on (flat leads, clean controls, exogenous timing), and exactly what would falsify you (tilted leads, or a story tying adoption timing to local lending). Draft 3 is long, and that is correct: the identifying-assumption sentence is allowed to be the longest sentence in your paper, because it is the most important one. The paragraph that contains it is the spine of the whole project. --- ## 2. The standard menu of threats, by design You are not inventing threats from scratch. Each design from Weeks 3–4 comes with a *standard menu* of named failures — the ones a referee will reach for first, because they are the ones that bite that design. Knowing the menu is half the battle: it tells you, before you have written a word, which alternative explanations you must pre-empt. Here is the menu, design by design, each threat stated as the specific way the identifying assumption fails. These are the recurring villains of Weeks 3–4; the memo is where you finally name them out loud against your own data. **OLS on observables (selection-on-observables / conditional independence).** The assumption is that, *conditional on the controls*, treatment is as-good-as-random — there is no confounder you left out. The standard threats are **omitted-variable bias** (a variable that drives both treatment and outcome and is not in your controls — Week 2's ghost, and the reason "we control for X" is a claim, not a spell) and **selection** (units choose into treatment on something you cannot see). This is the *weakest* design on the credibility ladder precisely because its assumption — "I have measured every confounder" — is the hardest to defend and the easiest to attack: a referee needs only to name one plausible omitted variable to wound you. **Instrumental variables (exclusion / weak instrument).** Two assumptions, and Week 3 drilled their asymmetry: **relevance** (the instrument moves the treatment) is *testable* — you report the first-stage $F$ — and **exclusion** (the instrument affects the outcome *only* through the treatment) is *arguable* — there is no test, no $p$-value, only an argument from institutions. The two named threats are therefore the **weak-instrument** problem (first-stage $F$ too small, so 2SLS drifts back toward OLS bias and its confidence interval lies about its own width) and the **exclusion violation** (the instrument has its own back-channel to the outcome). When a referee attacks an IV paper, they almost always attack exclusion, because it is the assumption the data can never defend. **Difference-in-differences (parallel trends / anticipation).** The assumption is **parallel trends**: absent treatment, treated and control units' outcomes would have moved together. The named threats are a **pre-existing differential trend** (the treated group was already diverging before treatment, so a slope gets mislabeled an effect), **anticipation** (units respond *before* the official treatment date — they saw it coming — so the "pre" period is already contaminated), and, in staggered designs, the **forbidden comparison** of Chapter 4.2 (already-treated units used as controls, injecting the negative-weights bias when effects are dynamic). Remember the one sentence underlined three times in Chapter 4.1: parallel trends is a claim about a counterfactual, so a pre-trend can *refute* it but a flat pre-trend can never *confirm* it. **Regression discontinuity (manipulation / compound treatment).** The assumption is **continuity** of potential outcomes at the cutoff. The named threats are **manipulation/sorting** (units precisely control which side of the cutoff they land on, so the just-above and just-below groups are no longer comparable — the McCrary fingerprint is a jump in the density), **covariate imbalance** at the cutoff (something predetermined jumps where only treatment should), and the most insidious, **compound treatment** (more than one thing switches at the *same* threshold, so your jump is the bundled effect of all of them, not the clean treatment you wanted). Manipulation and imbalance leave fingerprints you can test for; compound treatment passes every statistical test and must be ruled out by institutional argument alone. **Synthetic control (donor pool / anticipation).** The assumption is that a weighted average of untreated **donor** units reproduces the treated unit's *untreated* path. The named threats are a **bad donor pool** (no convex combination of donors actually matches the treated unit's pre-period, or the donors are themselves contaminated by spillovers from the treatment), **anticipation** (the treated unit reacts before the event, fouling the pre-period fit), and **interpolation/overfitting** (the weights fit pre-period noise and so mispredict the counterfactual). With an effective sample size of one, inference is not a t-test but a **placebo permutation** — assign the fake treatment to each donor in turn and ask how extreme the real effect looks against that distribution. You will use exactly one of these designs for your primary specification — the one you committed to in your PAP. Pull its row off this menu and you have your starting list of threats. Then add any that are specific to *your* data (a survivorship filter, a look-ahead leak from Chapter 7.4, a measurement-error problem from Week 1). The menu guarantees you do not forget the obvious ones; your own data supplies the rest. --- ## 3. The threats-and-responses table Now build the table. It is the operational heart of the memo, and it has exactly four columns. Write it as a literal Markdown table in your paper, because its rigid shape is what forces honesty: every threat must have a response, and every response must end in a residual concern, so you cannot quietly drop the threats you have no answer for. | Column | What goes in it | |---|---| | **Threat** | The named failure of the identifying assumption — from the menu above, stated in one phrase. *Not* "endogeneity." | | **Why it's plausible** | One sentence on the concrete story under which this threat is *real* in your setting. If you cannot tell a plausible story, the threat is a strawman and does not belong in the table. | | **What we do about it** | The specific test, design choice, or robustness check that addresses it — with the expected result that would *reassure* a reader. | | **Residual concern** | What still worries you after you have done everything in column 3. The honest leftover. If this column is blank, you are not being honest. | The fourth column is the one students want to leave empty, and it is the one a good mentor reads first. A threats table whose Residual-concern column is all "none" is not a stronger paper — it is a *less* credible one, because every real design has residual concerns and a reader who sees none assumes you have not looked hard enough. The point of the table is not to pretend the threats are vanquished; it is to show you have *seen* them, *engaged* them, and *bounded* what remains. That is what intellectual honesty looks like on a page, and it is the single trait that most distinguishes a paper a referee trusts from one they do not. A practical note on ordering: put your rows in *descending order of danger*. The first row should be the threat that, if true, would most cleanly kill the paper — the one a seminar audience asks about in the first ninety seconds. Burying your most serious threat in row four reads as evasion; leading with it reads as confidence. You are signaling that you are not afraid of your own hardest question. --- ## 4. Threats you can test vs. threats you can only argue Here is the distinction that organizes the entire third and fourth columns of your table, and it is the most important conceptual point in the chapter. **Some threats leave fingerprints in the data, and some do not.** The ones that do, you *test*. The ones that do not, you *argue*. Confusing the two — running a test for something that is fundamentally untestable, or arguing your way around something you could simply have checked — is the most common error in a weak identification memo. **Threats you can test** have a checkable empirical signature, so column 3 is a statistic and column 4 is "what the test can and cannot rule out." The canonical four, one per design: - **Pre-trends (DiD).** Estimate the event study and read the leads. Flat, insignificant leads are consistent with parallel trends. *But remember the asymmetry:* a flat pre-trend cannot *confirm* the counterfactual — it can only fail to refute it — so even a clean test leaves a residual concern, and you must say so. - **First-stage $F$ (IV).** Report it. An $F$ comfortably above the modern weak-instrument thresholds (and an Anderson–Rubin interval that does not blow up) reassures on *relevance*. It says nothing about exclusion. - **Balance (OLS / RD).** A balance table — covariate means by treatment status, or covariates run as outcomes in an RD — shows whether the groups look alike on observables. Balance on observables is reassuring but, by construction, silent on *unobservables*: the very confounder you fear is the one not in the table. - **McCrary density (RD).** Test for a jump in the density of the running variable at the cutoff. A smooth density is consistent with no manipulation. A jump is often fatal. **Threats you can only argue** have *no* fingerprint — the data are logically incapable of revealing them — so column 3 is an institutional fact or an economic argument, and column 4 is "this rests on a claim the data cannot confirm." The canonical members: - **The exclusion restriction (IV).** Untestable, full stop. You argue it from how the instrument works, and you defend it against every alternative path a critic proposes. Week 3's slogan: relevance is testable, exclusion is arguable. - **Compound treatment (RD).** Passes every statistical test and still breaks the design. Only institutional knowledge — "I have read the rule and *only* my treatment switches at this cutoff" — can rule it out. - **The parallel-trends counterfactual (DiD), beyond the pre-period.** The pre-trend test checks the periods you observe; whether trends *would have stayed* parallel in the post-period absent treatment is a counterfactual no test can reach. You argue it from the exogeneity of treatment timing. - **Anticipation, when it occurs silently.** If units reacted before any date you can observe, no lead coefficient catches it; you argue from institutions whether the treatment was anticipable. The discipline is simple to state and hard to live by: **for every threat in your table, decide first whether it is testable or arguable, and put the right kind of thing in column 3.** A memo that runs a clever test against an untestable threat (e.g., claiming a balance table "shows the exclusion restriction holds" — it cannot) advertises that the author does not understand their own design. A memo that argues its way past a threat it could have simply tested looks lazy. Match the response to the nature of the threat, every time. --- ## 5. Intellectual honesty: state the residual, don't oversell The temptation, as the data come in and a coefficient turns up significant with three stars, is to write the memo *backward* — to start from the result you want and reverse-engineer an identifying assumption that delivers it. This is the spec-search and the p-hacking you were warned about in Week 1, wearing the costume of a research design. The PAP you filed in Chapter 7.3 exists precisely to stop this: it pinned your primary specification *before* you peeked, and your identification memo must defend *that* spec, not whichever one the data later flatters. The memo and the PAP are written in the same spirit — commit first, then test — and a memo that quietly drifts to a different design than the one you registered has broken the contract. Three honesty rules, which double as the things a grader looks for: 1. **State the residual concern, every time.** The Residual-concern column is mandatory and must be substantive. "We cannot rule out a contemporaneous policy shock that hit treated states in the same year" is worth more to your credibility than a triumphant "no concerns remain." Referees reward candor because candor is rare and easy to verify: they will think of the residual concern whether or not you write it, so writing it first robs the attack of its force. 2. **Match the verb to the evidence.** If your assumption is untestable and merely argued, your sentence says *credible*, not *causal*, and your prose says "consistent with" and "we cannot reject," never "proves" or "establishes." Calibrate every claim to the strength of its support. A two-star result on a design with a serious untestable threat is "suggestive evidence," not "we show that." 3. **Do not bury the hard question.** Lead the table with your most dangerous threat. The instinct to hide it is exactly backward: the hidden threat is the one the referee finds anyway, and finding it themselves is worse for you than your having flagged it. The deep point is that the identification memo is not a defensive document you write to *survive* review. It is a thinking tool you write to *find out whether your project is real*. If, while filling the table, you discover a threat for which column 3 is empty and column 4 is "this could be the entire effect," you have learned something invaluable *before* sinking Week 8 into a paper that cannot be saved — and the right response is to change the design, not to soften the language. The memo's first reader is you, and its first job is to talk you out of a project that will not survive contact with a skeptic. --- ## 6. A worked identification memo: Maya's HMDA fair-lending DiD Let us write the whole thing, end to end, for the project Maya has carried since Week 1 and built in Lab 4: a staggered difference-in-differences on HMDA, asking whether state **fair-lending examination** programs reduced the minority–white gap in mortgage-denial rates. This is the exact design of Chapter 4.2 (staggered adoption, dynamic effects, Callaway–Sant'Anna against clean controls), pointed at the dataset Prof. Gao and his coauthor used to ask whether same-sex couples were treated differently when they applied to borrow.[^gaosun] **The specification, in CONVENTIONS §4 form.** Outcome: county-year minority–white mortgage-denial gap (denial rate for minority applicants minus denial rate for white applicants), from the HMDA LAR. Treatment: indicator for whether the applicant's state had an active fair-lending examination program in that year (staggered adoption, cohort $G_i$ = year the state's program took effect; never-adopters as clean controls). Controls: county-year applicant-composition covariates (income, loan-to-value, requested-loan-amount distributions) entering via covariate-conditional parallel trends. Fixed effects: county and year, absorbed inside the group-time estimator. Clustering: by state (the level at which treatment varies). Sample: HMDA LAR, all U.S. counties, years spanning the adoption waves, pinned to a fixed data vintage. **Identifying assumption (one sentence):** absent the examination programs, adopting and never-adopting states' county-level minority–white denial gaps would have followed parallel trends. **The identifying-assumption paragraph.** *Our estimate of the effect of state fair-lending examination programs on the county-level minority–white mortgage-denial gap is credible as long as, absent the programs, the denial gaps in adopting and never-adopting states would have evolved in parallel — a per-cohort parallel-trends assumption that we defend three ways: with an event study whose pre-treatment leads are flat and statistically indistinguishable from zero; with a Callaway–Sant'Anna estimator that forms only clean comparisons against never- and not-yet-treated states, never the already-treated states that contaminate pooled TWFE under dynamic effects; and with the institutional fact that the timing of each state's program was set by its legislative and budget calendar, which we argue is unrelated to contemporaneous shocks to local mortgage markets. We report a single overall ATT and an event-time profile, with state-clustered inference and a placebo check that randomly reassigns adoption years.* That paragraph is the contract. Everything in the paper defends it. **The threats-and-responses table.** | Threat | Why it's plausible | What we do about it | Residual concern | |---|---|---|---| | **Differential pre-trend** (parallel trends already failing before treatment) | States that chose to adopt examinations may have been on a different denial-gap trajectory — e.g., already-improving states legislate improvement. | **Testable.** Event-study leads, normalized at $k=-1$; report and plot them. Reassuring result: leads flat and insignificant. | A flat pre-trend cannot *confirm* the post-period counterfactual; it only fails to refute it (Ch 4.1). | | **Forbidden comparison / negative weights** (staggered TWFE using already-treated states as controls) | Adoption is staggered (Ch 4.2) and the effect plausibly *builds* as examiners ramp up, the exact recipe for the bias. | **Design choice.** Use Callaway–Sant'Anna group-time ATTs against never- and not-yet-treated controls only; run a Goodman–Bacon decomposition to show how much weight pooled TWFE put on forbidden comparisons. | Clean estimator needs a sufficient never-treated pool; if few states never adopt, late cohorts' effects are thinly identified. | | **Anticipation** (lenders tighten or loosen before the program's official start) | Programs are debated publicly for months; lenders could adjust ahead of the effective date, contaminating the "pre" period. | **Partly testable.** Inspect leads in the quarters just before $G_i$; optionally exclude the year before adoption and re-estimate. | Anticipation that began before any observed lead is invisible to the leads; rests on the institutional-timing argument. | | **Compound policy shock** (another state policy moves with examination adoption) | States that strengthen fair-lending examinations may *simultaneously* change CRA enforcement, licensing, or down-payment-assistance rules. | **Arguable.** Document each adopting state's contemporaneous lending-policy changes from legislative records; argue no co-timed policy plausibly moves the denial *gap*. | No statistical test can detect a perfectly co-timed policy; this rests on the completeness of the institutional review. | | **Outcome is a composite measure** (the denial *gap* mixes applicant composition with treatment) | The raw gap can move if the *mix* of who applies changes, not because anyone was treated differently — a Week 1 measurement issue. | **Testable / design.** Covariate-conditional (doubly-robust) DiD on income/LTV/loan-size distributions; show the gap moves for composition-matched applicants. | Conditioning only on *observed* applicant characteristics; unobserved creditworthiness differences could still shift the gap. | | **Few-treated-cluster inference** (state-clustered SEs strained when adopters are few) | Cluster-robust SEs need many treated clusters (Ch 4.1); a handful of adopting states strains the t-approximation and the BDM correction. | **Inference choice.** Report state-clustered SEs *and* a placebo permutation test (randomly reassign adoption years, locate the real ATT in that distribution). | With genuinely few adopters, the uncertainty is large and hard to pin precisely; the placebo distribution is the honest summary. | Read that table the way a referee would. The first row is the hardest question — parallel trends — and it leads, as it should. Every row's column 3 is the *right kind* of response: a test where the threat is testable (pre-trends, composition, placebo) and an institutional argument where it is not (compound policy). And no Residual-concern cell is empty, because none honestly could be. This is a memo that does not oversell: the sentence says *credible*, not *causal*, and every leftover worry is on the page where a reader can see Maya already saw it. A paper built on this spine can be attacked — every paper can — but it cannot be ambushed. --- ## 7. The bridge to Week 8 You have now closed the design half of your project. Chapters 7.1–7.4 took you from a question to a feasible dataset to a filed pre-analysis plan to a clean, documented analysis dataset; Chapter 7.5 distilled the design into the one page the whole paper defends. This memo is what Assessment 7 grades — the PAP and the identification memo together, on the research-design rubric — and it is the **spine of the Week-8 paper**. Here is how the spine becomes the paper. The identifying-assumption paragraph becomes your *empirical strategy* section, nearly verbatim. The threats-and-responses table becomes the skeleton of your *robustness* section: Chapter 8.2 (execution and robustness) is, almost literally, the instruction "go run column 3 for every row of your table" — the placebos, the alternative standard errors, the bandwidth and sample perturbations that turn each promised defense into a real result. Chapter 8.3 (writing the paper) will have you turn the memo's prose into an introduction that *promises* exactly what the design can deliver and no more — the calibrated-verb discipline from §5, applied to the abstract. And Chapter 8.4 (peer review) will hand your memo to another student, who will read it the way you learned to read Fama–French in Week 5: tables first, identifying assumption second, prose last and skeptically — and who will attack the row you hoped to bury. Write the memo so that there is no such row. The thread that runs from Week 5's reading discipline through this memo to Week 8's defense is a single idea, stated once more: *the skeptic who reads a stranger's table and the author who builds an honest one are the same person.* You spent Weeks 5–6 learning to find the load-bearing assumption in someone else's paper. The identification memo is you doing it to yourself, on purpose, before anyone else gets the chance. --- ## Your Turn Open **nb7.5 — "First-look regressions (frozen until the PAP is filed)."** This notebook runs the *first* estimate of your primary specification — the headline number your memo defends. It is **frozen** for a reason that is the whole ethic of this week: you do not run it until your pre-analysis plan (Chapter 7.3) and this identification memo are filed as a tagged commit (Lab 7). The freeze is the technological enforcement of §5's honesty rules: with the spec and the threats committed in advance, the first-look regression can *inform* you but can no longer *seduce* you into rewriting the design to fit the result. Run it, compare the point estimate to what your PAP predicted, and — before you touch anything — re-read your threats table and ask which row the first look just made more or less worrying. Three reflection prompts to write up alongside the notebook: 1. **Find your most dangerous threat.** Of the rows in your own threats table, which one, if it were true, would most cleanly kill your paper? Is it testable or only arguable? If it is arguable, write the two strongest sentences of institutional or economic argument you have — and be honest about whether they are enough. 2. **Audit your verb.** Reread your identifying-assumption sentence. Did you write *causal* or *credible*? Defend the choice against the actual strength of your evidence. If you wrote *causal*, name the testable fingerprint that earns it; if you cannot, downgrade the verb and adjust every claim in the memo to match. 3. **The empty cell test.** Is any Residual-concern cell in your table blank or "none"? If so, you have either a genuinely bulletproof threat (rare — defend it) or a threat you have not looked at hard enough (likely — go look). Conversely, is there a threat whose residual concern is "this could be the entire effect"? If so, the memo just did its most important job: it found the project-killer before Week 8 did. Decide now whether to redesign or to proceed with the limitation stated plainly. [^gaosun]: Sun, H., & Gao, L. (2019). Lending practices to same-sex borrowers. *Proceedings of the National Academy of Sciences*, 116(19), 9293–9302.