nb7.1 — The Idea-to-Spec Template

Companion to Ch 7.1, “Idea-Generation Workshop.”

For six weeks the questions were handed to you. From here on the question is yours, and Ch 7.1 revealed the trick for finding a good one: take a vague interest and pass it through three filters — Does anyone care? Is it new? Can I actually do it? — until what comes out is a single sentence of the form

the effect of [a specific treatment] on [a specific, measurable outcome], identified by [a specific source of variation], using [a specific dataset I can get].

This notebook is the machine that produces that sentence. It is mostly a fill-in-the-blanks template — you do the thinking in markdown cells — with one piece of code that actually runs: a feasibility/novelty scoring function implementing the Ch 7.1.7 rubric. You will:

Walk your raw puzzle through the pipeline: puzzle → refined question → directional hypothesis (with H_0) → identifying variation → a CONVENTIONS §4 spec line.
Score and rank two or three candidate questions with the rubric (5 dimensions, 0–2 each, out of 10), including the veto rule that kills any idea scoring 0 on Identifiable? or Feasible?.
Read a fully worked filled-in example (Priya’s disaster DiD) so you see a finished model.
Fill in a blank template for your own locked project.

Nothing here touches the network or any licensed data. The only imports are the standard stack.

1. Setup

We fix one seeded generator (rng = np.random.default_rng(42)) — the reproducibility rule from CONVENTIONS.md §5 — even though this notebook barely uses randomness (only to shuffle a tie-break demo). We force matplotlib’s non-interactive Agg backend so the notebook runs headless anywhere.

import matplotlib
matplotlib.use("Agg")  # headless, non-interactive backend

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from dataclasses import dataclass, field

rng = np.random.default_rng(42)  # one seeded generator for the whole notebook

print("numpy", np.__version__, "| pandas", pd.__version__)

2. The pipeline, one stage at a time

Ch 7.1 broke idea-generation into stages. We walk them in order. Each stage is a markdown cell you edit — replace the bracketed prompts with your own words. There is no autograder for prose; the discipline is the point. Keep your raw puzzle in front of you and refuse to skip a stage, because every stage you skip is a question the audience will ask in Phase 2.

The stages, and the slot of the final spec each one fills:

Stage	What you produce	Spec slot it fills (CONVENTIONS §4)
2.1 Puzzle	A mood: something that nags at you	(none yet — a mood is not a question)
2.2 Refined question	A specific, answerable question	outcome · treatment · sample
2.3 Directional hypothesis	A signed prediction with H_0	the falsifiable claim
2.4 Identifying variation	Where the clean comparison comes from	identifying assumption
2.5 Spec line	The one-sentence §4 spec	all of it, assembled

2.1 Puzzle — the mood you start fromA puzzle is a feeling that something is interesting. You cannot be wrong about a puzzle, so itis not yet research — but you should not work for five weeks on something that bores you, so starthonest about what nags at you.> YOUR PUZZLE (edit me):> e.g. “Crypto crashes seem to feed on themselves.” / “Insurers are fleeing wildfire country.” /> “Banks seem to treat some borrowers differently.”>> Replace this line with your own one-sentence puzzle.

2.2 Refined question — fill the four slots

Sharpen the puzzle into a question by naming the four slots of the hypothesis grammar (Ch 7.1.1). If you cannot fill a slot, that slot is where your idea is still soft.

Slot	Question to answer	Your answer (edit me)
Outcome y	What number goes in the y column, for many units?	e.g. daily DEX trading volume (log)
Treatment / variation D	What thing’s effect do you want, and how does it vary?	e.g. a stablecoin depeg event
Unit	Whose outcome — firm, person, county, token?	e.g. ERC-20 stablecoin × day
Window	Over what period / window?	e.g. 2022–2024, [-30,+5] days around event

YOUR REFINED QUESTION (edit me): Write the four slots above as one sentence. e.g. “When an ERC-20 stablecoin loses its dollar peg by >5%, how does same-day DEX volume respond, for major stablecoins over 2022–2024?”

2.3 Directional hypothesis — commit to a sign, and write H_0

A question becomes falsifiable the moment you commit to a direction: up, down, larger-for-A. State the alternative H_1 as a signed prediction, and the null H_0 as the boring “no effect” against which data could kill it. If you cannot write a sign, you still have a question, not a hypothesis.

YOUR HYPOTHESIS (edit me): - H_1 (signed): e.g. “DEX volume rises sharply on the depeg day vs. the prior 30-day average.” - H_0 (null): e.g. “No change in DEX volume on the depeg day relative to baseline.”

What pattern in the data would force you to abandon H_1? Write it down — that is the whole point of falsifiability.

2.4 Identifying variation — run the Wk 3–4 catalog backwards

Do not pick a method and hunt for a question. Pick the question, then ask: where, in the world that generates my data, is there variation in my treatment that is plausibly unrelated to the confounders I fear? Three patterns are realistic for a five-week project; a fourth is real but hard.

Pattern	Design (chapter)	What you need
A sharp rule / cutoff	Regression discontinuity (Ch 4.3)	a number crossing a threshold switches treatment on
A dated policy / shock hitting some units	Difference-in-differences (Ch 4.1)	clean before/after, a treated group and a comparable untreated group
A natural event with a timestamp	Event study (Ch 4.2)	measure outcome in a tight window around the event
A found / built instrument	IV / shift-share (Ch 3.4–3.5, 4.5)	an exogenous wiggle you can defend in one sentence — hardest; avoid unless very clean

If none of these is available, be honest: your project is descriptive, not causal. That is allowed (often wise for a first project). What is forbidden (CONVENTIONS §4) is a descriptive correlation dressed in causal language.

YOUR IDENTIFICATION (edit me): - Source of variation: e.g. the depeg is a dated on-chain shock → event study. - Single most dangerous confounder it does not handle: e.g. the depeg was caused by something that independently moved volume — addressed by a tight window + a pre-trend check. - Causal or descriptive? Circle one and mean it.

2.5 The spec line — assemble the CONVENTIONS §4 sentence

Now write the one line the whole rest of Week 7 is downstream of. CONVENTIONS §4 demands every spec name, explicitly: outcome · treatment/key regressor · controls · fixed effects · clustering · sample · the identifying assumption in one sentence.

YOUR SPEC (edit me — fill every field): - outcome = … - treatment = … - controls = … - fixed effects = … - clustering = … - sample = … - identifying assumption (one sentence) = absent [the treatment], [units] would have …

The code below gives you a tiny helper that just formats these fields into the canonical one-line string, so your spec comes out in the exact §4 order every time.

def spec_line(outcome, treatment, controls, fixed_effects, clustering, sample, identifying_assumption):
    """Format the seven CONVENTIONS §4 fields into one canonical spec sentence.

    This does NO analysis — it is a discipline device. It refuses to silently accept a blank field,
    because a blank field in a spec is exactly the gap an audience will find in Phase 2.
    """
    fields = {
        "outcome": outcome, "treatment": treatment, "controls": controls,
        "fixed effects": fixed_effects, "clustering": clustering, "sample": sample,
        "identifying assumption": identifying_assumption,
    }
    missing = [k for k, v in fields.items() if not str(v).strip()]
    if missing:
        raise ValueError(f"Spec is incomplete — fill these fields before you proceed: {missing}")
    head = " · ".join(f"{k}: {v}" for k, v in fields.items() if k != "identifying assumption")
    return f"{head}\n  identifying assumption: {identifying_assumption}"


# Demo with Devon's depeg spec (Worked Example A from Ch 7.1.6):
print(spec_line(
    outcome="daily DEX volume (log)",
    treatment="depeg event indicator (>5% off peg)",
    controls="token liquidity, market-wide volume",
    fixed_effects="token + day",
    clustering="by token",
    sample="major ERC-20 stablecoins, 2022–2024 (Etherscan + CoinGecko)",
    identifying_assumption="absent the depeg, the token's volume would have continued on its pre-event path.",
))

3. The feasibility / novelty scoring function

You will not carry one idea into Week 7 — you will carry three or four, and you need a fast, honest way to rank them. Here is the Ch 7.1.7 rubric as runnable code. Five dimensions, 0–2 points each, total out of 10:

Dimension	0 — fatal	1 — workable	2 — strong
Falsifiable?	A mood; no direction.	A direction, vaguely stated.	A sharp signed hypothesis with H_0.
So what / who cares?	No one acts on either answer.	A niche audience cares.	A real conversation cares; both answers matter.
Novel?	Re-run of a known result, same data.	New setting or period for a known idea.	New data, outcome, or identification.
Identifiable?	Pure correlation dressed as causal.	Descriptive, honestly labeled.	A real design (RD/DiD/event/IV) from Weeks 3–4.
Feasible in 5 weeks?	Data unavailable or out of scope.	Data obtainable; scope tight but real.	Data in hand by Ch 7.2; one clean question.

Read the score like a checklist with veto power, not a sum to maximize. The trick the code below encodes: any 0 on Identifiable? or Feasible? is a veto — the idea is killed regardless of its total, because an unidentifiable question is not research and an infeasible one will not finish. A solid project lands at 7 or above with no zeros.

DIMENSIONS = ["falsifiable", "so_what", "novel", "identifiable", "feasible"]
VETO_DIMENSIONS = ["identifiable", "feasible"]  # a 0 here kills the idea outright

@dataclass
class Idea:
    """One candidate research question scored on the five Ch 7.1.7 dimensions (each 0, 1, or 2)."""
    name: str
    falsifiable: int
    so_what: int
    novel: int
    identifiable: int
    feasible: int

    def scores(self):
        return {d: getattr(self, d) for d in DIMENSIONS}

    def total(self):
        return sum(self.scores().values())

    def vetoed(self):
        """True if any veto dimension (Identifiable / Feasible) scored 0."""
        return any(getattr(self, d) == 0 for d in VETO_DIMENSIONS)

    def verdict(self):
        if self.vetoed():
            zeros = [d for d in VETO_DIMENSIONS if getattr(self, d) == 0]
            return f"VETOED (0 on {', '.join(zeros)})"
        if self.total() >= 7:
            return "KEEP (>=7, no veto)"
        if self.total() >= 4:
            return "SALVAGE by narrowing (4-6)"
        return "DROP (too weak)"

    def __post_init__(self):
        for d in DIMENSIONS:
            v = getattr(self, d)
            if v not in (0, 1, 2):
                raise ValueError(f"{self.name}: '{d}' must be 0, 1, or 2 (got {v!r})")


def rank_ideas(ideas):
    """Return a DataFrame of ideas, sorted best-first.

    Sort key respects the veto: vetoed ideas sink to the bottom no matter their raw total, because
    the rubric is a checklist with veto power, not a sum. Among non-vetoed ideas, higher total wins.
    """
    rows = []
    for idea in ideas:
        row = {"idea": idea.name, **idea.scores(),
               "total": idea.total(), "vetoed": idea.vetoed(), "verdict": idea.verdict()}
        rows.append(row)
    df = pd.DataFrame(rows).copy()
    # not-vetoed (False) sorts before vetoed (True); then by total descending.
    df = df.sort_values(by=["vetoed", "total"], ascending=[True, False]).reset_index(drop=True)
    df.insert(0, "rank", df.index + 1)
    return df


# Quick self-check: a clean idea, a borderline idea, and one vetoed on Feasible.
_demo = [
    Idea("clean",     falsifiable=2, so_what=2, novel=2, identifiable=2, feasible=2),  # 10
    Idea("borderline",falsifiable=1, so_what=1, novel=1, identifiable=1, feasible=1),  # 5
    Idea("no_data",   falsifiable=2, so_what=2, novel=2, identifiable=2, feasible=0),  # 8 but VETO
]
rank_ideas(_demo)

4. Score three candidate questions and rank them

Now the rubric earns its keep. We score three candidate questions drawn from the cast (Ch 7.1.6) so you can see how the veto changes the ranking. Watch the third one: it has a strong total but gets sent to the bottom because its data cannot be obtained in five weeks.

Devon — depeg event study. Sharp signed hypothesis, live policy conversation, brand-new on-chain data, a clean dated event, and Etherscan/CoinGecko data is free. This should score near the top.
Maya — HMDA fair lending. Falsifiable, a real regulatory conversation, and HMDA is open and she already used it in Week 4 — strong feasibility. Novelty is “new angle on a known idea,” so a 1.
“Bloomberg whisper numbers.” A genuinely interesting falsifiable idea about analyst whisper forecasts — but the clean whisper dataset is a paywalled vendor feed a high-schooler cannot get in five weeks. Feasible = 0 → veto, no matter how interesting.

candidates = [
    Idea("Devon: stablecoin depeg -> DEX volume (event study)",
         falsifiable=2, so_what=2, novel=2, identifiable=2, feasible=2),   # 10, clean
    Idea("Maya: HMDA denial gap by race after controls (DiD/RD)",
         falsifiable=2, so_what=2, novel=1, identifiable=2, feasible=2),   # 9, strong
    Idea("Whisper numbers vs. earnings surprise (Bloomberg feed)",
         falsifiable=2, so_what=2, novel=1, identifiable=1, feasible=0),   # 6 but VETOED on data
]

ranked = rank_ideas(candidates)
ranked

4.1 Read the ranking — and notice the veto fired

Three things to confirm in the table above, the same checks the code asserts in the next cell:

Devon ranks #1. A perfect 10, no veto — the model project.
Maya ranks #2. A 9, docked one point only on novelty (a new angle on a known idea, not new data). Still a clear KEEP. A first runner-up to hold in reserve.
Whisper numbers sinks to last despite a respectable total of 6, because feasible = 0 trips the veto. The verdict column says VETOED. This is the rubric doing its job: killing an infeasible idea early, before you waste a week chasing a dataset you can never get.

The cell below turns those three observations into hard asserts, so if anyone later edits the scoring logic and breaks the veto, the notebook fails loudly instead of silently mis-ranking.

# Sanity checks: the ranking must behave sensibly and the veto must fire.
top = ranked.iloc[0]
bottom = ranked.iloc[-1]

assert top["idea"].startswith("Devon"), "the clean 10/10 idea should rank first"
assert bottom["vetoed"], "the bottom-ranked idea should be the vetoed one"
assert "VETOED" in bottom["verdict"], "vetoed idea must be flagged in its verdict"
assert bottom["total"] == 6 and bottom["total"] > ranked.iloc[1]["total"] - 99, \
    "veto must sink the idea BELOW higher-totaled non-vetoed ideas (rank, not raw total, wins)"

# The non-vetoed ideas must be ordered by total, descending.
non_vetoed = ranked[~ranked["vetoed"]]
assert list(non_vetoed["total"]) == sorted(non_vetoed["total"], reverse=True), \
    "non-vetoed ideas must be sorted by total descending"

print("All ranking checks passed.")
print(f"  #1: {top['idea']}  (total {top['total']}, {top['verdict']})")
print(f"  last: {bottom['idea']}  (total {bottom['total']}, {bottom['verdict']})")

4.2 A glance at the scores

A quick stacked bar makes the veto visible: an idea can be tall and still be dead. We render to the Agg backend (no window pops up) and show the figure inline.

fig, ax = plt.subplots(figsize=(8, 4))
order = ranked["idea"].tolist()
bottoms = np.zeros(len(order))
for dim in DIMENSIONS:
    vals = ranked[dim].to_numpy(dtype=float)
    ax.bar(range(len(order)), vals, bottom=bottoms, label=dim)
    bottoms += vals

# Mark vetoed ideas with an X over the bar.
for i, vetoed in enumerate(ranked["vetoed"]):
    if vetoed:
        ax.text(i, bottoms[i] + 0.15, "VETO", ha="center", va="bottom",
                fontweight="bold", color="firebrick")

ax.set_xticks(range(len(order)))
ax.set_xticklabels([s.split(":")[0] for s in order], rotation=0)
ax.set_ylabel("rubric points (out of 10)")
ax.set_title("Candidate ideas: rubric score, stacked by dimension")
ax.axhline(7, ls="--", lw=1, color="gray")
ax.text(len(order) - 0.5, 7.1, "keep threshold (7)", ha="right", va="bottom", fontsize=8, color="gray")
ax.legend(fontsize=7, ncol=5, loc="upper center", bbox_to_anchor=(0.5, -0.12))
fig.tight_layout()
plt.show()

5. A fully worked example — Priya’s disaster DiD

So you have a finished model in front of you, here is the entire pipeline run end-to-end for Priya’s project (Worked Example B from Ch 7.1.6), every stage filled in. Read it as the standard your own template (Section 6) should meet.

Stage 2.1 — Puzzle. “Insurers are fleeing wildfire and hurricane country.”

Stage 2.2 — Refined question. Do counties hit by federally declared disasters see homeowner-insurance costs rise faster than comparable counties that were not hit, in the years after the disaster? (outcome = county insurance-cost proxy; treatment = FEMA disaster declaration; unit = county × year; window = 2010–2023.)

Stage 2.3 — Directional hypothesis. H_1: insurance cost grows faster in disaster-hit counties post-event than in matched un-hit counties. H_0: parallel growth (no differential effect).

Stage 2.4 — Identifying variation. Difference-in-differences (Ch 4.1). Treated = counties with a FEMA declaration in a given year; control = comparable un-hit counties; the declaration is the dated treatment. The threat — disaster-prone counties differ systematically — is swept out by the difference in differences (which removes fixed county characteristics) plus a parallel-trends check on pre-disaster cost growth. This is a causal design, honestly identified.

Stage 2.5 — Spec line. Assembled by the helper below.

priya_spec = spec_line(
    outcome="county homeowner insurance-cost proxy (FRED/CPI insurance series or premium index)",
    treatment="FEMA major-disaster declaration (dated)",
    controls="(swept by fixed effects; add county-year covariates if available)",
    fixed_effects="county + year",
    clustering="by county",
    sample="U.S. counties, 2010–2023 (NOAA storm events + FEMA declarations + FRED)",
    identifying_assumption=("absent the disaster, hit and un-hit counties' insurance costs would "
                            "have trended in parallel."),
)
print("PRIYA — finished CONVENTIONS §4 spec:\n")
print(priya_spec)

# And here is how Priya's idea scores on the rubric — a clean keeper.
priya_idea = Idea("Priya: disaster DiD on insurance cost",
                  falsifiable=2, so_what=2, novel=2, identifiable=2, feasible=2)
print(f"\nRubric: total {priya_idea.total()}/10 -> {priya_idea.verdict()}")

6. Your turn — the blank template

Everything above was scaffolding and worked examples. This is the cell you actually fill in for your own locked project. Replace every "..." with your own content, run it, and the helper will either print your finished spec or raise an error naming the field you left blank. Then score your idea, and at least one runner-up, with the rubric.

If spec_line raises ValueError, that is the feature — it is telling you which slot of your research question is still soft. Go back to the matching stage in Section 2 and pin it down.

# === FILL THIS IN FOR YOUR OWN PROJECT ===========================================================
# Replace every "..." with your own words. Run the cell. Fix any field the error names.

MY_OUTCOME               = "..."   # the number in the y column, for many units
MY_TREATMENT             = "..."   # the key regressor / variation whose effect you want
MY_CONTROLS              = "..."   # legitimate factors you hold fixed (or '(swept by FE)')
MY_FIXED_EFFECTS         = "..."   # e.g. "firm + year", "county + year", or "none"
MY_CLUSTERING            = "..."   # e.g. "by firm", "by county", or "none"
MY_SAMPLE                = "..."   # units, period, and the dataset you can actually get
MY_IDENTIFYING_ASSUMPTION= "..."   # one sentence: "absent [treatment], [units] would have ..."

try:
    my_spec = spec_line(MY_OUTCOME, MY_TREATMENT, MY_CONTROLS, MY_FIXED_EFFECTS,
                        MY_CLUSTERING, MY_SAMPLE, MY_IDENTIFYING_ASSUMPTION)
    print("YOUR SPEC:\n")
    print(my_spec)
except ValueError as e:
    print("Spec not ready yet ->", e)

# --- Score your top idea and a runner-up, then rank ---
# Edit the five 0/1/2 scores. Add or remove Idea(...) rows as needed.
my_candidates = [
    Idea("MY TOP IDEA",    falsifiable=1, so_what=1, novel=1, identifiable=1, feasible=1),
    Idea("MY RUNNER-UP",   falsifiable=1, so_what=1, novel=1, identifiable=1, feasible=1),
]
rank_ideas(my_candidates)

Your Turn

You now hold the machine that turns a mood into a defensible sentence. Before the workshop:

Run your favorite puzzle through Section 2 end to end, refusing to skip a stage. Which slot — outcome, treatment, direction, or window — was hardest to pin down? That is where your idea is still soft.
Score three or four candidates in Section 4’s pattern, not just one. Keep the top one or two; let the veto kill the rest without mercy. A good question you finish beats a brilliant one you don’t.
Stress-test your top idea’s identification. Name the single most dangerous confounder your design does not handle. If you cannot name a cutoff, a dated policy, or a clean event, be honest that your project is descriptive — and label it that way (CONVENTIONS §4), rather than dressing a correlation in causal language.

The spec you lock here is the input to everything that follows: Ch 7.2 acquires the data, Ch 7.3 freezes it into a pre-analysis plan, Ch 7.4 builds the dataset, and Ch 7.5 makes you defend the identification in a memo. Get this one sentence right and the project almost runs itself.