nb6.3 — The Loughran–McDonald Sentiment Pipeline

Companion to Ch 6.3, the reader’s guide to Loughran & McDonald (2011), “When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks” (Journal of Finance 66(1): 35–65).

The chapter handed you a riddle disguised as a title. When is a liability not a liability? Answer: almost always, inside a 10-K. To a general-English dictionary the word “liability” is a burden, a black mark, a negative word; to an accountant it is Tuesday — “deferred tax liability,” “current liabilities,” “liability insurance.” Loughran and McDonald’s whole paper is the discovery that a sentiment dictionary built for novels and newspapers, pointed at a corporate annual report, flags as “negative” a mountain of words that carry no negative meaning in finance at all — and that a finance-specific word list does dramatically better.

We are going to rebuild that discovery with our own hands, on a small corpus we write ourselves so the notebook runs offline. The plan, lifted straight from §7 of the chapter:

Write a tiny self-contained corpus of synthetic “10-K / 8-K sentence” snippets — some genuinely grim, some studded with accounting words that a general dictionary mistakes for gloom.
Implement a bag-of-words sentiment scorer from scratch.
Build two negative word lists: a naive / general-English one and a small finance-specific (LM-style) one (illustrative subsets, clearly labeled; we also show how to load the full public LM Master Dictionary in a cell we deliberately do not run).
Score the corpus both ways and quantify how badly the general list over-flags the benign-finance sentences.
Add tf–idf weighting and discuss what it buys you.
Break the bag-of-words model on purpose with negation (“not a bad quarter”) — the failure that motivates the embeddings and LLMs of Ch 6.5.

Everything is numpy, pandas, scikit-learn, and matplotlib. No internet, no licensed data.

Setup

We force matplotlib’s non-interactive Agg backend so the notebook runs headless — laptop, server, or CI — with no display attached (the CONVENTIONS.md rule). We also fix a seeded generator rng = np.random.default_rng(42) so any randomness reproduces bit-for-bit; in this notebook we use it only to shuffle a held-out split.

import matplotlib
matplotlib.use("Agg")  # headless, non-interactive backend

import re
from collections import Counter

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

rng = np.random.default_rng(42)

pd.set_option("display.width", 100)
pd.set_option("display.max_colwidth", 60)
print("environment ready")

1. A tiny corpus of “10-K / 8-K” sentences

Real text-as-data starts with fifty thousand words of messy HTML pulled from SEC EDGAR. That is the engineering cost the chapter warned about, and it would put this notebook online and slow. So we do what good measurement practice allows: build a small, transparent, synthetic corpus that has the same structure as the real thing, where we ourselves know the ground truth for each sentence. (When you want the real pipeline, the EDGAR full-text search API and the sec-edgar-downloader package fetch actual 10-Ks; the data card data-cards/sec-edgar.md has the recipe.)

We label each sentence with a truth tag a human reader would assign:

negative — genuinely bad news a person would read as gloomy (“we incurred a substantial loss,” “going concern doubt”).
benign_finance — neutral accounting / business prose that happens to contain words a general-English dictionary treats as negative (“liability,” “tax,” “cost,” “depreciation,” “vice president,” “crude oil”). This is the trap.
positive — genuinely good news.

The whole experiment hinges on the benign_finance rows. A good sentiment tool should leave them alone. A general-English dictionary will light them up like a Christmas tree.

# Synthetic 10-K / 8-K style sentences. Written in-notebook so this runs offline.
# truth = the label a careful human reader would assign.
corpus_rows = [
    # --- genuinely negative: a human reads these as bad news ---
    ("The Company incurred a substantial net loss and impairment charge during the fiscal year.", "negative"),
    ("Our auditors have expressed substantial doubt about our ability to continue as a going concern.", "negative"),
    ("We are the defendant in litigation alleging fraud, and an adverse judgment could be material.", "negative"),
    ("Revenue declined sharply and the board terminated the restructuring plan after weak demand.", "negative"),
    ("The restated financial statements revealed a material weakness and a deficiency in controls.", "negative"),
    ("Management warns of severe losses, default risk, and a possible bankruptcy filing.", "negative"),

    # --- benign finance: accounting prose stuffed with general-English "negative" words ---
    ("The deferred tax liability increased as depreciation and amortization expense rose.", "benign_finance"),
    ("Our Vice President of Tax reviewed the cost of goods sold and the liability accounts.", "benign_finance"),
    ("Crude oil and capital expenditure costs are recorded net of the depreciation reserve.", "benign_finance"),
    ("The liability for income tax and the cost of capital were disclosed in the notes.", "benign_finance"),
    ("Depreciation, tax expense, and the warranty liability are estimated each quarter.", "benign_finance"),
    ("The board's vice chair approved the crude inventory cost and deferred tax liability schedule.", "benign_finance"),

    # --- genuinely positive ---
    ("Revenue grew strongly and the company achieved record profit and improved margins.", "positive"),
    ("We successfully gained market share and delivered outstanding shareholder returns.", "positive"),
    ("Earnings beat expectations and the board approved a higher dividend on solid demand.", "positive"),
]
corpus = pd.DataFrame(corpus_rows, columns=["text", "truth"])
corpus.index.name = "doc_id"
print(f"{len(corpus)} sentences  |  truth label counts:")
print(corpus["truth"].value_counts().to_string())
corpus

2. The bag-of-words model, in code

The chapter derived the model in one sentence: throw away word order, grammar, and sentence structure; what remains is a multiset — a bag — of words and their counts. “The company expects continued losses” and “Continued losses the company expects” are, to this model, the same object.

To get there we tokenize: lowercase the text and split it into word tokens, discarding punctuation. Then a document is just a Counter of its tokens. The negativity score is the count form from §2 of the chapter:

\text{NegScore}_d \;=\; \frac{\#\{\text{negative-list words in document } d\}}{\#\{\text{total words in document } d\}},

the share of the document made of negative-list words. The only thing that changes between two competing measures is which words are on the list — which is exactly why the chapter calls the choice of dictionary the “treatment” (the term from §2).

TOKEN_RE = re.compile(r"[a-z]+")

def tokenize(text):
    """Lowercase and split into alphabetic word tokens. Punctuation dropped."""
    return TOKEN_RE.findall(text.lower())

def bag_of_words(text):
    """A document as a multiset {word: count}. Order is thrown away here."""
    return Counter(tokenize(text))

def neg_score(text, neg_words):
    """NegScore = (# negative-list words) / (# total words). Share, in [0, 1]."""
    bag = bag_of_words(text)
    total = sum(bag.values())
    if total == 0:
        return 0.0
    hits = sum(count for word, count in bag.items() if word in neg_words)
    return hits / total

def neg_hits(text, neg_words):
    """Which negative-list words fired, and how many times (for diagnostics)."""
    bag = bag_of_words(text)
    return {w: c for w, c in bag.items() if w in neg_words}

# sanity check: order genuinely does not matter to the bag
a = bag_of_words("continued losses the company expects")
b = bag_of_words("the company expects continued losses")
print("bag-of-words is order-invariant:", a == b)
print("tokens of sentence 0:", tokenize(corpus.loc[0, "text"]))

3a. Two dictionaries: naive general-English vs finance-specific (LM-style)

Now the two contestants. Both are illustrative subsets, written by hand and clearly labeled as such — they are teaching toys, not the real lists. (The next cell shows how to load the genuine, full LM Master Dictionary.)

The naive / general-English negative list is the kind of thing you would scrape from a generic “negative words” page: words that are negative in ordinary English. Crucially it includes accounting vocabulary — liability, tax, cost, depreciation, crude, vice, capital — because in a novel or a tweet those words really do skew dark. This is a stand-in for the Harvard-IV-4 / General Inquirer list that Loughran and McDonald were arguing against.

The finance-specific (LM-style) negative list keeps only words that are negative when a company writes them about itself — loss, litigation, adverse, deficiency, restated, termination, bankruptcy, default — and pointedly excludes the benign accounting terms. That single editorial decision is the entire paper.

# --- Contestant A: naive / general-English negative list (Harvard-GI-style stand-in) ---
# NOTE: illustrative teaching subset, NOT the real General Inquirer list.
naive_neg = {
    # words that are negative in general English AND benign in finance (the trap):
    "liability", "liabilities", "tax", "taxes", "cost", "costs",
    "depreciation", "crude", "vice", "capital", "expense", "expenses",
    # words that are negative in general English and also negative-ish anywhere:
    "loss", "losses", "lose", "bad", "decline", "declined", "weak", "doubt",
    "adverse", "fraud", "default", "severe", "fail", "failed", "risk",
}

# --- Contestant B: finance-specific (LM-style) negative list ---
# NOTE: illustrative teaching subset, NOT the full LM Master Dictionary.
# Built on what words MEAN in a financial disclosure; benign accounting terms excluded.
lm_neg = {
    "loss", "losses", "litigation", "adverse", "deficiency", "restated",
    "termination", "terminated", "bankruptcy", "default", "doubt",
    "impairment", "fraud", "weakness", "decline", "declined", "weak",
    "severe", "restructuring", "alleging",
}

print(f"naive general-English list:   {len(naive_neg)} words")
print(f"LM-style finance list:        {len(lm_neg)} words")
print()
print("Words the naive list flags that the LM list deliberately drops:")
print(sorted(naive_neg - lm_neg))

3b. Loading the full public LM Master Dictionary (reference only — not executed)

The toy lists above exist so the notebook runs offline and so you can read every word. In real work you download the genuine article: the Loughran–McDonald Master Dictionary, maintained publicly at the University of Notre Dame’s Software Repository for Accounting and Finance. It is a single CSV with one row per word and integer columns marking each sentiment category (Negative, Positive, Uncertainty, Litigious, Strong_Modal, Weak_Modal, …); a non-zero entry is the year the word entered that category.

The cell below is the real loader. We do not run it (it needs the downloaded file), but it is exactly the code you would use — and it is why the chapter calls the dictionary the paper’s most durable deliverable: today “we measure tone with the LM dictionary” means “we downloaded this CSV and counted.”

# ----- REFERENCE ONLY: do NOT run unless you have downloaded the CSV. -----
# Source: https://sraf.nd.edu/loughranmcdonald-master-dictionary/
# File:   Loughran-McDonald_MasterDictionary_1993-2024.csv  (~9 MB, ~86k words)
import pandas as pd

md_df = pd.read_csv("Loughran-McDonald_MasterDictionary_1993-2024.csv")
md_df["Word"] = md_df["Word"].str.lower()

# A non-zero category column = the word belongs to that category.
lm_negative_full    = set(md_df.loc[md_df["Negative"]    > 0, "Word"])   # ~2,300 words
lm_positive_full    = set(md_df.loc[md_df["Positive"]    > 0, "Word"])
lm_uncertainty_full = set(md_df.loc[md_df["Uncertainty"] > 0, "Word"])
lm_litigious_full   = set(md_df.loc[md_df["Litigious"]   > 0, "Word"])

print(len(lm_negative_full), "negative words in the full LM dictionary")
# ...then swap `lm_negative_full` in wherever this notebook uses `lm_neg`.

Notice the categories beyond good/bad — Uncertainty and Litigious are finance-shaped (lawsuits and forward-looking hedging), not standard psychology categories. That taxonomy was itself a contribution (§5 of the chapter).

4. Score the corpus both ways

Now the horse race. We score every sentence twice — once with the naive list, once with the LM-style list — and lay the two NegScores next to the human truth label. Watch the benign_finance block.

scored = corpus.copy()
scored["naive_neg_score"] = scored["text"].apply(lambda t: neg_score(t, naive_neg))
scored["lm_neg_score"]    = scored["text"].apply(lambda t: neg_score(t, lm_neg))

# A document is "flagged negative" by a dictionary if its NegScore exceeds a threshold.
THRESHOLD = 0.05  # >= 5% of words on the negative list => flagged
scored["naive_flag"] = scored["naive_neg_score"] >= THRESHOLD
scored["lm_flag"]    = scored["lm_neg_score"]    >= THRESHOLD

cols = ["truth", "naive_neg_score", "lm_neg_score", "naive_flag", "lm_flag", "text"]
with pd.option_context("display.max_colwidth", 70):
    print(scored[cols].round(3).to_string())

Reading the table

Three things jump out, and they are the paper:

On the negative rows both lists fire — good, the bad news is genuinely bad.
On the positive rows both lists stay quiet — also good.
On the benign_finance rows the naive list lights up (those liability / tax / cost / depreciation / vice hits) while the LM-style list stays near zero. Same sentences, opposite verdicts, and the LM verdict is the human one.

A false flag on a benign_finance sentence is a false positive: the tool calls a neutral accounting sentence “negative.” Let us count them.

4b. Quantify the misclassification

The cleanest number is the false-flag rate on the benign-finance sentences: of the sentences a human calls neutral accounting prose, what fraction does each dictionary wrongly flag as negative? This is the notebook’s headline, the same shape as the paper’s headline.

benign = scored[scored["truth"] == "benign_finance"]

naive_false_rate = benign["naive_flag"].mean()
lm_false_rate    = benign["lm_flag"].mean()

print(f"Benign-finance sentences: {len(benign)}")
print(f"  naive general-English list  flagged {benign['naive_flag'].sum():>2d}/{len(benign)} "
      f"=> false-flag rate {naive_false_rate:.0%}")
print(f"  LM-style finance list       flagged {benign['lm_flag'].sum():>2d}/{len(benign)} "
      f"=> false-flag rate {lm_false_rate:.0%}")
print()
verdict = ("PASS: naive > LM, as the paper predicts"
           if naive_false_rate > lm_false_rate else "FAIL: did not reproduce")
print(f"naive false-flag rate ({naive_false_rate:.0%}) > LM false-flag rate "
      f"({lm_false_rate:.0%}) ?  {verdict}")

Where exactly does the naive list go wrong?

The paper’s first exhibit is not the validation regression — it is the list of which “negative” words actually dominate 10-Ks, and the finding that they are benign accounting terms. Let us rebuild that diagnostic: tally every word the naive list flags across the benign-finance sentences, and tag which are genuinely negative versus benign vocabulary.

# Tally naive-list hits across the benign-finance block.
hit_counter = Counter()
for txt in benign["text"]:
    hit_counter.update(neg_hits(txt, naive_neg))

hits_df = (pd.DataFrame(sorted(hit_counter.items(), key=lambda kv: -kv[1]),
                        columns=["word", "naive_neg_hits"]))
# A word is "benign in finance" if the LM-style list does NOT consider it negative.
hits_df["benign_in_finance"] = ~hits_df["word"].isin(lm_neg)

benign_share = (hits_df.loc[hits_df["benign_in_finance"], "naive_neg_hits"].sum()
                / hits_df["naive_neg_hits"].sum())

print(hits_df.to_string(index=False))
print()
print(f"Share of naive-flagged 'negativity' in the benign block that is "
      f"actually benign accounting vocabulary: {benign_share:.0%}")
print("That share is the notebook's version of the paper's headline statistic.")

A picture of the over-flagging

The bar chart makes the misclassification impossible to miss: mean NegScore by truth-label, the two dictionaries side by side. The benign_finance pair is the whole story — the naive bar towers, the LM bar barely registers.

means = scored.groupby("truth")[["naive_neg_score", "lm_neg_score"]].mean()
order = ["negative", "benign_finance", "positive"]
means = means.loc[order]

fig, ax = plt.subplots(figsize=(8.0, 4.6))
x = np.arange(len(order))
w = 0.38
ax.bar(x - w/2, means["naive_neg_score"], w, label="naive general-English list", color="#c0392b")
ax.bar(x + w/2, means["lm_neg_score"],    w, label="LM-style finance list",      color="#2e86c1")
ax.axhline(THRESHOLD, color="k", ls="--", lw=1.0, label=f"flag threshold ({THRESHOLD:.0%})")
ax.set_xticks(x); ax.set_xticklabels(order)
ax.set_ylabel("mean NegScore  (share of words on neg list)")
ax.set_title("The general dictionary over-flags benign accounting prose")
ax.legend()
fig.tight_layout(); fig.savefig("nb6.3-negscore-by-truth.png", dpi=110)
plt.close(fig)
print("saved nb6.3-negscore-by-truth.png")
print(means.round(3).to_string())

5. tf–idf weighting: how you count, not just what you count

So far every word counted the same. But the chapter’s last robustness idea — tf–idf, term frequency times inverse document frequency — says some words deserve more weight than others.

Term frequency \text{tf}(w, d): how often word w appears in document d.
Inverse document frequency \text{idf}(w) = \log\frac{N}{\text{df}(w)}, where N is the number of documents and \text{df}(w) is how many documents contain w. A word in every filing (think boilerplate “company,” “fiscal,” or — in our corpus — “the”) gets \text{idf}\approx 0 and is down-weighted toward irrelevance; a word that is rare across filings but frequent in this one gets a high weight.

The intuition for sentiment: a negative word that shows up in every single filing carries little distinguishing signal; a negative word that is rare overall but concentrated in this filing is a louder alarm. We let scikit-learn’s TfidfVectorizer do the bookkeeping, then build a tf–idf-weighted negativity score: the summed tf–idf weight of the negative-list words, normalized by the document’s total tf–idf weight.

from sklearn.feature_extraction.text import TfidfVectorizer

# Fit tf-idf over the whole corpus. token_pattern matches our tokenizer (alphabetic words).
vec = TfidfVectorizer(lowercase=True, token_pattern=r"[a-z]+", norm=None)
X = vec.fit_transform(scored["text"])          # (n_docs x n_vocab) sparse tf-idf matrix
vocab = np.array(vec.get_feature_names_out())

# Boolean mask over the vocabulary: which columns are LM-negative words?
lm_mask = np.array([w in lm_neg for w in vocab])

X_arr = X.toarray()
total_weight = X_arr.sum(axis=1)
neg_weight   = X_arr[:, lm_mask].sum(axis=1)
with np.errstate(invalid="ignore", divide="ignore"):
    tfidf_lm_score = np.where(total_weight > 0, neg_weight / total_weight, 0.0)

scored["lm_tfidf_score"] = tfidf_lm_score

# Compare raw-count LM ranking vs tf-idf-weighted LM ranking of the most-negative docs.
cmp = scored[["truth", "lm_neg_score", "lm_tfidf_score", "text"]].copy()
cmp["rank_raw"]   = cmp["lm_neg_score"].rank(ascending=False, method="min").astype(int)
cmp["rank_tfidf"] = cmp["lm_tfidf_score"].rank(ascending=False, method="min").astype(int)
print("Most-negative sentences by each LM scheme (lower rank = more negative):")
with pd.option_context("display.max_colwidth", 55):
    print(cmp.sort_values("rank_tfidf").round(3).to_string())

What tf–idf bought us (and what it did not)

Notice the re-ranking is mild here: on a 15-sentence toy corpus most words appear in only one document, so idf barely discriminates. That is itself the lesson. tf–idf earns its keep on large corpora — thousands of real 10-Ks — where genuine boilerplate (“forward-looking statements,” “pursuant to,” “fiscal year”) appears in nearly every filing and gets correctly muffled, while a sharp, rare negative word stands out. On a tiny corpus there is no boilerplate to suppress, so the weighting is nearly cosmetic.

The deeper point from §4 of the chapter: tf–idf changes how you count, not what you count. It cannot rescue a bad dictionary — weighting the word “tax” more or less cleverly does not change the fact that “tax” is not negative in finance. The dictionary choice dominates; weighting is a refinement on top.

6. Breaking the bag: negation and context

Here is the limitation that ends the bag-of-words era and opens Ch 6.5. The model throws away word order, so it cannot see that a word was negated. “This is not a bad quarter” is good news — but a bag-of-words scorer sees the word bad and flags it. The “not” is right there, three tokens away, and the model is structurally blind to it.

We hand-build a handful of these adversarial sentences — negation, conditionals, double negation — where the human reading and the bag-of-words reading disagree, and we watch the scorer fail.

# Sentences engineered so word order flips the meaning. human_label = correct reading.
trick_rows = [
    ("This was not a bad quarter; demand held up better than feared.", "positive"),
    ("We are not exposed to any material litigation or adverse judgment.", "positive"),
    ("Far from a loss, the segment posted its strongest result on record.", "positive"),
    ("We would face severe losses only if the unlikely default were to occur.", "neutral"),
    ("The company failed to lose money this year, exceeding every target.", "positive"),
]
tricks = pd.DataFrame(trick_rows, columns=["text", "human_label"])

# Use ANY reasonable bag-of-words negative vocabulary (LM + general-English combined):
# the failure below is intrinsic to word-counting, not specific to one list.
bow_neg = lm_neg | naive_neg
tricks["bow_neg_score"] = tricks["text"].apply(lambda t: neg_score(t, bow_neg))
tricks["bow_says"] = np.where(tricks["bow_neg_score"] >= THRESHOLD, "NEGATIVE", "not negative")
tricks["fired_words"] = tricks["text"].apply(lambda t: list(neg_hits(t, bow_neg).keys()))

with pd.option_context("display.max_colwidth", 60):
    print(tricks[["human_label", "bow_neg_score", "bow_says", "fired_words", "text"]]
          .round(3).to_string())

mismatches = (tricks["bow_says"] == "NEGATIVE") & (tricks["human_label"] != "negative")
print()
print(f"Bag-of-words disagrees with the human on {mismatches.sum()}/{len(tricks)} trick sentences.")
print("Each is a sentence a person reads as fine, but the word-counter calls negative —")
print("because it literally cannot see the 'not', the 'far from', or the 'failed to'.")

Why this is the bridge to Ch 6.5

Every miss above has the same root cause, and it is the chapter’s central vulnerability: a word is not an atom of meaning; meaning lives in context. The token bad means opposite things in “a bad quarter” and “not a bad quarter,” and a bag-of-words model has no way to represent that — it sees the same bad, the same count, the same score.

Loughran and McDonald patched simple negation with crude within-a-few-words rules, but sarcasm, conditionals (“severe losses only if the unlikely event occurred”), and long-range dependencies stay invisible. This is exactly the gap the next era fills:

Word embeddings (word2vec, GloVe) place each word as a dense vector learned from the company it keeps, so “good” and “great” land near each other — the cosine-similarity geometry of Ch 6.2.
Contextual models / LLMs (the transformers of Ch 6.5) represent each word as a function of the whole sentence around it, so “not a bad quarter” and “a bad quarter” finally get different representations.

Loughran–McDonald is the high-water mark of counting words. Seeing it fail on “not a bad quarter,” on your own screen, is the best possible motivation for the model that comes next — and the hardest lesson carries forward: a fancier model is also just an instrument, only as trustworthy as its out-of-sample validation.

A taste of out-of-sample discipline

The chapter’s §6 and Exercise 3 hammer one rule: a dictionary tuned on the same data it is tested on is grading itself with the answer key. We can show the shape of an honest test even on this toy corpus by splitting it into a build half and a held-out half (shuffled with our seeded rng), confirming the misclassification gap on data the comparison never “saw.” With 15 sentences this is a demonstration of the workflow, not a real statistical test — the real version uses thousands of filings and matched returns.

idx = np.arange(len(scored))
rng.shuffle(idx)                       # seeded shuffle -> reproducible split
cut = len(idx) // 2
build_idx, test_idx = idx[:cut], idx[cut:]

def benign_false_rate(frame, flag_col):
    b = frame[frame["truth"] == "benign_finance"]
    return b[flag_col].mean() if len(b) else np.nan

build, test = scored.iloc[build_idx], scored.iloc[test_idx]

print(f"{'split':<12}{'n':>4}{'n_benign':>10}{'naive_false':>14}{'lm_false':>12}")
for name, frame in [("BUILD", build), ("HELD-OUT", test), ("ALL", scored)]:
    nb_ = (frame["truth"] == "benign_finance").sum()
    print(f"{name:<12}{len(frame):>4}{nb_:>10}"
          f"{benign_false_rate(frame, 'naive_flag'):>14.0%}"
          f"{benign_false_rate(frame, 'lm_flag'):>12.0%}")
print()
print("On BOTH halves the naive list's false-flag rate on benign-finance prose")
print("exceeds the LM list's: the qualitative result is not an artifact of the split.")

What to carry forward

Four things from this notebook will keep paying off.

Meaning is a property of word-in-domain, not of the word. “Liability,” “tax,” “cost,” and “depreciation” are dark in a novel and neutral in a balance sheet. The naive list flagged the benign-finance sentences; the LM-style list left them alone. A measurement instrument does not transfer across domains without re-validation — Leah’s law for any text project.

Diagnosis by enumeration beats diagnosis by assertion. We did not argue the general dictionary was miscalibrated; we counted exactly which words did the damage and what share of the “negativity” was benign accounting vocabulary. That is what made Loughran and McDonald’s case unanswerable.

How you count is a second, smaller lever. tf–idf reweights words by distinctiveness and earns its keep on large corpora by muffling boilerplate — but it cannot rescue a wrong dictionary. Which words you count dominates how you weight them.

Bag-of-words is blind to context, and that is structural. “Not a bad quarter” defeats every word counter, because the model cannot represent that the same word means opposite things in two sentences. That blindness — not a tuning problem, a representational one — is precisely the gap embeddings and LLMs were built to close in Ch 6.5.

Your Turn

You have rebuilt the paper’s headline and broken the bag-of-words model on purpose. Now drive it yourself.

Extend both dictionaries. Add five more genuinely-negative finance words to lm_neg (e.g. delisting, covenant, writedown, shortfall, subpoena) and five more general-English-but-benign-in-finance words to naive_neg (e.g. interest, charge, obligation, reserve, provision). Re-run Sections 4–4b. Does the gap between the two false-flag rates widen? Which of your new “benign” words is actually a judgment call — could charge be negative in some sentences?
Write your own trap sentences. Append three new benign_finance rows to corpus — real-sounding accounting prose packed with words the naive list flags — and three new genuinely-negative rows. Re-score. Did the LM list stay quiet on your benign rows and fire on your negative ones? Where, if anywhere, did it slip?
Beat the negation, crudely. Loughran and McDonald patched negation with a within-N-words rule. Implement one: in neg_score, before counting, scan for a negator (not, no, never, without) and cancel any negative-list word that appears within three tokens after it. Re-run Section 6. How many of the trick sentences does your patch fix — and which (sarcasm, conditionals, “far from a loss”) does it still miss? That residue is the argument for Ch 6.5.
Add a positive dictionary and a net-tone score. Build a small lm_pos list (profit, growth, record, strong, gained, improved) and define \text{NetTone}_d = (\#\text{pos} - \#\text{neg}) / \#\text{total}. Score the corpus. Does net tone separate the three truth groups more cleanly than negativity alone? This is the seed of PS 6.3.