Failure catalogue · The file lies · 15 of 51

OBSERVED FAILURE MODE

Products living only in free text.

The thing a row is about exists nowhere but a prose field: a description, a memo line, a subject. Extraction rules grow into dozens of patterns nobody audits, buried in code, and a whole product breakdown rests on matching no reader can see, review, or ever tell has stopped covering half the rows.

What we saw

At Meridian Journal Group the title a line item concerned appeared nowhere except a free-text description written by whoever raised the order. The extraction pulling titles out of it grew, one pattern at a time, into dozens of rules living inside the build. Nobody could say what share of rows the rules matched, because the build never counted. New titles arrived under wordings no pattern covered and fell out of the breakdown entirely, which looks identical on a page to a title that simply did not sell that month.

Why it passes a glance

Unmatched rows leave the breakdown without leaving the totals, so the grand total still ties while the categories under it are short. A missing product looks like a product with no sales, which is an ordinary thing to see. The rules are code, so a reviewer reading the page cannot count them, and a page that never states its coverage gives a reader nothing to be suspicious of.

What addresses it

Doctrine principle 8, thresholds are configuration, not code, covers extraction terms as much as tolerances: the pattern list is a setting a person can read and change, empty configuration fails loudly rather than matching nothing quietly, and coverage is reported as a number. The anchor-discovery skill writes those patterns into the source spec with explicit operators, so a rule covers the rows that arrive next month, not only the ones visible today.

Check your own file in two minutes

1. Count the rows your extraction matched and the rows it did not, and put both numbers on the page.
2. Read twenty unmatched rows and decide whether they are genuinely unclassifiable or simply worded a new way.
3. Move the pattern list out of the code and into a configuration file a person can read without opening the build.

What this does not catch

Coverage tells you how many rows matched, not how many matched correctly. A pattern filing two products under one name reports full coverage, and free text written by people will always carry cases no pattern anticipated.

Quick answers

What number makes extraction auditable?

The share of rows the patterns matched, published on the page beside the breakdown those patterns produced.

Why does an unmatched row look harmless?

Because it leaves the grand total alone, so a missing product reads as a product that had no sales.

Where should extraction terms live?

In configuration, where a person can read the list, add to it, and see when it was last changed.

Nearby failures

Last updated 2026-09-02 · Dylan, founder · one of 51 observed failure modes, every one seen in a real build or in our own audits, none invented.