Failure catalogue · The wrapper betrays the numbers · 50 of 51
OBSERVED FAILURE MODE
The estimate outside its own interval.
Two code paths computing the same figure will apply different exclusions eventually, and the page will show both results side by side. When a headline figure sits outside the confidence interval published beside it, the report is contradicting itself in public and no check on either path can see it. Each path is internally correct.
What we saw
On a Meridian Journal Group page, a headline figure and its published interval were produced by two code paths that had drifted apart on which rows they excluded. Each path was internally consistent, each reconciled against the source, and each passed its own checks. Rendered side by side, the estimate sat outside the interval meant to contain it. A reader who noticed had no way to tell which half was wrong, and a reader who did not carried away a figure with false precision attached to it.
Why it passes a glance
Nothing compares the two paths, because each was verified against the data rather than against the other. Drift arrives as an ordinary change: an exclusion added where the figure is computed and not where the interval is. Both surfaces keep passing, and the contradiction is visible only to a reader holding both numbers at once and knowing what an interval claims.
What addresses it
The metric-store skill gives every named figure one definition, one formula and one anchor in a versioned file the build consults before computing, and it demands a mechanism rather than a promise: either the stored formula is executable, or a firing-tested check re-derives each rendered figure and fails naming the key when code and store disagree. Doctrine principle 6, Test that a check fires, not that it exists, is what turns that check into evidence.
Check your own file in two minutes
- Find every figure on the page computed in more than one place and name the second place.
- Add a check that re-derives the displayed figure from the stored definition and fails on a mismatch.
- Change one exclusion, rebuild, and confirm the check turns red rather than the interval moving quietly.
What this does not catch
One source of computation makes figures consistent, not correct. A definition everyone shares can still be the wrong definition, which is why each entry carries an owner and the fork it settled. Review the store whenever the business changes shape.
Quick answers
- How does a figure end up outside its own interval?
- Two code paths, and one exclusion added to only one of them. Each stays internally correct and each keeps passing its own checks, so only a reader comparing the two numbers ever sees the contradiction.
- Does a shared definitions file fix it?
- Only with a mechanism behind it. Either the stored formula is executable, or a firing-tested check re-derives each rendered figure and fails naming the key when the code and the store disagree.
- What about archived reports built under an older definition?
- They keep their as-built definition in their own bytes. A definition change adds a superseding entry rather than editing history, because last quarter's page was correct under last quarter's rule.
Nearby failures
Last updated 2026-09-02 · Dylan, founder · one of 51 observed failure modes, every one seen in a real build or in our own audits, none invented.