Failure catalogue · The number is plausible, and wrong · 5 of 51
OBSERVED FAILURE MODE
The lift a holiday window tripled.
A period-over-period report names two windows: the period it reports and the baseline it is measured against. When the baseline window quietly absorbs a holiday, the lift is measured against a week that was never ordinary, and a promotion inherits credit for a season. Nothing in the file announces which window is which.
What we saw
A subscription promotion analysis at Meridian Journal Group headlined a lift near three times its true size. The promotion was real and the arithmetic was clean; the comparison window had simply drifted far enough to swallow a holiday week, and the campaign was credited with a seasonal swing it did not cause. No gate objected, because every date the report used really was present in the file. The heading named the promotion period, and the figure underneath it was measured against a baseline nobody had declared out loud.
Why it passes a glance
A lift figure has no total to tie to, so reconciliation has nothing to say about it. Every date involved is a real date from the file. A reader checking the reporting period sees the right months at the top of the page, because the window that moved is the comparison window, and that window usually appears nowhere on the page at all.
What addresses it
The period-freshness skill separates the reporting period from the comparison window and refuses to infer either from the minimum and maximum of the dates present, since a file carrying last year spans both. Doctrine principle 4, filenames lie, is the parent rule: the period a report is about is proved from the contents and declared as a role, never inherited from a name or from the extent of the dates in the file.
Check your own file in two minutes
- Write down both windows your comparison uses, with exact start and end dates, before you read the lift.
- Check the baseline window against a calendar for holidays, closures, and anything else that made that stretch unusual.
- Recompute the lift against the window either side of the baseline; if the number moves materially, the baseline is doing the work, not the promotion.
What this does not catch
Declaring both windows makes the comparison inspectable. It does not make the baseline representative: a window with no holiday in it can still be unusual for reasons the file never records, such as a supply gap, a price change, or a competitor's own promotion running against it.
Quick answers
- Why is the comparison window more dangerous than the reporting period?
- Because the reporting period is printed at the top of the page where a reader checks it, and the comparison window usually is not printed anywhere.
- Can a gate catch a baseline that spans a holiday?
- A gate can prove which dates the baseline covers and print them beside the figure, but whether those dates were ordinary is a judgement a person has to make.
- What should a lift figure carry on the page?
- Both windows, in dates, beside the number, so a reader can see what the lift was measured against without opening the build.
Nearby failures
Last updated 2026-09-02 · Dylan, founder · one of 51 observed failure modes, every one seen in a real build or in our own audits, none invented.