Failure catalogue · The file lies · 17 of 51
OBSERVED FAILURE MODE
The platform grading its own homework.
An ad platform reports the conversions it believes it caused, on its own window and its own model. Every platform answers its own question in its own favour, so summed across platforms they routinely claim more revenue than the business made. These are not errors. They are assertions by an interested party.
What we saw
Meridian Journal Group ran subscription acquisition across several ad platforms and pulled each platform's reported conversions into one table. Added together, the platforms claimed more new subscriptions than the billing system had recorded, and no arithmetic anywhere was wrong: each platform was counting the conversions it believed it had caused, on its own attribution window, and the same subscription was counted by more than one of them. The table was headed paid acquisition revenue, and that heading is the step that turned several claims into one measurement.
Why it passes a glance
Each platform's figure is internally consistent and reconciles against that platform's own reporting. The sum is arithmetically correct. The failure is in the label, and a label is prose, so no gate reads it. A reader seeing a single figure headed paid revenue has no way to know it is the sum of several parties, each answering a different question about the same subscriber.
What addresses it
Doctrine principle 9, a platform's own numbers are claims, not facts, requires each platform figure to be labelled with its claimant and its attribution window and kept out of any reconciled total, which comes from a system of record and ties to an anchor. The contested-metrics skill carries the matching rule for definitions, refusing to place two differently based figures on one axis or into one sum, which is what keeps an attributed conversion out of reconciled revenue.
Check your own file in two minutes
- Add up what every platform claims for the period and compare that sum against what your billing or order system recorded.
- Relabel each platform figure with the platform's name and the attribution window it used.
- Keep the reconciled total on its own line, sourced from the system of record, and never add a platform claim into it.
What this does not catch
Labelling claims stops them being summed into a measurement. It does not tell you which platform actually caused a subscription, and your own attribution model is a claim too, labelled as yours, never laundered into the reconciled column.
Quick answers
- Are the platforms wrong?
- No, each is answering its own question on its own window, which is why their answers cannot be added together.
- Can a platform number appear on a report at all?
- Yes, beside the reconciled figure and never added to it, labelled with who is claiming it and over what window.
- What should reconciled revenue come from?
- A system of record, the store, the billing system, or the ledger, tied to a total that source states about itself.
Nearby failures
Last updated 2026-09-02 · Dylan, founder · one of 51 observed failure modes, every one seen in a real build or in our own audits, none invented.