Failure catalogue · The metric means something else · 21 of 51
OBSERVED FAILURE MODE
Lift as a multiple or a percent.
Promotional lift is reported as a multiple in some toolsets and as a percentage in others. A stored 1.4 means 1.4 times base in one system and 40 above base in another. The same digits carry a hundred-point difference with no notation to warn a reader. Both readings pass every arithmetic check they meet.
What we saw
Meridian Journal Group runs subscription promotions and pulls lift from two places: a campaign tool that returns a multiple and a reporting export that returns a percent. One file said 1.4 and meant a multiple. The other said 40 and meant a percent above base. Read either one under the other convention and the campaign either barely moved or transformed the quarter. Both column headers said lift, and no unit appeared anywhere on the page that carried the figure.
Why it passes a glance
A unit is not a value, so no reconciliation touches it. Both files tie to their own totals and both figures reproduce exactly on a re-run. The tell would be a unit label, and a unit label is the first thing an export drops. A percent check that fires in only one direction lets the other convention through untouched.
What addresses it
The metric-store skill records lift once with its unit and display format alongside the formula, so every page renders the figure the same way and a build that finds no entry stops instead of guessing. Principle 10, a metric without its definition is not a number, covers the unit fork explicitly: settled once, rendered with its unit, computed one way everywhere the metric appears.
Check your own file in two minutes
- Find one campaign whose base and post-promotion values you can read directly, and compute the lift by hand.
- Compare your figure with the reported one to see which convention it matches.
- Scan the column for values clustered near one, which usually means multiples are mixed in with percentages.
- Add the unit to the stored definition and to the label rendered on the page.
What this does not catch
A stored unit closes the notation fork. It does not verify the lift itself, because lift divides by a counterfactual that never happened. Different providers return different lifts for the same event, so the method and its parameters are disclosed and the figure stays a model output.
Quick answers
- How do I tell a multiple from a percent in an export?
- Compute one campaign by hand from a base and a post-promotion value in the same file, then see which convention reproduces the reported figure. Column headers rarely carry the unit, so the file has to be made to prove it.
- Is lift verifiable at all?
- The arithmetic is. The claim is not, because the denominator is a counterfactual that never happened. Disclose the method and its parameters, label the figure as modelled, and keep it out of any reconciled total.
- What stops the unit changing between reports?
- One stored definition with a unit and a format, read by the build rather than retyped per page, plus a check that fails naming the metric when a rendered figure and the stored formula disagree.
Nearby failures
Last updated 2026-09-02 · Dylan, founder · one of 51 observed failure modes, every one seen in a real build or in our own audits, none invented.