How wide should a tolerance be?

Wide enough for the measurement error of the two methods and no wider. Rounding and stated precision are legitimate sources. An unexplained gap is not, and the correct response to one is investigation.

What if two reads disagree and I cannot find out why?

Report the disagreement. A build that blocks with the gap named leaves the reader in a better position than one that passes under a bound chosen to accommodate it.

Why keep the tolerance in configuration?

Because it is a judgment, not a mechanism. In configuration it is versioned, attributable, and visible on the page. Buried in code it changes without anyone watching it happen.

Failure catalogue · The verification is theater · 32 of 51

OBSERVED FAILURE MODE

The tolerance that ate the error.

Two reads disagree, and the tolerance is widened until they agree. The disagreement was the signal, and the fix deleted it. A tolerance chosen after seeing the gap is a record of the gap, not a bound on it. Every run afterwards passes, including the ones still carrying the defect that caused the original mismatch.

What we saw

In our own audits, two reads disagreed and the tolerance grew until they did not. The build went green. The underlying difference was never investigated, so whatever made the two paths diverge stayed in the pipeline with a bound now wide enough to cover it. From outside, the check still read as a dual read within tolerance. The only trace of the incident was the size of a number in configuration, which no reader of the page ever sees.

Why it passes a glance

A tolerance is legitimate. Every dual read has one, and no reviewer can tell a bound chosen from the measurement error of the method from a bound chosen to fit an unexplained gap. The check keeps its name, its row, and its green status, and the page still states that the two paths agree.

What addresses it

The verification-practice skill requires the tolerance to be written down before the compare and the disagreement to be investigated rather than absorbed. Principle 2, two independent reads beat one careful one, says it directly: you never auto settle a disagreement and never widen the tolerance to make it go away. The bound is configuration, so a change is visible and attributable.

Check your own file in two minutes

Find the tolerance value and the date it last changed.
Ask what disagreement prompted the change, and whether the cause was ever explained.
Restore the previous tolerance and rerun to see whether the gap is still there.
Record the bound with its reason beside it, in configuration rather than in code.

What this does not catch

A written, versioned tolerance makes widening visible. It does not decide what bound is right for a given method, and a tolerance set too wide on the first run has no earlier value to compare against. Where the correct bound is unknown, the disagreement is reported rather than covered.

Quick answers

How wide should a tolerance be?: Wide enough for the measurement error of the two methods and no wider. Rounding and stated precision are legitimate sources. An unexplained gap is not, and the correct response to one is investigation.
What if two reads disagree and I cannot find out why?: Report the disagreement. A build that blocks with the gap named leaves the reader in a better position than one that passes under a bound chosen to accommodate it.
Why keep the tolerance in configuration?: Because it is a judgment, not a mechanism. In configuration it is versioned, attributable, and visible on the page. Buried in code it changes without anyone watching it happen.

Nearby failures

Both rails from one blueprintThe verification is theater The screen calibrated by its own targetThe verification is theater The check that cannot failThe verification is theater

Last updated 2026-09-02 · Dylan, founder · one of 51 observed failure modes, every one seen in a real build or in our own audits, none invented.

Get the free skill → The doctrine All 51 failure modes