Failure catalogue · The verification is theater · 28 of 51
OBSERVED FAILURE MODE
The painted row.
A verdict written into the page as text rather than computed by the run. The scan executes, produces findings, and the findings are discarded before render, while the row shows the outcome its author expected. The page reports a conclusion the build never reached. Re-running changes nothing, because no part of the row depends on the data.
What we saw
In our own audits we found verdict text hardcoded to green while the scan behind it computed its findings and dropped them. The scan worked. The render ignored it. On the page the row sat among rows that were computed on the run, in the same styling, and it stayed green through inputs that should have turned it red. A reader auditing the page by reading it would have found nothing, because the defect was the absence of a link between two steps.
Why it passes a glance
The work behind the row is real, which is what makes the row convincing. Logs show the scan running. Timings look right. Only the last step, moving a computed verdict into the rendered output, is missing, and no reader can see a missing assignment from the front of a page.
What addresses it
The period-freshness skill states the rule in the form a builder needs: no painted rows and no dead knobs, every rendered row computed on this run with a fire test that can turn that exact row red, and a row that cannot change deleted rather than rendered. Principle 6, test that a check fires, not that it exists, is what the rule enforces on every surface.
Check your own file in two minutes
- Trace each rendered verdict back to the variable the run computed, and confirm the render reads it.
- Break the input that row is meant to catch, then confirm the printed text changes.
- Search the render code for verdict words written as string literals.
- Delete any row whose text no run can alter.
What this does not catch
Tracing verdicts to their computation catches text that never moves. It does not prove the computation is correct, and a row wired to the wrong variable still changes on the wrong input. Wiring is checkable by machine; whether the row measures the right thing is not.
Quick answers
- How is a painted row different from a check that cannot fail?
- A check that cannot fail never computes anything. A painted row computes a real result and then discards it before rendering. Both print a guarantee the build did not earn, and a captured red run catches both.
- Would a code review catch this?
- Often not. The scan is real and the code around it looks complete. What catches it is running the build against input that should turn the row red and watching the printed text stay exactly the same.
- What should the row show when a check is informational?
- An INFO status with no green mark and no implied guarantee. The status vocabulary is PASS, INFO, and BLOCKED, and only PASS claims anything about the numbers.
Nearby failures
Last updated 2026-09-02 · Dylan, founder · one of 51 observed failure modes, every one seen in a real build or in our own audits, none invented.