How long a wrong sentence survives

On 14 August a sentence went onto the about page saying that we deliberately do not measure bounce rate or time on page, because both would need a session followed across page views, and that would need a consent banner on the embedding site.

On 27 August we shipped time on page. The sentence stayed.

Thirteen days true, twenty-eight hours false

The dates are exact because the repository knows them. The sentence arrives at 22:41 on 14 August. Dwell time, in six buckets, arrives at 17:53 on 27 August — and from that minute the second half of the sentence is false. It is corrected at 21:57 on 28 August.

So it was right for thirteen days and wrong for twenty-eight hours, which sounds like a good ratio until the next observation: nothing in the twenty-eight hours was going to end them. No test failed. No check fired. The page rendered, the feature worked, and the two contradicted each other quietly in eleven languages.

Why no check could have caught it

Every automated check in this project compares two things that are both in the system: a row count against eleven, a label width against the space it has, a list of tables against the schema. A claim on a page is not comparable to anything. It is prose about the world, and the world in this case was a table that had been created in another part of the codebase entirely.

A check that greps the about text for feature names and looks them up is imaginable. It would be long, fragile, and it would still miss the interesting cases, because the sentence did not name a table. It named a reason: this would need a consent banner. That reason had quietly stopped applying.

What ended it

A question. Somebody asked whether we show a bounce rate, and the honest answer required reading the sentence next to it, which is when it fell over.

That is the same mechanism that took four other claims off this page earlier this month, and it is not a mechanism, it is luck. The only thing that improves the odds is writing the claim narrowly enough that it stays checkable: not «we do not measure X, because Y», but «X is not on this page» — a sentence that describes the page it sits on rather than the reasons behind it.

What generalizes

Code rots loudly. Prose about code rots silently, and it usually rots at the moment the code gets better.

Thirteen places had to be edited to fix one sentence: eleven language versions of one interface string, and two paragraphs of hand-written HTML. That number is worth knowing before writing the next explanation, because it is the price of every claim that reaches past the page it is printed on.