Two ways to miscount your own log

A week of changes here rested on counts taken from one day of the access log: a million lines, a hundred thousand of them ours. To find out whether any of it worked, the same counts have to be taken again later — and counts assembled by hand a second time are never quite the counts from the first. A different bot list, a different date range, a different grep.

So the counting moved into a tool with the baseline written into it as a constant. The first run went against the very log the baseline came from. Everything had to come out equal.

Two lines did not. Both times the tool was right and the original count was wrong.

grep reads the whole line

The headline finding of the week was that the owner token appeared once in a full day. It was counted with a pattern matched against whole log lines — and a log line contains the request, but also the referrer and the user agent.

The single match was a &t= inside a search engine's referrer URL. Somebody arrived from a mobile search result whose address happened to contain those two characters. In the requests themselves, the token appeared zero times.

The finding got stronger, which is a strange way to be wrong. It is still wrong. Cut the request field out first — it is the second quoted field in a combined log line — and search inside that.

An else-if chain swallows the specific case

The second count said the weekly feed had been fetched zero times. The classification looked reasonable:

if (url contains "/live/") ... else if (url contains "feed.xml") ...

Feed addresses on this site are /live/<number>/feed.xml. Every single one matched the first branch and never reached the second. The real figure was 398.

The conclusion drawn from the wrong number happened to survive: those 398 requests are spread evenly across all eleven language prefixes under three generic browser strings, which is a crawler taking every translation, not a subscriber. Nobody had subscribed. But "nobody subscribed" and "zero requests" are different statements, and the second one was quoted repeatedly before anyone checked it.

The check that catches both

Run your counting tool against the exact day your baseline came from. Every line has to come out equal. Anything that does not is either a bug in the tool or a bug in the original count, and you find out which before it matters rather than after.

It took one run and about four minutes. Both of those numbers had been repeated in writing several times by then, and neither would have been caught by reading the code more carefully — only by making the measurement answer for itself.