Stats4U 3 is here

The site this post sits on was rebuilt this month — not restyled, rebuilt, starting at the counting path. There is a separate post about the eleven quiet years that led up to it. This one is about what actually changed, and the measurements that made the case for doing it at all.

The number that decided it

Before any work started, someone counted how many counter images were being delivered correctly. The answer was about half.

Of roughly 4,200 counter requests a day, 2,125 were returning something other than a visitor count. Designs that had quietly stopped drawing years earlier because they queried database tables that no longer existed. Blank images. Errors where a number should have been. Nobody had reported any of it, for a simple reason: a counter showing nothing looks a lot like a counter showing a small number.

That figure is now about 25 a day. Half a percent.

A system in that condition does not get fixed by patching around the edges, so it was taken apart and put back together instead.

Lighter, and measured rather than claimed

The statistics page used to load a 208 KB charting library. It drew five charts, four of which showed the same information as the lists sitting next to them, and it made up roughly 90% of the page — the page's own HTML was 8 KB. The library is gone. The charts are drawn directly as SVG now, under 1 KB, with no script involved at all.

• Statistics page: about 16 KB compressed, down from roughly 78 KB.
• Front page: 44 KB of HTML, down from 85 KB — and the 652 KB of preview images that were rebuilt on every single visit are now static files with a one-year cache.
• Start-up cost per request: 0.94 ms, down from 7.26 ms.

That last figure is worth a sentence. Every counter request — including the ones whose entire job is to return a small image — was loading a translation table out of the database and opening a 63 MB geolocation file, then discarding both results unused. That was about 6.4 milliseconds spent and discarded on every request, several thousand times a day, for years.

What is new

• No cookie in the counting path, and nothing read from the visitor's device. Visitors are told apart by a hash that is discarded after two days.
• Bots are recognised and measured. The statistics page now shows what share of the hits came from crawlers rather than people. They still count in the totals — there is a post of its own about why.
• Private statistics are protected by a key in the link — without it the page does not open.
• The data can be exported and taken elsewhere.
• A live view: who is on the site at this moment, and today's traffic hour by hour.
• Dark mode, and the whole site in ten languages.
• No advertising on the statistics pages. A statistics page is not a place to sell anything.

What deliberately did not change

• Counter IDs and embed codes. Nothing needs re-pasting. Code from 2013 still works.
• The history. Every visit counted before the rebuild is still counted after it.
• The old designs. The bitmap counters from 2006 are still on offer next to the new ones. They are part of why people are here, and a rebuild is not a reason to throw away somebody's chosen counter.
• Free, with no account required. Same as it was in 2006.

What is next

Most of the list above deserves more than one line, and several items will get a post of their own — how the counting works without a cookie, how much of the traffic turns out to be crawlers, and what happened to the old designs.

For anyone with a counter here that stopped working at some point over the last few years, this is a good moment to look at it again. Given the numbers at the top of this post, there is a reasonable chance it was one of the broken half, and a reasonable chance it is working now.