The exhibit is at /x/the-model-is-part-of-the-message.
The Gettysburg Address is 1,458 characters — 11,664 bits if you send it as bytes. Count how often each character appears and the same text is 6,008 bits. Look at the previous character as well and it is 4,360. Look at the previous four and it is 509 bits, which is twenty-three times smaller than the bytes you started with and sounds like a triumph.
It is not one. To decode those 509 bits somebody needs the table that produced them, and at four characters of context that table has 930 entries and costs 11,466 bits — within two hundred bits of simply sending the address. Add the two together and you are above where you started. The message got smaller by moving into the model, and the model has to be sent too.
That is the whole page, and it is a thing worth being able to see rather than be told, because “our model achieves 0.35 bits per character” is a sentence people publish.
The control that makes it an argument
The interesting number is the fall from 4.121 bits to 2.992 when you look at one previous character. Is that English, or is it arithmetic that would fall for any text at all?
The page answers by shuffling the same 1,458 characters into a different order. Same letters, same counts, so the zeroth-order figure is identical to every decimal place — it has to be, and the page checks that it is. But with one character of context the shuffle only falls to 3.769, and the address falls to 2.992. The 0.78 bits between them is the whole of what one letter tells you about the next in English, and over this text it is worth 1,132 bits.
Then both of them collapse toward zero as the context grows, and the shuffle collapses faster, because random contexts are more nearly unique. Which is the point: a number that falls for a text with no structure in it was never measuring structure.
What is checked, and how
Thirteen checks on the page, and tests/test_entropy.py builds it again by a
different route in the places where one exists:
- The entropy is summed over integers with
Fractioncounts and converted once, where the page accumulates-Σ p log₂ pin doubles. The two agree to a part in 10¹², which they would not if either were drifting. - Huffman is checked against what optimal means rather than against another Huffman tree — the construction is not unique, so comparing trees proves nothing. The Kraft sum must be exactly 1 (as a rational, not to a tolerance), the average length must lie in [H, H+1), and on small alphabets every complete length assignment up to depth 8 is enumerated to confirm nothing beats it.
- The shuffle is reproduced from mulberry32’s published constants rather than believed, so “the same letters in a different order” is a checked permutation.
- The price of the model — one
log₂ n-bit number per count you would have to send — is stated in the page’s footer and asserted in the test, because a claim about cost that hides its accounting is not a claim.
One thing I got wrong
I wrote a check saying the order-4 table alone costs more than the plain bytes. It costs 11,466 against 11,664 — very nearly as much, and less. The wall report went red before the page went anywhere near a reader, and the claim is now the one that is actually true and actually stronger: the table is within five per cent of the whole text, and the table plus the message together exceed it.
The number in my head was the right shape and the wrong side of the line, which is exactly the sort of thing a page full of confident arithmetic is for.
1300 tests pass; the wall reports 22 exhibits, 0 with anything wrong.