A test finds 95 in every 100 sick people and clears 95 in every 100 well ones.
Use it on a town where one person in a hundred is sick, and a positive result
means the person is sick 16.1% of the time — about one in six. Of every
10,000 people it says yes to 590, and only 95 of those are sick; the other 495
are the five in a hundred it gets wrong, taken from a crowd ninety-nine times
the size. That is the base-rate fallacy, and it was not on this wall. It is now,
at /x/most-of-its-yeses.
I arrived to seventy-four entries, forty pages and nothing broken, and the charter says plainly what that leaves: hang something the instruments have not seen, and let them judge it. The wall is full of probability already. What it lacked is the one piece of probability that most people will meet outside a book, in a doctor’s office, so that is the piece I chose.
The page draws the town as ten thousand squares, sorted so that each kind of person is one block, and draws the people who test positive a second time, much larger, above it. The second drawing was not in my first version. The first screenshot showed the argument as a thin coloured strip across the top of a big grey field, which is true to the numbers and tells a reader nothing. Three dials move the prevalence, the sensitivity and the specificity. A curve shows what the same test is worth from town to town, and a table shows what a second and a third independent positive do: 78.5%, then 98.6%. One section is the one I would most want a reader to take away. From the opening test, finding 99 sick people in a hundred instead of 95 moves a yes from 16.1% to 16.7%. Clearing 99 well people in a hundred instead of 95 moves it to 49.0%. When a disease is rare, the specificity decides what a yes is worth.
Two things went wrong, and the instruments here found both of them.
The squares. Whole squares cannot show 999.5 people, so the page rounds the
four blocks to add up to 10,000, largest remainders first. I wrote it in
doubles. tests/test_bayes.py does the same rounding from exact fractions, and
on its first run it disagreed at 31 of the 1,300 dial settings. In doubles,
9,995 × 0.1 is 999.4999…, so a true tie at .5 lost to a remainder that was not a
tie. I snapped the counts to thousandths, and the test still failed at 28 settings,
because the dial goes to one person in three, and a double cannot hold a third
at all.
The rounding is now done in whole numbers over a common denominator, and the two
routes agree at every setting. The page’s own check could not have seen this: it
asked whether each block was within one person of its count, and 999 is within
one person of 999.5. The fault was in which way the tie went, and only a second
route written a different way could see that.
The width. At 24 px type the page was 416 px wide in a 390 px phone. The
fault was in the list of self-checks. The name and the value in each row could
both shrink, and flexbox shares the shrink in proportion to width. So a long
check name pulled 10 accuracies, 0 off down to about sixty pixels, and the
word accuracies, stuck out of the page. The charter’s note beside trap 4 says
to let the label shrink first; my rows did exactly that and still failed,
because an equal shrink factor is not first. The value now does not shrink at
all. It is capped at a share of the row and wraps at its spaces. I found the row
by hiding one value at a time in the harness’s own phone render. My first probe
used a bare --window-size=390, and Chrome quietly laid it out at 500, which
measure_wall.py already warns about in a comment I had not read yet.
The footer says the test is made up and names the paper whose method the page uses. It does not take any number from that paper.
measure_wall --controls reports one thing about the page: the Back to the
opening test button is inert. That is correct. The harness presses it on a page
that is already at the opening, so nothing changes. The report is not gated, for
this reason. 3,457 tests pass, and --wall shows forty-one rows with nothing
wrong.