Guide · Checking a number
A program that worked at every size it was tested on broke at the next size up
If an AI writes you a program, a formula or a script and it gives right answers on every case you tried, you know it works on those cases. A real example of one that broke on the very next case, and what to say about a result instead of "it works."
In August I was extending a published table from chemistry: counts of molecule shapes built from hexagons, sorted by how many hexagons each shape has. The program that did the counting was written by Claude, an AI, under my direction. Before it counted anything new, it had to reproduce the published counts, and it did, exactly, at every size it was tested on, up to 14 hexagons.
The program read its shapes from a file made by another program. In that file, one character marks the start of the file's header, and the computer stores that character as the number 62. Inside each shape's entry, a number says how many corner points the shape has. No shape with 14 or fewer hexagons has 62 corners, so up to size 14 a 62 could only mean the header. Shapes with 15 hexagons can have exactly 62 corners. On August 15, the first time the program ran at size 15, it read one of those shapes as a header and crashed.
No test on real shapes at the sizes already tested could have found this, because the number 62 never appeared there as a corner count. The program was fixed to recognize the header by its full opening text instead of one character, a small earlier test was run again and still passed, a trial on 20,000 size-15 shapes ran cleanly, and then the full size-15 count was run. Its total came to 74,207,910 shapes, exactly the published number of shapes with 15 hexagons.
Here the failure was a crash, so it was obvious. The same kind of error could instead have let the program keep running and give wrong counts with no sign of trouble.
The habit that follows: describe a result by what was actually tested. "Checked against the published counts up to 14 hexagons" is true. "The program works" is a claim about sizes nobody has run.
To use it: the next time an AI builds you a formula, spreadsheet or script that works on your test data, write down the largest or most unusual case you tried. Before you rely on it for something bigger, such as a new month of data, more rows or a new category, run one case that goes past what you tested and compare it with an answer you can get another way.