Guide
How the lessons fit together
An answer from an AI looks the same whether anyone checked it or not. The lessons in this guide are the habits I use to tell the two apart, and here they are in the order they come up in a project.
The rule under all of them: a result counts as finished only after a deliberate attempt to show it is wrong, such as recomputing it another way or comparing it with a published table, has failed to do so. This is one person's working method, built from the errors my own checks caught, and a result checked this way can still be wrong.
Step 1 of 5
Before you compute
Write down the published values your work must reproduce and the result that would make you stop, search for earlier work, and give each AI helper its target numbers from the project's files.
Step 2 of 5
While computing
Check the work by a second route, and when two checks agree, ask what step they share; then describe a result by the cases you actually ran.
- Five ways to check an AI answer, from weakest to strongest
- A second program run to check the first gave the same wrong answer
- A program that worked at every size it was tested on broke at the next size up
- How to have separate AI helpers check each step of your work
- Pick a comparison group that differs only in ways that cannot cause the pattern
Step 3 of 5
Before you claim
Words like "saved", "running" and "done", and every total, are claims, so check the real state before you write them, and test an AI's explanation the way you test a number.
- Read a change back before recording that it was made
- Check that your work was really backed up, whatever the tool said
- A status page said "running" for six days after the work had stopped
- Recount the total from the list, and count each item once
- Why a wrong AI answer looks right, and one question that exposes it
Step 4 of 5
The record
Keep one row per claim with how well it is supported and the evidence, and leave claims that turn out false in the table, marked false.
Step 5 of 5
Writing it up
Write up a "no" as carefully as a "yes", and before anything goes public, have a reader with no context read it and fix every place they get lost.