Ask an AI to confirm each fact by name, and tell it where to check

When you ask an AI about a dataset, it can state a fact about that data that it never checked. On one of my projects, two AI helpers were given different instructions about the same data, and only one of them got a basic fact right.

The project asked whether 25.8 million Los Angeles parking tickets, published by the city, held a new research paper. A lead AI, Claude Opus 4.8, which I picked for this run, ran it. It sent out helper AIs, each a separate session with written instructions, called a brief. Early on, it sent out two of them.

The first helper's job was to establish what the dataset actually contains. Its brief said: "Verify facts against primary sources; do NOT assume fields exist, confirm them. Where you cannot confirm, say so explicitly." It then named every column to check, including a parking-meter ID column and a patrol-route column, asked for each one to be marked present or absent, and told the helper to check the city's data portal itself. That helper reported both columns present. It added that the meter column was mostly empty, and that where it was filled it held stray symbols such as "|" and "!" that could not be used as meter numbers.

The second helper's job was different: list the problems in this data that could sink a research paper. Its brief described the data in one sentence, naming the columns the team expected, and that list left out the meter and route columns. It told the helper to search the web and reason it out, and it did not ask the helper to look at the data's columns. The helper reported that the data had no meter column and no route column, which matched the list in its brief rather than the data.

Minutes later, the lead queried the city's live database itself. Both columns were there. The route column was filled in on about one ticket in six, and the meter column on a similar share, though its codes could not be used as meter numbers. So the first helper had it right. The lead overruled the second helper's claim and wrote down why, and kept the rest of its report, about problems that affect any study of this data.

I can't prove the wording of the second brief caused the error. Naming the facts you need in a request costs a few lines, and it gave the right answer here.

Before you send your next request about a spreadsheet, a database or a report, read it once as the AI would. List the facts your answer depends on, such as which columns exist or what dates are covered, and ask for each one by name. Tell the AI to confirm each one in the file itself, and add the sentence the first brief used: "Where you cannot confirm, say so explicitly." Treat any fact it states that you did not ask it to confirm as unchecked until you look yourself.