Guide · Already building with AI

How to have separate AI helpers check each step of your work

At each major step, give the results to fresh AI sessions whose only job is to find what was missed, check what they say, and record what you accepted and why before moving on. How to do it with ordinary chat windows, and what happened when an AI research run did it.

I'm Tyler Orden, an independent researcher who does mathematics with AI agents. My research record is at Cyborg Mathematics.

When you ask the same AI chat that did a piece of work to check it, you mostly get agreement, because the check runs on the same conversation and the same assumptions. A fresh session that sees only the result, with one job, find what was missed, starts without that conversation. It still has the model's own blind spots, though, so for at least one checker use a different model or a person.

To do this with ordinary chat windows: when you finish a step that matters, such as a draft analysis or a plan, open a new chat with memory turned off (ChatGPT calls this a Temporary Chat; other tools have a similar setting), so it can't see your earlier conversation. Use a different AI model for one of your checkers. Paste in only the result and one sentence on what it is meant to show, plus this:

Audit instruction

Audit this independently. Identify anything the person who did this has missed: assumptions that have not been tested, data that should have been checked, alternative explanations, problems with the method, or other work that should be done before going further. For each point, say how I could check it.

Then check what comes back, because helpers make things up too, and write one line per point in a log:

Decision log, one line per point

The log is what lets someone check your conclusion later, including you in six months. It shows which points changed the work and which were waved off, and why.

I used this on an AI research project about Los Angeles parking tickets, led by Claude Opus 4.8, the AI agent I picked to run it. From the start the lead sent out helper AIs to check its ideas. Late in the run I made it a rule: I described what I wanted to ChatGPT, it wrote the rule, and I gave the rule to the lead. My request to ChatGPT read: "on every single stage ask it to join a couple sub agents to look into every stage and see if there's anything we missed". Each helper was a separate AI session that got one written set of instructions and never saw the lead's conversation; its report was saved word for word.

The tally the record supports: eleven helpers ran. One helper reported that two columns were missing from the city's ticket data; the lead queried the live database, found both, and overruled it in writing. One helper's review changed the analysis plan (below). The helpers searching for earlier work weakened both of the run's leading ideas: for the gap between an officer marking a parked car and ticketing it, one found a 2012 patent on working out officer revisits from ticket times; for the timing of street-sweeping tickets, another found the pattern already shown by public projects, including one in San Francisco. The record does not count how many individual points the helpers raised, so there is no hit rate. Each timed check took four to six minutes; the money cost was not recorded.

The helper that changed the plan was told to try to prove the planned street-sweeping analysis couldn't show what it claimed. It found that the lead's main measure, the jump in tickets the moment sweeping starts, was guaranteed by the rules: no sweeping ticket can be written before the window opens. It also warned that the rise in tickets after the start might come from mixing many streets. As a made-up illustration, if officers reach one street at 8:00 and another at 8:20, and tickets jump sharply on each, the two added together look like a gradual rise. The lead dropped the jump as a measure and logged that the rise could be an artifact until checked street by street.

In this run I let the lead agent make the accept-or-reject calls, and the record doesn't show me reading each report as it arrived. Nobody outside the run reviewed those calls while it ran. If the conclusion matters to you, read the decision log yourself before acting on it.

For teams: the same setup works with people. The project lead picks one reviewer per major step, someone who didn't share the author's assumptions, and the review happens before the next step starts, not at the end. The decision log lives in the project's shared folder or ticket, and the lead fills it in before moving on.

For agent builders: the rule the lead agent got

For EVERY major stage, spawn approximately 2-4 independent Sonnet-class subagents. Their job is NOT merely to perform the same task. Their explicit instruction is: "Audit this stage independently. Identify anything the principal investigator has missed, any assumptions that have not been tested, any additional data sources, methodological problems, literature we have not searched, alternative interpretations, statistical problems, reproducibility problems, or other work that should be completed before proceeding." Record: how many agents, their roles, the prompt given to them, what each investigated, what each found, which findings were accepted, which were rejected, why, what changed because of them.

In that rule the "principal investigator" is the lead agent and "Sonnet-class" means a smaller Claude model. What these checks ended up saving the project is in the lesson "What quick AI checks caught before a project's expensive step, and when to stop adding them."