Chad Mathews
Oct 01, 2026Source: RL7 periodic-review.md, run 2026-06-21

The scheduled disagreement

The AI that helped you build it will not find what is wrong with it. One cold review with no memory of the work returned 47 findings, including two systems that had been quietly doing nothing.

A field note from Chad Mathews, drafted with Claude. Chad directs the work and lived these decisions; Claude does the writing; Chad edits every line and approves it before it publishes. Where an idea or a reference came from Claude, the note says so.

Ask the assistant that just spent six hours building something with you whether it is any good, and it will tell you it is good. Not because it is flattering you. Because it is holding the same assumptions you are. It helped choose them. Everything downstream of those assumptions looks consistent from inside the conversation, and consistent is what a review is checking for.

That is not a failure of the tool. It is a description of what it means to be in a conversation. Human reviewers have the same problem, which is why nobody sensible reviews their own pull request and calls it covered.

So I put a disagreement on the calendar.

Run it cold

The protocol is a document I keep in the project. It gets pasted into a completely fresh session, with one instruction: follow the reading order exactly, run the passes in order, and record every finding as you go.

Two rules do most of the work. First, fresh session, no history. The reviewer must not have watched the thing get built, must not know which parts I was proud of, and must not have been party to any of the arguments that produced the current design. Second, record findings as they happen, one row at a time, instead of writing a summary at the end. Summaries at the end sand off the specifics, and the specifics are the entire product.

The document also fixes the severity words in advance, with definitions, so that “critical” means “will crash or corrupt or produce plainly wrong behavior” and not “the reviewer felt strongly.” Deciding what the labels mean while you are looking at findings is how a review turns into a mood.

What one run actually returned

Forty-seven findings in a single pass. Two are worth describing, because neither one is the kind of thing a person playing the game would ever surface.

The first: my debug tools had been doing nothing for weeks. The on-screen state readout and the developer console both looked up the game’s central state object in a way that always came back empty in this engine version. So they silently did nothing. No crash, no error, no red text. They opened, they drew, they reported nothing, and I had been treating their silence as information.

The second: a brief window of invulnerability during a dash move was fully wired on both ends. The sender declared it, the receiver was connected and listening. Nothing ever actually sent the message. The window never existed. The dash still dashes, still looks right, still feels roughly right, and a playtest would have to be very lucky and very suspicious to catch that the protection was missing.

Neither of those is visible from playing. Both are trivially visible from reading. That gap is the whole reason this review exists, because reading the code is the thing I cannot do myself.

Where this sits relative to everything else

I already had a check before the work, a short preflight that loads the right context and names the known traps for that kind of task. I already had a check after the work, which is playing it and watching what actually happens on screen.

This is a third thing and it belongs to neither. It is not scoped to a task and it is not triggered by a bug. It is periodic and adversarial on purpose, and its job is the class of problem that has no symptom yet: the contradiction between two design documents that nobody has built from, the dead code, the wire that was never connected, the shortcut that is fine at this size and will not be at the next one.

Claude pointed out that the setup is close to how audits and red teams are staged, and that starting cold is a deliberate move against anchoring bias, the tendency to stay near whatever numbers and choices you saw first. I had not read any of that. I had just noticed that asking the assistant I had been working with all day never produced anything I did not already suspect.

The honest limit

A review you do not act on is theater, and forty-seven findings is not a score. It is forty-seven pieces of homework, some of which will still be open at the next run. The number going up is not progress and the number going down is not necessarily progress either.

The reviewer is also confidently wrong sometimes. In that run, two of the findings were real contradictions between documents that had already been settled by a later decision the reviewer had no way to know about. It flagged current truth as an error. Both went into the log marked historical rather than being quietly deleted, because the next run will trip over the same pair and I would rather it find a note than rediscover them.

And I cannot personally verify most of what a review like this tells me, which is the standing condition of how I build. What I can do is make it show its work: name the file, name the line, say what would break and under what conditions. Then I fix the findings that predict a visible symptom first, and check whether the symptom was really there.

Where this came fromThis note was extracted from RL7 periodic-review.md, run 2026-06-21, a working document from a live project, written to solve the problem before it was written to explain it.