I build Withmere, a personal AI environment that runs on my Mac. Agents write most of its code, and other agents review that code in a fresh session before anything merges. I am the only human in the loop, so the review loop needs a rule for when it is done.
Through the spring the rule was simple: after each repair, run a new independent review, and stop when no finding of medium severity or worse remains. It had the shape of Geoffrey Huntley’s Ralph loop, which reruns an agent until its work is done, applied to checking instead of building. Two parts of that rule were deliberate. A fresh session means the reviewer has not watched the repair being argued for, so it cannot be talked into accepting it. A severity threshold means style comments and small improvements do not hold a change hostage. A cap of eight rounds sat behind it as a backstop. The rule converged, usually within a few rounds, and I trusted it because each round found less than the one before.
In July it stopped converging. This is an account of why, of the first fix I made, and of the second one I needed because the first did not answer the real question.
Better reviewers, longer loops
Over the summer I moved review to a newer generation of models. The first commit co-authored with Anthropic’s Fable 5 landed on 12 June; on 10 July OpenAI’s GPT-5.6 models joined the configuration and became the default for my agents. That same evening I also tightened the loop’s procedure, defining what counted as a fresh round and stopping any repair that failed twice; the stop condition itself did not change.
The findings changed. They got finer, and when I checked them, most were real: a race when an export replaced the record it hung from, a missing connection between an outer adapter and the event store it was meant to read. Reviewers also began looking past the edges of what earlier reviewers had considered, into corners nobody had asked them to examine. I have not counted findings per round across the change, so this is my experience of reading the reports rather than a measurement.
On 13 July, a plan for a read-only view over the system’s memory went through three independent review rounds. Each round found real gaps, and each repair absorbed them. By the third round the plan required that one read-only view be complete across every current interface, every channel’s authentication, every capability and all legacy history. Nothing in it was wrong. It had simply become a different and much larger project. I stopped the loop by hand and reset the plan to its problem statement.
01 / The 13 July plan
Accept every true finding. Watch the plan.
Each gap below is real. Repairing it means writing it into the plan. Accept them in any order.
The problem
Unchanged
A read-only view over the system’s memory.
The plan
A different project
- Show the memory, read-only
- Complete across every current interface
- Complete across every channel’s authentication
- Complete across every capability
- Complete across all legacy history
The obvious response was to let the loop run longer. On 16 July I retired the mechanism instead.
A budget instead of a convergence test
The replacement fixed the amount of review in advance, by the worst thing a change could do: wrong text that a revert undoes, wrong behavior that rolling back the code restores, or lost data and irreversible actions that nothing restores. The higher the stakes, the more fresh review the change gets, and that is decided before the first finding arrives. After a repair, the reviewer rechecks the repaired surface; it does not start over. Exhausting the budget with findings still open sends the work back to diagnosis, not to another round. The rule now says it in one line: repeated review is not a repair.
02 / A budget fixed in advance
What is the worst this change can do?
The answer sets the review before it starts. Findings never buy an extra round.
Wrong text. A clean revert undoes it.
- 01 Plan review Not required
- 02 Acceptance review Required
- 03 Irreversible operations checked Not required
One fresh review at acceptance, then a targeted recheck of whatever was repaired.
Wrong behavior. Rolling back the code restores it.
- 01 Plan review Required
- 02 Acceptance review Required
- 03 Irreversible operations checked Not required
A fresh review of the plan before any code, and a full acceptance review after.
Lost data or an irreversible action. Nothing restores it.
- 01 Plan review Required
- 02 Acceptance review Required
- 03 Irreversible operations checked Required
As above, and each irreversible operation is checked against exactly what I authorized before it runs.
- After a repair
- Recheck the repaired surface. Do not start over.
- Budget spent, findings open
- Back to diagnosis. Repeated review is not a repair.
This stopped the loops. It did not tell me which findings mattered. A budget answers how much review a change deserves; it says nothing about what to do with a true finding still open when the budget runs out. The memory plan closed on 16 July without a clean review. Its final pass found the export race above, which I accepted as a known risk, and raised again a timeout concern that had already been ruled outside the plan’s scope. The reviewer had no way to know that. The ruling sat in that plan’s record as a one-off decision, and a fresh session, by design, had not read it.
What the stop rule had assumed
Looking back, “no finding of medium severity or worse” carried an assumption I had never stated. Severity is not a property of a code path alone. A shared file is a low-severity issue if only my own agents can write to it and a critical one if hostile software can. Every severity grade answers, without saying so, the question of who the attacker is and what they can do.
03 / One finding, three attackers
Every agent session can write to a shared file.
The code path stays the same. Change only who the reviewer assumes is doing the writing.
Only my own agents. Following instructions, occasionally wrong.
A reviewer’s grade
Low
A confused agent can corrupt the file. Rebuilding it repairs the damage.
Withmere’s declaration
In scope
If the file is ever treated as authority, that is a defect.
An agent steered by outside content. A web page or document carrying instructions.
A reviewer’s grade
High
Text the owner never wrote can now reach a file other components trust.
Withmere’s declaration
In scope
Declared: outside content never counts as authority. A gap here is a defect.
Hostile software running as my macOS user. Same account, same permissions as the agents.
A reviewer’s grade
Critical
It can write the file. It can also rewrite the code that reads it.
Withmere’s declaration
Not defended
Outside the declared actors. A finding here is a proposal to widen the scope.
I had never written that answer down. While reviewers stayed roughly within the threats I had in mind, the gap did not matter: their implicit attacker and mine were close enough. A stronger reviewer imagines a stronger attacker, and it is not wrong to do so. When it does, every finding is true under its model, the severity threshold is never met, and the loop runs until someone tires of it.
A stop rule like “no medium findings” quietly lets the reviewer decide the scope.
The August case
The case that made this concrete came in August. I was building a delivery tool that lets my agents put files in my iCloud Drive and send email to me. Its protections were deliberately narrow: a wrapper that parses each call, a capability granted per agent, a delivery that must trace back to a message I actually sent, and destinations limited to endpoints I had approved.
The product walk, the review step that exercises the feature as I would use it, came back with three findings. Two were ordinary defects in protections I had declared, and I fixed them. The third I could not dismiss on the facts. My agents run shell commands as my own macOS user. A hostile program running under that same account could bypass the wrapper and call the delivery code directly. The proposed repair was a global restriction on the shell available to every agent session.
The attack was real. But I had never decided to defend against it, and nothing at that layer could. Code running as my user can already edit the repository, import the internal modules, or start a detached background process. None of the usual hardening inside one tool (a broker process, a one-time token, a second approver) stops an actor that can rewrite the tool. Only a boundary enforced by the operating system can, and Withmere does not have one. The restriction would have cost every agent ordinary shell use without closing the attack, so I rejected it. The closing record says why: ordinary agent shell access is not a sandbox against a hostile caller.
That rejection showed me a second cost of an unwritten threat model. Besides the loops, it invites mechanisms that claim more than they enforce. Had I accepted the restriction, the system would have carried complexity in the name of a threat it still could not stop, and a security claim stronger than its enforcement. The review rule I wrote later gives that kind of mechanism its own name.
Writing the scope down
On 21 August I added a security section to the project’s charter, the document that sets its first principles, and a rule for component threat models. I wrote down the reason at the time: the delivery tool’s review had shown how a local design gets pulled along by a system-wide threat model nobody had adopted.
The system-level declaration is short. Withmere commits that model output and external content never count as authority by themselves; that replayed or stale requests, mixed identities and partial failures do not succeed silently; that secrets stay out of the repository and ordinary prompts; and that uncertain results are reported as uncertain. It also states what it does not defend:
Not defended:
- the owner, root, or a compromised host
- arbitrary hostile code running under the same OS identity
- cryptographic provenance or strong process isolation between
agents and processes on the same account
- multi-tenant isolation
- what a model provider or transport does with data already sent to it
A component that changes a trust or consequence boundary now declares what it protects, which actors are in scope and with what access, where enforcement sits, and which actor, capability and property combinations it explicitly does not cover. Review then reads findings against that declaration:
| Finding | Classification | What happens |
|---|---|---|
| A declared property lacks effective enforcement | Defect | Fixed within the change |
| A mechanism serves only an undeclared, stronger actor | Architecture finding | Candidate for deletion |
| A plausible attack outside the declared actors | Threat-model expansion proposal | I decide, with its system-wide cost |
| The declaration itself looks wrong | Product judgment | Raised separately from defects |
The third row is the one that changed the loop. An out-of-scope finding is not discarded, and it is not a blocker either. It becomes a proposal to widen the scope, which states the property at risk, the system-wide cost and the boundary that is missing. Accepting it is a decision about the system, made once, rather than a repair argued again in every component.
04 / The August product walk
Three real findings, read three ways.
The same report from 15 August under the rule I started with, the budget I actually used, and the scope I wrote six days later. Start on the left.
July stop rule. No medium-or-worse finding may remain
- F2 Usage text bypassed the wrapper
The tool’s generated instructions told agents to run it through bash, contradicting the wrapper that parses every call.
BlocksRepair, then review again.
- F3 Email not tied to my message
A sent email was not recorded as caused by a specific message from me.
BlocksRepair, then review again.
- F1 Same-user code skips the wrapper
A hostile program running as my user could call the delivery code directly. Proposed repair: restrict shell access for every agent.
BlocksRestrict every agent’s shell, then review again.
The loop stops when all three are gone, including the one no repair inside this tool can close.
Budget, as it happened. Fixed review, then my judgment
- F2 Usage text bypassed the wrapper
The tool’s generated instructions told agents to run it through bash, contradicting the wrapper that parses every call.
FixedTargeted recheck passed.
- F3 Email not tied to my message
A sent email was not recorded as caused by a specific message from me.
FixedTargeted recheck passed.
- F1 Same-user code skips the wrapper
A hostile program running as my user could call the delivery code directly. Proposed repair: restrict shell access for every agent.
RejectedI rejected the restriction as too strict. The reasoning lived in one closing record.
The review stopped on budget. Whether F1 counted was decided once, by hand, for this tool only.
Written scope. Findings read against a declaration
- F2 Usage text bypassed the wrapper
The tool’s generated instructions told agents to run it through bash, contradicting the wrapper that parses every call.
Declared: every call is parsed by the wrapper.
DefectA declared property lacks enforcement. Fixed within the change.
- F3 Email not tied to my message
A sent email was not recorded as caused by a specific message from me.
Declared: every delivery traces to a message I sent.
DefectA declared property lacks enforcement. Fixed within the change.
- F1 Same-user code skips the wrapper
A hostile program running as my user could call the delivery code directly. Proposed repair: restrict shell access for every agent.
Not declared: hostile code under my OS identity.
ProposalAn actor outside the declaration. I decide on the system, with its cost.
Two defects block the change. The third finding is kept, and it is routed to a decision rather than another round.
The rule met its first large review in late September, on a tool that serves local files privately to my phone. Its budgeted review had passed in the early hours of 24 September; using the tool from my phone later that day exposed failures the review had not caught. After those were repaired, a full audit reported, among real defects, two findings that assumed stronger attackers: hostile code running as my user reading the tool’s access token, and one shared view reading another. Both were classified as proposals to widen the scope and left unrepaired. The real defects were fixed, two further fresh audits found and closed more gaps, and the final one left nothing open. That was more review than the budget asks for, and I chose it because real use had just shown the review missing things. What the extra rounds never did was argue again about who the attacker was; the scope had settled that in the first audit. One case is not a measurement. It is the first time the rule did what the August case had needed.
The price is in writing
The cost of this approach is that I have to say, in a document anyone working in the repository can quote back to me, what I do not defend. That is uncomfortable in a way a review loop never was. A loop that keeps running looks like diligence. A list of non-goals looks like a list of weaknesses, and each entry is a decision I own if it turns out to be the wrong one.
It is also more honest. The same-user attacker was never defended against; the earlier design simply did not say so, and a sufficiently capable reviewer was the first party to notice. The declaration also has to stay alive. Each component model names the evidence or architectural change that would require revisiting it, so a non-goal is a current position rather than a permanent exemption.
The same failure, elsewhere
Two engineers at Cockroach Labs recently wrote about running coding agents like a hospital, treating bugs as patients and giving planning, coding and review to separate agents. In the Hacker News discussion, one of the authors described a plan reviewer that pushed “a technically valid but negligible security finding through five review rounds and about $70 before a human closed it as won’t-fix.” Their fixes were mostly about “giving a stage permission to stop”: a cost ledger that turns such loops into numbers, and a plan reviewer that may now propose won’t-fix when the exposure is negligible.
That is the same first step I took with a fixed budget, reached by a team moving customer data rather than one person building for himself. What the comment does not say, and what my budget also left open, is what makes an exposure negligible. If it is judged finding by finding, each won’t-fix is a fresh argument, and the next reviewer cannot see the reasoning behind the last one. A written scope turns those judgments into a position that accumulates.
The reviewers will keep improving, and I want them to. A reviewer that finds attacks beyond my model is useful precisely because it sees what I did not decide. What has changed is where its findings go. It can still find the attack. It can no longer decide that I defend against it.