The Intervention Penalty: costs of AI agent checkpoints
- My contribution
- I wrote the initial draft; Sofia Plajutin edited.
- Scope
- April 2026 preprint · a bounded simulation hypothesis about human checkpoints.
- Evidence
- Paper and goals-harness material; no verified public simulation dataset.
Page by Julian Ramirez ·
An approval request is short. Answering it may not be. You have to stop what you are doing, reconstruct the task, understand the proposed action and decide whether it fits the objective. The visible click is only the end of that work.
The Intervention Penalty examines a bounded version of this problem: whether frequent human checkpoints can degrade AI coding performance when capability is sufficient for the task. The paper formalizes a hypothesis and explores it through simulation. It does not establish a measured productivity gain for a deployed engineering team.
I wrote the initial draft; Sofia Plajutin edited. The question behind this study is how to think about supervision without treating every interruption as free.
A checkpoint costs more than its response time
The accompanying goals-harness material separates several kinds of burden: context switching, communication, delay and friction. They describe different parts of the same pause. A request can be technically simple while still arriving at an expensive moment for the person receiving it.
checkpoint Delay Friction
Sources & implementation notes
Those categories help explain the hypothesis, but they are not measured quantities in this article. In particular, the public repository does not provide a verified simulation dataset from which to reconstruct empirical curves. The visuals here are schematics, not experimental results.
What changes when checkpoints become frequent?
The paper explores whether intervention frequency can become a stronger contributor to modeled task degradation than residual capability differences. That is a conditional claim: it depends on the simulation’s assumptions, including a setting where the agent has sufficient capability.
The comparison below changes the number of pauses in an illustrative work sequence. It deliberately does not attach minutes, percentages or an invented productivity score to them. Its purpose is to make the modeled mechanism legible.
Sources & implementation notes
The diagram should leave you with a question about your own workflow, not a multiplier to apply to your team’s output: which checkpoints produce a decision that actually needs a person, and which ask them to restate criteria that could already be explicit?
Move repeatable judgment into acceptance criteria
Criteria-based governance changes where some judgment happens. Instead of repeatedly asking whether an action looks acceptable, a workflow states what acceptable work must satisfy and gathers evidence against those criteria.
That still requires design effort. Someone must choose useful requirements, decide what can be checked automatically and define how uncertainty is escalated. The comparison below shows the difference in placement, not proof that either arrangement is always better.
Sources & implementation notes
The goals-harness README makes stronger claims about eliminating the penalty. I would keep those claims separate from the available evidence. A proposed mechanism and a simulation finding do not establish its effectiveness in production.
Chest is a related implementation of explicit criteria and reports. The paper does not validate Chest, and Chest’s fallback behavior needs its own inspection.
Keep consequential decisions attached to accountable people
Reducing routine interruptions is compatible with retaining human decisions at meaningful boundaries. Architectural security choices, exceptions and irreversible consequences need owners who understand what authority they are exercising.
Sources & implementation notes
The useful outcome is a workflow in which supervision has a clear purpose. A checkpoint should resolve something: an ambiguity, a consequence or a decision beyond the agent’s authority. When it cannot, the better intervention may be to improve the task’s criteria before the work begins.
Plot’s exact-target approvals offer another perspective on placing judgment at an explicit boundary. Both ideas invite inspection of where a decision belongs; neither makes accountability disappear.