A coding agent’s quality often declines not because the model has become less capable, but because the project lacks a clear operating system that governs what the agent reads, what it does, and how its changes are reviewed. As work extends over weeks and thousands of modifications, the agent may rediscover the same files, contradict a previous decision, guess the test command, or stop writing tests if these steps are not enforced.
In the sixth installment of the “Operating LLM Systems in Production” series, the Stack Overflow Blog proposes building an organizational layer around the model, including a concise project constitution, autonomy boundaries, multilevel context, persistent memory, a review committee, and a hook that runs after every modification. When multiple agents operate on the same repository, a file-ownership map and shared prompt and decision logs are added.
The Problem Is in the Operating Harness, Not the Model Alone
The article defines an agent as the model plus instructions, tools, context policies, hooks, isolation, subagents, and monitoring mechanisms. It cites a general result from the Terminal-Bench 2.0 benchmark, in which LangChain moved an agent from outside the top 30 into the top 5 by changing only the operating harness, raising the score from 52.8% to 66.5%—an increase of 13.7 points—while keeping the same model.
The practical takeaway is that incomplete tools, ambiguous rules, or overloaded context may be the most treatable causes of failure. Unlike the model, this layer can be modified by the development team and aligned with its repository.
Six Elements of a Disciplined Single Agent
- Project constitution: A file such as CLAUDE.md or AGENTS.md defines the workflow: planning, documentation, implementation, testing, review, and then committing. It should include quality standards and limits on the use of vendor packages, while remaining short because it is read in every session.
- Autonomy boundaries: The agent is allowed reversible operations such as reading, editing, testing, and creating commits, while pushing, deploying, and applying infrastructure changes require approval. The semantics of command matching must be verified so that broad rules, such as npm:*, do not accidentally permit deployment.
- Multilevel context: The root CONTEXT.md contains the project map and instructions for running and testing it, while each module’s file explains its responsibility, files, and dependencies. DECISIONS.md records decisions, their reasons, and rejected alternatives. Outdated context is considered a defect that should be fixed in the change itself.
- Persistent memory: The material proposes separate memory files for each fact, a concise index, and a graph.jsonl file linking modules and decisions through relationships such as depends on or replaces.
- Multilens review committee: Subagents review the change from the perspectives of product, engineering, infrastructure, security, and testing. A change should be blocked only by a blocker or a serious, high-confidence issue.
- Post-modification hook: PostToolUse runs after every Edit, Write, or MultiEdit and produces a checklist reminding users to update the context, memory, and decision log in the change itself, rather than silently rewriting files.
When Operating a Fleet of Agents
A new problem appears when multiple agents work on the same repository: modifying the same files or reversing one another’s decisions. The article does not recommend relying on direct conversations between them, but on written, auditable shared state, including a path-ownership map, CLAIM and RELEASE records, a progress log, and an independent branch for each task with human merging.
These claims are not a true lock; two agents may read the same state and claim the same path at the same time. When actual isolation is needed, the article recommends an atomic mechanism such as a uniquely named Git branch or a conditional write in a coordination store.
Why Does This Approach Matter?
The real change here is shifting the focus from the agent’s ability to write a function to the ability of the surrounding system to keep it directed and reviewable over time. The decision log serves a contractual role, not merely a documentation role: old decisions are not modified; new decisions are added to replace them, with the reason and rejected alternatives explained. This keeps the design history visible and allows it to be linked to automated checks, such as blocking certain imports.
This approach remains organizational guidance, not a guarantee of output correctness. The article also acknowledges that file-based claims are advisory when collisions occur concurrently, and that human review remains the control point for merging, pushing, and deploying. The material’s conclusion is that containing the model and building auditable traces may have a greater effect on long-running work reliability than waiting for a more intelligent model.