AI context engineering consists of designing everything surrounding an agent in terms of information, boundaries, and instructions, so that it knows what it can see, what it should do, and how to behave when it encounters a new situation or incomplete data. According to Doug Whitley, Director of Engineering at Stack Overflow, and Ash Zade, Director of Product, the goal is not to overwhelm the agent with all available data, but to provide it with the specific context it needs to complete a task with a predictable outcome.
This came in a conversation published by the Stack Overflow Blog as part of the “No Dumb Questions” series. It discussed the differences between context engineering, context infrastructure, and context engineering, as well as the relationship of these concepts to RAG and MCP, and why companies turn to buying ready-made solutions instead of building everything internally.
From Infrastructure to Engineering
Whitley distinguishes context infrastructure, which concerns how context is stored, displayed, and delivered to the agent, from context engineering, which focuses on designing the system and the reasons for organizing its components in a particular way. Context engineering in the implementation sense concerns actually building the system, such as choosing the algorithms, programming languages, and tools used.
He places RAG in an area that brings together all three levels. Indexes, context stores, and search methods represent the infrastructure aspect, while building the system in a language such as .NET, Python, or Rust represents the engineering aspect. By contrast, the architecture determines the system’s overall form and the rules governing how it operates. Whitley views MCP as a protocol with specific requirements, but the decision to integrate it into the system, the languages used, and the features it will support are all design decisions.
Reducing the Range of Decisions an Agent Makes
Zade explains the idea using the example of searching for car tires. If an agent is sent to a library to search for “tires,” it may find information about aircraft, bicycles, and wheelbarrows because they all match the requested word. A well-designed system, however, establishes from the outset that the task concerns car tires, or specifically sports-car tires, and limits the available information to that scope.
Zade believes that placing these instructions in a text prompt alone does not guarantee that the agent will follow them. Context engineering therefore requires controlling the data the agent can access, in addition to defining what it should do when it encounters incomplete or incorrect information. In this way, some variables are removed from the agent’s decisions instead of leaving it to decide for itself whether information is reliable or whether it should broaden the search.
The system also includes the agent’s memory, allowing it to retain what it has accomplished, learned, discussed, and built so far. This memory becomes more important when multiple agents are working, or when a task stops and resumes later.
Trust, Permissions, and Human Intervention
Stack Internal, according to the speakers, provides an example of connecting context to a trust system and an expert-verification workflow. Knowledge is classified as high, medium, or low confidence. If the confidence level is medium or low, the user can be directed to a subject-matter expert to verify the information or complete the gaps, rather than having the agent make a decision based on uncertain knowledge.
Data protection goes beyond asking whether the agent can see a particular piece of information. The information may be available to the user but unsuitable for the task the agent is performing. For this reason, the speakers describe two levels of control:
- Source permissions: The agent inherits the user’s permissions in the systems it searches, such as Slack, MS Teams, Google Drive, or SharePoint.
- Scopes: The user can narrow the range of data available to the agent even if the user has permission to access a broader set of data.
- New data: The system can restrict what the agent creates or returns, so that it sends it to the user first instead of automatically adding it to the general knowledge base or sharing it with the team.
This distinction indicates that context management is not limited to retrieval; it also includes controlling the memory and knowledge that the agent creates.
Why Might a Company Buy the Solution Instead of Building It?
Whitley says that building context engineering is possible, but the challenge is not limited to writing code. When a company collects data from Slack, MS Teams, Google Drive, SharePoint, Confluence, GitHub, and Jira, it must determine how to index, rank, and rerank it, and how to handle conflicting, incomplete, or incorrect information.
Zade believes that a large part of the work involves defining trust itself. An expert user may trust AI outputs when they align with their expectations and previous experience, but this approach is not equally available to someone who lacks expertise in the field in which they are asking AI to work. The system therefore requires discussions about what makes information trustworthy, not merely technical solutions for collecting it.
Whitley adds that buying a ready-made solution may give a company the accumulated experience of problems encountered by other customers, including edge cases and everyday issues that he says his team categorizes into approximately 20 categories. Nevertheless, the speakers do not rule out building internally; it may be appropriate when the use case is new or when the company needs its own specific design decisions.
For Whitley and Zade, the quality of context engineering is measured by the system’s flexibility and its ability to provide consistent, predictable outputs, not merely by narrowing its scope to a single task. Filtering information early can also reduce the number of tokens the agent processes: searching car books is less costly than searching an entire library, and searching tire pages is less costly than reviewing all the books. The ultimate goal remains to enable the agent to perform the expected task and to stop or request human assistance when it reaches limits it should not cross.