Engineering systems based on large language models has witnessed a radical shift in production environments, moving from mere random texts and prompts linked with simple relational codes to structured execution frameworks known as Agentic Harnesses. This evolution aims to surround probabilistic models with the necessary software support to ensure dependability, reliability, and observability in real-world operating environments.
Phase One: Chains and Simple Linear Operations
Early systems relied on a simplified linear workflow known as chains. This approach is characterized by fast execution and low computational cost in narrow tasks such as simple summarization. The idea is based on passing the output of the first step to be the input of the second step, and so on, using models as black boxes that receive and produce texts.
For example, LinkedIn used fixed chains in its early stages involving classification and routing, then data retrieval via search tools to capitalize the response, and finally generation and filtering. Despite rapid development, these systems suffered structurally from fragility; any error in an early step causes the entire workflow to crash due to the lack of response and automatic correction mechanisms.
Phase Two: Graphs for Enhanced Structure and Control
As use cases became more complex, developers moved towards using graph structures to introduce conditional branching, parallelism, retries, and state management. Nodes in these structures represent various tasks, while edges govern the control path.
These structures enabled the design of multi-agent systems with a primary coordinator managing task flow and asynchronous messaging to ensure ordering and continuity. These architectures were used to launch the Hiring Assistant to enable recruitment officers to automate routine tasks. However, these frameworks remained restricted by human-defined workflows, making them complex to maintain as exception cases scale.
Phase Three: Programming as a Core Reasoning Tool
The latest evolution relies on code instead of visual language prompts as a core tool for reasoning. Rather than merely interpreting problems textually, these structures generate and execute code in isolated and tested environments.
Code is characterized by its deterministic, verifiable nature compared to the non-guaranteed probabilistic outputs of language models. This architecture relies on an isolated execution environment, logging and monitoring tools, automated verification structures, and iterative debugging loops that correct errors and re-run codes until they achieve the required standards. LinkedIn applied this approach internally to enable employees to query data via structured query languages and Python according to privacy and governance frameworks.
Best Practices for Agentic Systems Engineering
- Treating outputs as untrusted data: The necessity of validating model outputs before passing them to any subsequent software logic.
- Separating planning from execution: Allowing the model to propose plans while a strict software framework handles the execution process.
- Investing in observability: Accurately tracking execution instances to review errors and state logs without compromising sensitive data.
- Continuous development: Building analytical feedback loops to improve system performance over time.
Choosing the Right Engineering Abstraction
Not every technical problem requires the operational complexity of graphs or advanced program codes. Simple chains remain the most appropriate and least costly choice in terms of time and computational capabilities for direct tasks such as summarization and content processing. Meanwhile, graphs become necessary when branching and state control are required, and programming frameworks are used in critical areas that require strict precision and verification such as software engineering and complex data analysis.