AI applications built in Rust are gradually moving from initial experiments toward more practical models, according to a live demonstration organized by JetBrains in cooperation with the Rust Foundation. The first session focused on the open-source Rig library and how to use it to build applications based on large language models and agents capable of calling tools and performing tasks within the application.
Orhun Parmaksız, a Developer Advocate, demonstrated a small coding agent he built using Rig and Ratatui, while Stephen Korzeniewski, the lead maintainer of Rig at 0xPlaygrounds, explained the project's internal architecture and the choices behind its APIs.
A Unified Interface for Model Providers
Rig addresses a common practical problem in large language model applications: OpenAI, Anthropic, Gemini, and others each have different APIs, even when some services claim compatibility with the OpenAI API. Rather than connecting parts of the application directly to one provider's interface, Rig provides a unified Rust interface through which the provider can be switched without rewriting the model-related parts.
The library organizes the application around several core components, including the provider client, the completion model, the agent, and the tools available to it. The agent operates as a layer above a direct model request; it adds the instructions, token limits, and capabilities needed to perform tasks that go beyond an input-text-to-output-text flow.
Agent instructions are passed in what Rig calls the preamble, instructions that serve the role of a system prompt and are included at the beginning of every request. Network communications with model providers are managed through asynchronous interfaces, while many details of asynchronous operation remain inside the library so that application code can focus on the desired configuration and behavior.
Tools Make the Agent Part of the Application
Rig defines a tool through Rust's Tool trait. A tool definition includes its name, input, output, and error types, along with the function executed when it is called. Its definition also describes the expected arguments using JSON Schema, giving the model a structured description of the data it can provide.
After a tool is registered with the agent, a model that supports tool calling can determine when to use it and incorporate its result into the response. The function may perform a calculation, query a database, retrieve information, or connect to another service. A tool can also receive context that preserves state such as the number of calls or cached results. The idea extends to agents themselves: one agent can be exposed as a tool to another agent so that tasks can be delegated between them.
From Demonstration to a Testable Application
The Rat Code project brought these components together in a small coding agent that runs in the terminal using Rig and Ratatui. The main.rs file initializes the provider client, model, and agent, while capabilities for reading and writing files and running shell commands are defined separately and then registered with the agent through Rig's tool interface.
The application also uses Rig's streaming interface to process the response in chunks and update the terminal interface while the model generates its answer. The demonstration also covered the hook system, recovery from tool calls, and the design choices behind the abstractions provided by the library. The full code walkthrough begins in the recording at 27:28.
RAG, Local Models, and Testing
Rig is not limited to hosted model providers. It supports retrieval-augmented generation applications, in which documents and user queries are converted into vector embeddings, after which the semantically closest documents are searched for and added to the model's context. The library provides integrations with databases and abstractions for vector stores, with the ability to implement the vector-store interface when using a database that is not directly supported.
Local models can also be run through Ollama and llama.cpp, or the rig-candle integration can be used to perform inference directly inside a Rust application. This option makes it possible to embed model weights in the application and run supported models through WebAssembly, without relying on a hosted model API or a separate local inference server.
The importance of these options becomes apparent in where models and data are run, but they also present an integration-testing challenge. Rig primarily relies on a recording system: tests are run with real providers, HTTP traffic is saved in YAML files, and a mock server replays the requests and responses in continuous integration tests. The project contains approximately 1,700 recorded interactions, and these interactions are replayed with every pull request within a few seconds.
These tests verify that the software integration continues to work, but they do not measure the quality of the model's outputs, which may change when a provider updates its model. Therefore, the demonstration referred to scheduled tests on live models as a separate approach for applications that depend on answer quality.
Editorial Takeaway: What Matters to Developers?
The practical value of Rig lies not merely in adding a new provider, but in separating the application layer from the details of model interfaces while keeping tools, retrieval, and local inference within a single Rust architecture. This may reduce the cost of changing the provider or operating method, but it does not eliminate behavioral differences between models, nor does it solve the problem of evaluating output quality on its own. The project's testing mechanism shows that ensuring the connection works is one thing, while verifying that the model produces the desired result is another matter requiring live tests and independent evaluation criteria.