Running a large language model in a production environment is no longer limited to training, testing, deploying, and monitoring dashboards. A modern system may connect prompts, vector databases, and knowledge sources, then produce open-ended text evaluated for tone, safety, and confidence, in addition to accuracy. This is why Daniel Bryant, in an article published on the CNCF blog, poses a practical question: who should own the AI pipeline?
The author argues that the answer does not lie in giving LLMOps a separate kingdom, but in integrating it into a well-organized engineering platform that exposes the required capabilities through the same interfaces development teams use for the rest of their workloads.
What Is LLMOps?
LLMOps refers to the set of practices, tools, and workflows required to develop, deploy, and manage large language models throughout their production life cycle. This life cycle includes data management, prompt engineering, fine-tuning, deployment and inference serving, monitoring and evaluation, as well as security and governance.
According to the analysis, LLMOps is not simply a rebranding of MLOps. Large language models are more expensive to fine-tune and serve, and evaluating their outputs is more difficult than reducing performance to a clear accuracy figure. It is not enough for a model to be accurate; it must also be safe and trustworthy, characteristics that are more complex to measure.
Operating the model also does not end with the initial deployment. Models may drift from their previous behavior, costs may rise, prompts may stop working as usual, and integrations with customer relationship management systems or internal knowledge bases require continuous monitoring.
A Life Cycle That Intersects with Platform Engineering
The LLMOps life cycle extends from data preparation and prompt engineering, treating prompts as versionable artifacts rather than temporary text, to fine-tuning open foundation models using libraries such as Hugging Face Transformers. It also includes versioning models and prompts and tracking their lineage, providing inference through GPU-backed endpoints, and monitoring based on human feedback to detect drift and cost.
Every part of this life cycle requires infrastructure, access controls, and a runtime environment—areas that already fall within the scope of platform engineering. However, the author distinguishes between the two domains: platform engineering centers on infrastructure, while MLOps centers on models. Therefore, he does not see the useful question as “Who owns the pipeline?” but rather as “Who owns each layer, and is there an entity that actually coordinates them?”
The Risk of Creating a Parallel Stack
The analysis divides the software delivery landscape into DevOps teams that may be overwhelmed by deployment requests, platform engineering teams that build standardized self-service workflows, and MLOps teams that created parallel stacks because DevOps tools were not designed to manage data versions or monitor drift. With the addition of LLMOps, a third independent stack may emerge for prompts, vector databases, and RAG pipelines, separate from the party responsible for governance.
The author connects this possibility to the problem of “shadow AI operations.” One team may create its own RAG pipeline connected to an unreviewed vector database, without the responsible party having a clear view of what is actually running. According to the analysis, the greatest operational risk is not merely a chatbot producing hallucinations, but the spread of these capabilities outside the platform and their subsequent removal from the scope of visibility and control.
The author does not suggest slowing teams down to prevent this, but rather making the platform capable of meeting requests quickly while embedding governance into the usage path itself. The absence of a ready-made path may push teams to build their capabilities outside the platform, whereas providing a standardized path helps bring those capabilities back into a manageable environment.
LLMOps Within the Platform Layers
Bryant draws on the platform white paper published by CNCF’s TAG App Delivery group, which divides the environment into three layers: products at the top, platforms in the middle as the thinnest reasonable integration layer, and capability providers at the bottom.
Under this model, functions such as fine-tuning jobs, vector databases, prompt logs, and inference endpoints can be treated as another platform capability. Like other capabilities, they need APIs, versioning, and clear ownership. The author points to tools within the CNCF ecosystem that can support this model: Backstage presents standardized workflows at the product layer, while Crossplane assembles infrastructure at the lower layer, and frameworks such as Kratix, KusionStack, and KubeVela operate in the middle to expose an LLM pipeline through a self-service interface similar to that used by other services.
Practical Controls for Pipeline Governance
The analysis suggests that platform teams adopt a set of practical controls:
- Governed APIs instead of informal scripts: Fine-tuning jobs, prompt deployments, and inference endpoints should be requested through the same self-service interface used for the rest of developers’ needs.
- Policy enforcement at request time: Cost limits, data residency rules, and model access controls should be checked before the job begins, not after the cloud services bill appears.
- Human approval proportionate to impact: Not every prompt needs prior approval, but a model handling customers’ personal data or making autonomous decisions may require it.
- A clear audit trail: The record should answer questions such as what changed and why, whether the change involved the model, prompt, or data, and who approved it or what was approved.
The author concludes that an LLM pipeline is another automated consumer of platform capabilities and needs the same safeguards provided to a human developer or an autonomous agent. Teams that succeed with this model do not choose a side in the dispute among DevOps, platform, and MLOps; instead, they treat the entire pipeline as a versionable, monitorable, and cost-conscious product with continuous feedback loops.
In this sense, Bryant does not see LLMOps as eliminating or reinventing the question of ownership, but as placing existing platforms under greater pressure because of the models’ size, high cost, and evaluation difficulty. In his view, the solution is to build the standardized path once, then expose it as a governed capability to developers, data scientists, and intelligent agents through a shared API or user interface. He notes that this discussion is ongoing within CNCF, particularly in the Platforms Working Group of TAG App Delivery, which is open to anyone who wants to contribute.