Three overlapping disciplines, three sets of tools, one confused buyer. MLOps, LLMOps, AgentOps — they share goals but differ sharply in the artefacts they manage, the failure modes they prevent, and the tooling the team actually needs. Most platform teams in 2026 are buying tools from one and trying to do the job of all three.
Below is the honest, plain-language breakdown of what each one actually does — and how to tell which one your team should be staffing.
MLOps: the disciplined classical-ML factory
MLOps is the ten-year-old discipline of running classical machine-learning models in production. The core problems it solves:
- Repeatable training: a model can be re-trained from scratch given the same data and code.
- Model versioning: every deployed model traces back to a checkpoint, training run, and dataset.
- Feature consistency: training and serving use the same feature definitions.
- Drift detection: feature drift, prediction drift, label drift detected and alerted on.
- Scheduled retraining: a model that has aged past its drift threshold gets re-trained automatically.
- Rollback: a bad deployment can be swapped back to the prior version in seconds.
The tools are well-established: MLflow or Weights & Biases for tracking, Feast for feature stores, KServe / SageMaker / Vertex AI for serving, Evidently / WhyLabs / Arize for monitoring, Argo or Kubeflow Pipelines for orchestration.
The MLOps stack is mature. If you are running classical ML in production and your platform looks like a series of one-off Python scripts and notebooks, the problem is staffing and discipline, not tooling availability.
LLMOps: managing what you do not own
LLMOps is the operational discipline for systems where the model is an LLM — often one you did not train. The shape of the problem changes:
- You do not own the model weights. Versioning is mostly versioning the prompts and the upstream provider’s API version.
- The non-determinism is intrinsic. Outputs change between calls; evaluation must be statistical, not exact.
- The cost surface is tokens, not GPU-hours. A bad prompt change can 5× your monthly bill silently.
- Latency is unpredictable: provider-side load, retries, model upgrades all affect it.
- Drift is now in the prompt, the retrieval corpus, and the model — three sources at once.
- Hallucination, jailbreaks, and prompt injection are new failure modes with no analog in classical ML.
The artefacts LLMOps must manage:
- Prompts. Versioned, diffable, reviewable as code.
- Evaluations. Golden sets, automated metrics (faithfulness, groundedness, refusal-rate), human-in-the-loop scoring.
- Retrieval corpora. Snapshots, freshness, embedding model version pinning.
- Cost and latency budgets. Per-request, per-feature, per-tenant.
- Guardrails. Input filters, output filters, content policy enforcement.
The tooling landscape: LangSmith, Helicone, Phoenix (Arize), Galileo, Patronus, TruLens, Weights & Biases Weave, LangFuse, Maxim AI. The market is still consolidating; most teams use 2–3 tools together.
Critical: LLMOps does not replace MLOps. If you fine-tune your own models, you still need the MLOps stack underneath. If you call only third-party APIs, you mostly need LLMOps. Most enterprises in 2026 need both.
AgentOps: the newest, hardest layer
AgentOps is what you need when an LLM is not just producing an output — it is calling tools, taking actions, and chaining steps. The problems multiply:
- Execution traces are now graphs, not single calls. Debugging means replaying step-by-step.
- Tool calls have side effects. A bad agent run can update a database, send an email, or move money.
- Memory state matters. Two runs that started with the same input can diverge based on memory.
- Cost and latency become path-dependent: an agent that decided to call three tools costs more than the same agent that called one.
- Failure modes are new: hallucinated tool arguments, infinite loops, tool-result misinterpretation, multi-step misalignment.
The discipline AgentOps requires:
- Trace observability. Every step of every run captured, with inputs, outputs, tool calls, and decisions.
- Replay and time-travel debugging. Step through a failed run and re-execute from any point.
- Tool-call validation. Schema-enforce every tool invocation; reject malformed calls before they execute.
- Action authorisation. Policy gates between “the agent decided to do X” and “X actually happens.”
- Loop detection and circuit breakers. Stop the agent before it spends $400 in tokens chasing its own tail.
- Multi-agent coordination. When your agents talk to each other, observability has to scale across them.
Tools: LangSmith for traces (still the most mature), Phoenix, Langfuse, Maxim, Galileo Agents, and custom tracing built on OpenTelemetry. The category is two years old; expect rapid consolidation.
Side-by-side
| Dimension | MLOps | LLMOps | AgentOps |
|---|---|---|---|
| Primary artefact | Model weights | Prompts + retrieval | Execution graphs |
| Versioning unit | Model checkpoint | Prompt + model version | Agent config + tool registry |
| Evaluation | Accuracy, AUC, F1 | Groundedness, faithfulness | Task success, step-level correctness |
| Cost driver | GPU-hours, training | Tokens, retrieval cost | Tokens × path length |
| Worst failure | Silent accuracy drop | Confident hallucination | Wrong action taken |
| Observability | Metrics + logs | Traces + evals | Replayable graphs |
| Maturity | High | Medium | Low |
Which one does your team actually need?
A simple decision tree:
- If you train your own models in production: you need MLOps. No exceptions.
- If you call any LLM API in production: you need LLMOps on top of (1).
- If your LLMs take actions, use tools, or chain steps: you need AgentOps on top of (1) and (2).
Each layer adds complexity. Each layer’s tooling is younger and less mature than the layer beneath it. Most enterprises in 2026 are skating on thin LLMOps and almost no AgentOps — and the next major AI incident someone publicly suffers will probably be at the AgentOps layer, where the gap between sophistication and tooling is widest.
The pragmatic staffing pattern
For a 200–500 engineer organisation in 2026:
- 1–2 dedicated MLOps engineers if you have classical ML in production. More if you have 10+ models.
- 1–2 LLMOps-savvy engineers embedded in the AI product teams. Currently rare; train from inside.
- AgentOps is a capability, not a role yet. Senior engineers learning by doing, with observability tooling carrying the load until the discipline matures.
The most expensive mistake we see is treating these three as the same role. They are not. The tools, instincts, and failure modes diverge sharply. Staff them deliberately, scope each layer’s ownership, and review the boundaries every six months as the disciplines evolve.