# What is LLMOps? How it differs from MLOps, with a checklist

URL: https://sdclabs.com/glossary/llmops/

> LLMOps runs LLM apps in production: prompt and model versioning, evals, tracing, cost and model migrations. How it differs from MLOps.

# What is LLMOps?

[Sergey Almyashev](https://sdclabs.com/author/almyashev/) 2026-11-21

Definition

LLMOps is the practice of running applications built on large language models in production: versioning prompts and model IDs, testing every change with evals, tracing cost and quality on live traffic, and migrating before a provider retires a model. It adapts MLOps to apps where most teams call a hosted model instead of training one.

## What does LLMOps stand for, and what does it cover?

LLMOps stands for large language model operations. Microsoft's AI playbook describes it as the tools and processes that manage developing, deploying and maintaining LLM-based applications, and splits the work into two loops ([Microsoft Learn](https://learn.microsoft.com/en-us/ai/playbook/technology-guidance/generative-ai/mlops-in-openai/) , last updated April 15, 2025, so an older source):

| Stage | Loop | What you manage in practice |
|---|---|---|
| Data curation | Inner | The documents, examples and test cases the app depends on |
| Experimentation | Inner | Prompt versions, retrieval settings, model choice, sometimes fine-tuning |
| Evaluation | Inner | Metrics and test sets that compare each variant |
| Validate and deploy | Inner to outer | Release checks and A/B tests before a variant reaches all users |
| Inference | Outer | Latency, throughput and batch processing |
| Monitor | Outer | Resource use, alerts on anomalies or privacy breaches, review of responses |
| Feedback and data collection | Outer back to inner | User feedback that flows back into the test sets |

The inner loop is where a team builds and refines. The outer loop is where it runs the result for real users and learns from them.

## LLMOps vs MLOps: what changes?

Many explainers frame LLMOps as MLOps applied to bigger models, with training at the center; [Google Cloud's](https://cloud.google.com/discover/what-is-llmops) best practices, for example, open with training data quality, then training algorithms and hyperparameter tuning. For most product teams today the model is someone else's API, and that moves the work elsewhere.

|   | Classic MLOps | LLMOps for an app on a hosted model |
|---|---|---|
| What you train | Your own model, on your data | Usually nothing; you pick a model ID |
| What you version | Datasets, features, model weights | Prompts, model IDs, retrieval index, tool definitions, eval sets |
| Who changes the model | Your team, on your schedule | The provider, including retiring old versions |
| Typical failure | Data drift degrades accuracy | A prompt edit or model swap changes behavior on cases nobody retested |
| Main cost driver | Training and serving compute | Tokens per request, multiplied by traffic and agent steps |
| Rollback | Redeploy the previous model artifact | Point back to the previous prompt version and model ID |

## What is the release unit in LLMOps?

In classic software you ship a build. In an LLM app, behavior depends on several parts that can change independently:

1. **Model ID** , pinned to a dated snapshot rather than a moving alias where the provider offers one.
1. **Prompt version** , stored outside the code so it can change and roll back on its own.
1. **Retrieval index** and the documents in it.
1. **Tool definitions** the model can call.
1. **Eval suite version** that approved this combination.

Treat these five as one release. Registries help with the second item. In the [MLflow Prompt Registry](https://mlflow.org/docs/latest/genai/prompt-registry/) , prompt versions are immutable and commit-based, and aliases let you move production to a new version, run A/B tests and roll back without a code deploy.

## Why is model retirement an LLMOps problem?

Because a model you depend on can stop answering. Anthropic's [model deprecations page](https://platform.claude.com/docs/en/about-claude/model-deprecations) tracks each model as active, legacy, deprecated or retired. Once a model is retired, requests to it fail. For publicly released models, Anthropic promises at least 60 days' notice before retirement. A concrete case on that page: claude-sonnet-4-5-20250929 was deprecated on September 30, 2026, with retirement set for November 30, 2026 and claude-sonnet-5-5 as the recommended replacement. The page also notes that partner platforms such as Amazon Bedrock set their own retirement dates.

A migration is a release like any other: run your eval suite on the replacement, compare cost and latency, then switch. A small CI check can make the deadline visible before it becomes an outage:

from datetime import date # Model IDs pinned in our release manifest. PINNED = {"support-bot": "claude-sonnet-4-5-20250929", "summarizer": "claude-haiku-4-5-20251001"} # Retirement dates from the provider's deprecations page. RETIRES = {"claude-sonnet-4-5-20250929": date(2026, 11, 30)} def check(today, warn_days=60): failures = [] for app, model in PINNED.items(): retire = RETIRES.get(model) if retire and (retire - today).days <= warn_days: failures.append(f"{app}: {model} retires {retire}, run evals on a replacement") return failures for line in check(date(2026, 11, 19)): print(line)
Run with Python 3 on our machine on 2026-11-19, it prints support-bot: claude-sonnet-4-5-20250929 retires 2026-11-30, run evals on a replacement. In a real pipeline, fail the job when the list is not empty.

## What should an LLMOps setup have before production?

- **Evals on every change** to any of the five release parts, with a pass threshold. The [AI evals](https://sdclabs.com/glossary/ai-evals/) entry shows a runnable CI gate.
- **Traces of real runs** with tokens, cost and latency per request or agent run. See [agent observability](https://sdclabs.com/glossary/agent-observability/) .
- **A budget per request** and an alert when cost per task moves, because prompt and model changes shift it quietly.
- **A migration calendar** fed by provider deprecation notices.
- **A rollback path** that does not need a code deploy.

Many generative AI projects stall between pilot and production; our post on [why generative AI fails after the demo](https://sdclabs.com/blog/generative-ai-production/) looks at the evidence and at what the teams that ship do differently. Setting up this operating layer, from prompt versioning and tracing to cost tracking and release gates in CI, is part of our [generative AI development](https://sdclabs.com/services/generative-ai/) work. Founded in 2011 by the people behind Codeception and CodeceptJS, SDCLabs brings test discipline to it: automated tests verify every change, and engineers decide what ships.

## Sources

1. [LLMOps: Operational management of LLMs](https://learn.microsoft.com/en-us/ai/playbook/technology-guidance/generative-ai/mlops-in-openai/) , Microsoft Learn (AI playbook), 2025-04-15
1. [LLMOps: What it is and how it works](https://cloud.google.com/discover/what-is-llmops) , Google Cloud, 2026-11
1. [Prompt Registry](https://mlflow.org/docs/latest/genai/prompt-registry/) , MLflow, 2026-11
1. [Model deprecations](https://platform.claude.com/docs/en/about-claude/model-deprecations) , Anthropic (Claude Platform Docs), 2026-11

## Frequently asked questions

### What does LLMOps stand for?

Large language model operations: the processes and tools for building, deploying, monitoring and maintaining applications that use large language models.

### Is MLOps outdated now that LLMOps exists?

No. Teams that train or fine-tune their own models still need MLOps for data, training runs and model registries. LLMOps adds what apps on hosted models need on top: prompt versions, evals on every change, token costs and provider migrations.

### Do you need LLMOps if you only call a model API?

Yes, a lighter version of it. Even a single prompt behind an API needs a pinned model ID, an eval set to rerun when anything changes, cost tracking and a plan for the day the provider retires that model.

### Who owns LLMOps in a team?

Usually the platform or DevOps engineers who already own CI and monitoring, working with QA on the eval suites and with product owners on quality thresholds. The work fails when nobody owns model migrations.

Services

- [Generative AI](https://sdclabs.com/services/generative-ai/)
- [DevOps and TestOps](https://sdclabs.com/services/devops-and-testops/)

Read next

- [Generative AI fails after the demo, not before it](https://sdclabs.com/blog/generative-ai-production/)
- [The state of AI engineering in 2026](https://sdclabs.com/blog/state-of-ai-engineering-2026/)

Related terms

- [AI evals](https://sdclabs.com/glossary/ai-evals/)
- [Agent observability](https://sdclabs.com/glossary/agent-observability/)
- [AI guardrails](https://sdclabs.com/glossary/ai-guardrails/)

[Sergey Almyashev](https://sdclabs.com/author/almyashev/) Chief Operating Officer14+ years in QA, automated, exploratory and performance testing, and continuous integration and delivery within Scrum, helping teams in product and outsourcing companies organize teams, processes and tools.[LinkedIn](https://www.linkedin.com/in/sergeyalmyashev/)

## Building with AI agents? Talk to an engineer.

[Start a project](https://sdclabs.com/contact/) [Book a call with the CEO](https://calendly.com/sdclabs/60min)
