Agent observability and evaluation: traces, metrics and tests

This guide is for developers who already have a working agent and are preparing to put it in front of real users. By the end you will understand the difference between observability and evaluation, know which metrics to watch, and know how to combine offline and online evaluation.

What observability is

Observability means understanding what is happening inside an AI agent by looking at external signals such as logs, metrics and traces. For an agent, that means tracking its actions, tool usage, model calls and responses, so that you can debug it and improve its performance^[1].

Without observability, an agent is a black box. Observability tools make it transparent. They help you understand cost and accuracy trade-offs, measure latency, detect harmful language and prompt injection, and monitor user feedback. In other words, they are what turns a demo agent into one that is ready for production^[1].

Tools, traces and spans

Langfuse and Arize are common observability tools. They collect detailed traces and offer dashboards that monitor metrics in real time^[1]. The tools differ widely. Some are open source and have large communities shaping their roadmaps. Some specialise in observability, evaluation or prompt management, and others cover the whole LLMOps workflow. Read the documentation of several options before you choose one^[1].

Many agent frameworks, such as smolagents, use the OpenTelemetry standard to expose metadata to observability tools. The tools also build their own instrumentation to keep up with the fast-moving LLM field. Check the documentation of the tool you use to see what it supports^[1].

Observability tools usually represent an agent run as traces and spans. A trace is a complete agent task from start to finish, such as handling one user query. A span is an individual step inside the trace, such as a call to a language model or a data retrieval^[1].

Metrics to watch

Latency measures how quickly the agent responds. Long waits hurt the user experience. Measure latency for whole tasks and for each step by tracing runs. For example, if an agent spends 20 seconds on all of its model calls, you could switch to a faster model or run the model calls in parallel^[1].

Costs measure what each agent run spends. LLM calls are billed per token, and external APIs charge too. Frequent tool use or several prompts can raise costs quickly. Real-time monitoring also helps catch sudden spikes, such as a bug that causes excessive API loops^[1].

Request errors count the requests that fail, including API errors and failed tool calls. To make the agent more robust in production, set up fallbacks or retries. If provider A is down, for instance, the agent can switch to provider B as a backup^[1].

User feedback includes explicit signals such as thumbs up or thumbs down, one to five stars, and written comments. Consistent negative feedback means the agent is not working as expected^[1]. Implicit feedback comes from behaviour, such as rephrasing a question, repeating a query or clicking a retry button. If users keep asking the same question, that is also a sign of trouble^[1].

Accuracy measures how often the agent produces correct or desirable output. Start by defining what success looks like. Then track accuracy with automated checks, evaluation scores or task completion labels, such as marking each trace as succeeded or failed^[1].

Automated evaluation metrics use an LLM to score the agent's output, for example for helpfulness or accuracy^[1]. Open-source libraries cover other aspects. RAGAS is named for RAG agents, and LLM Guard is named for detecting harmful language and prompt injection^[1]. In practice, a combination of these metrics gives the best picture of an agent's health^[1].

Why evaluation is needed

Observability gives you metrics. Evaluation analyses that data and runs tests to judge how well the agent performs and how it can improve. Agents are often non-deterministic, and they change through updates or drifting model behaviour. Without regular evaluation, you cannot tell whether an agent has become worse^[1].

There are two kinds of evaluation, offline and online. They complement each other. Offline evaluation is usually the first step, and it is the minimum you should do before deploying any agent^[1].

Offline evaluation

Offline evaluation tests the agent in a controlled setting. It uses curated test datasets whose expected outputs are known, rather than live user queries. It is repeatable, and because there is ground truth, it gives clear accuracy metrics. Teams often run it during development, and it can be part of CI/CD pipelines to confirm that a change helps or to guard against regressions^[1].

For example, a math word-problem agent might have a test dataset of 100 problems with known answers^[1].

The main challenge is keeping the test dataset comprehensive and relevant. An agent can do well on a fixed set and still meet very different queries in production, so keep adding new edge cases. A mix works well: small smoke-test cases for quick checks, and larger evaluation sets for broader performance metrics^[1].

Online evaluation

Online evaluation runs in a live environment, during real use in production. It monitors how the agent performs on real user interactions. It can show model drift and unexpected queries that were not in the test data^[1].

Online evaluation can combine implicit and explicit user feedback. It can also use shadow tests or A/B tests, in which a new version of the agent runs alongside the old one so the two can be compared^[1].

Combining the two

Good evaluation blends offline and online methods. Run offline benchmarks regularly to score the agent on defined tasks, and monitor live traffic to catch what the benchmarks miss. Offline tests can show whether the success rate of a code-generation agent on a known set of problems is rising, while online monitoring may reveal that users have started asking a kind of question the agent handles poorly^[1].

Many teams follow a loop: offline evaluation, deploy a new version, monitor online metrics and collect new failure examples, add those examples to the offline test set, and iterate. Evaluation then becomes a continuous process that keeps improving the agent^[1].

Next steps

The next section shows, with examples, how observability tools can be used to monitor and evaluate an agent^[1].

Adapted from the bonus unit 2 of Hugging Face's open-source course *Agents Course*, which covers agent observability and evaluation^[1], released under the Apache-2.0 License.