You have LLM tracing set up. Every model call is logged with its prompt, completion, token counts, latency, and cost. Dashboards are green. Then a user reports that the agent gave them a confidently wrong answer about their own account data, and you spend three hours reading traces and still can’t explain it.
This is not a tooling failure. It’s a granularity mismatch. Tracing built for single-call LLM applications records the right things at the wrong level for agents.
What changes when a prompt becomes an agent
A single-call app has one interesting event: a request goes to a model, a response comes back. Trace that and you know essentially everything.
An agent run is a tree. One user request becomes turns; each turn becomes model calls and tool calls; tool calls hit MCP servers; some spawn sub-agents that repeat the whole structure with their own context. One question — “prepare a refund summary for ticket #4521” — routinely fans out into a dozen machine-to-machine calls, most of them decided on by software rather than a human.
Per-call tracing on that tree gives you a flat list of leaves. Every leaf may look individually fine while the run as a whole went badly wrong.
Three questions tracing can’t answer
“How much did that conversation cost?”
Not that model call — the whole conversation, including every sub-agent, every retry, and every tool invocation that happened to be billable. Cost only becomes actionable when it’s aggregated to the session, because the session is the unit a user or a business process maps to.
The practical consequence of not having this: you can’t tell an expensive agent from a cheap one, so you can’t prioritize optimization, so you optimize on vibes. Teams routinely discover that 5% of sessions account for half the spend, and that those sessions share one structural cause — a sub-agent that loops, a tool returning enormous payloads that get re-sent every turn.
“Where did the time go?”
A run took 90 seconds. Was that model latency, tool latency, or the harness waiting on an approval? Per-call traces contain the answer but not the shape, and the shape is what tells you what to fix.
The view that works is a per-turn breakdown by event type — system, user, model, tool. A turn that’s mostly tool time is tool-bound: your MCP server is slow, or the agent is making ten calls where two would do. A turn that’s mostly model time is model-bound: your context is bloated or you picked an oversized model. Those need opposite fixes, and a flat trace list lets you confuse them.
“Which agent did this, acting for whom?”
This is the one that matters for incidents and audits, and it’s the one flat tracing handles worst.
If your agents forward the end user’s token downstream, every log at the destination says “Jane did this.” If they use a shared service account, every log says “the service did this.” Either way, “the retrieval agent leaked that data” is unprovable after the fact — and the request that reached Salesforce looks identical whether it came from your sanctioned support copilot or from an agent someone spun up on a laptop.
What you need on every hop is two identities, not one: which agent made the call, and which user it was acting for. That’s per-hop attribution, and it has to be recorded at the enforcement point rather than reconstructed from application logs afterward.
The metrics that actually matter
A session-level view should carry at minimum:
| Metric | Why you need it |
|---|---|
| Turns | Run length; catches loops |
| Duration | Wall clock across all turns |
| Cost | Aggregated across every model and tool call |
| Tokens | Total consumed |
| Active context size | The most under-instrumented metric in agent systems |
| Tool calls | How many MCP or sandbox tools were invoked |
| Sub-agents | How many the run spawned |
| Errors | Count of failed steps |
Active context size deserves the emphasis. It’s the metric that explains the failure mode nobody can reproduce: an agent that’s sharp for twenty turns and then subtly degrades. What happened is that context hit the limit and compaction fired, summarizing away a detail the agent needed. If you’re not graphing context size per turn, that looks like randomness. If you are, it’s obvious.
Sub-agent count is the second sleeper. A run that spawned 40 sub-agents is either doing something genuinely parallel or stuck in a delegation loop, and the count tells you which before the bill does.
Instrument at the gateway, not in the agent
The usual approach is an SDK in the agent: wrap the model client, wrap tool calls, emit spans. It works, and it has three structural problems.
Coverage depends on discipline. Every new agent needs instrumentation added. The ones that skip it are invisible, and they’re disproportionately likely to be the ad-hoc ones you most want to see.
Heterogeneity breaks it. Your agents are on LangGraph, Bedrock AgentCore, a homegrown HTTP service, and a copilot embedded in a SaaS product. There is no single SDK you can put in all of them.
Denied requests vanish. If instrumentation lives in the agent, a call that got blocked before it ran often produces no trace at all. But a blocked hop is audit evidence — arguably the most important kind — and it should be logged as loudly as a successful one.
Instrumenting at the proxy layer inverts all three. Every governed hop passes through it, so observability is a side effect of enforcement rather than something each team opts into. Coverage is automatic, runtime-agnostic, and includes what was refused.
This is the design behind TrueFoundry’s agent observability model: because every hop goes through a gateway, per-hop attribution, guardrail results, and unified cross-agent metrics get recorded without anything in the agent. Guardrail execution shows up as its own span — what it checked, what it found, what it redacted, how long it took — which turns “did our PII filter actually run on that request” from an argument into a lookup. Their metrics dashboard puts model, MCP, and agent traffic in the same view, filterable by user, team, and agent, which is the part that’s painful to assemble yourself across three separate observability stacks.
The gap that’s still open industry-wide
Two capabilities remain hard everywhere, and it’s worth naming them.
The full actor chain as one trace. Per-hop attribution tells you each call’s caller. Rendering the complete user → agent → sub-agent → tool delegation path as a single connected trace across process and vendor boundaries is a distributed-tracing problem that the agent ecosystem hasn’t solved cleanly yet. Most platforms, honest ones included, list this as in progress.
Authority versus usage. Your agent was granted twelve permissions. It exercises three. The other nine are standing risk with no owner reviewing them. Comparing declared authority against observed behavior — drift detection — is the thing that would make agent access reviews tractable, and almost nobody has it in production.
If you’re building your own observability layer, don’t start with either. Start with session aggregation and per-hop attribution. They’re achievable, and between them they cover most of what you’ll actually be asked during an incident.
Where to begin
Pick your most-used agent. Answer three questions about its last hundred runs: what did each session cost, where did time go per turn, and how large was the context at the end. If any of the three requires a data pull rather than a dashboard, that’s your first gap — and it’s almost always the third one.