Capabilities and classification

A capability is a job your agent performs: process_refund, track_shipment, cancel_subscription. Every trace is classified into one, and everything else in AgentLasso (the scorecard, risk, coverage, the priorities) is measured per capability.

Each capability has:

  • a name and a plain-English description,
  • its target tools: the tools that identify it (e.g. stripe.refund, orders.lookup),
  • a business goal and a business value model, which you set in its side panel under Risk, business value and goal.

How traces are classified

A capability is what the user needed. The tools a trace called say what the agent did, which may be the wrong thing, so with semantic grouping on (Settings → Semantic grouping; on by default for new projects, and it needs an embeddings provider on the server), traces are placed by the meaning of their request first:

1. By meaning (free, once embedded). Each capability has a prototype: the average meaning of the requests a person or the AI placed there (at least 3; traces the tool rules placed don't count, so a mistake in the tools can't shape it). A new trace goes to the closest prototype only when the match is clear: similar enough (the cutoff Discovery uses, calibrated on your Validation answers) and clearly closer than the next capability. About 10% of these are still sent to Claude, and the Assignment page shows how often the two agree.

2. Claude, when the meaning isn't clear. Claude picks one of your capabilities, or none, now seeing each capability's goal and two requests a person placed there. The tools don't decide for an embedded trace, unless there's no Anthropic API key (then they're the last resort).

If a trace placed this way called another capability's tool, it's flagged as an unexpected path (see Work with your capability map).

Without semantic grouping, or for traces not embedded yet, classification works as follows:

1. Tool matching (free, instant). If a trace's tool calls overlap a capability's target tools, the capability with the most overlapping tools wins. The rules step aside when they can't decide: no tools called, no capability overlaps, or two capabilities tie. No model is involved, and it costs nothing.

2. Claude (for the rest). Traces the rules can't place are classified by Claude, which assigns each one to a registered capability or to none, and flags edge cases worth a human's attention (prompt-injection attempts, ambiguous multi-intent requests, replies that look wrong). This needs an Anthropic API key on the server (ANTHROPIC_API_KEY). Without one, those traces wait, marked as waiting for AI classification, and the Semantic Agent Map shows how many.

Two more details:

  • Free edge-case signals. Traces the rules classify are also checked for concrete signals, such as an escalation to a human (a tool named like escalate, handoff or transfer_to_human).
  • Audit sample. About 10% of rule-classified traces are still sent to Claude, chosen deterministically by trace ID. That's how routing accuracy (how often Claude's classification agrees with the capability the trace's tools point to) is measured, and how edge cases in rule-classified traffic still get noticed.

When classification runs

  • Your first traces are classified as they arrive. While a project has fewer than 10 classified traces, the telemetry endpoint classifies new traces before responding, so you see results right away.
  • After that, in the background, as traffic arrives. After responding, the telemetry endpoint classifies the project's waiting traces. A small backlog is classified right away. A large one (50 or more, such as an import) goes to Anthropic's Batch API, all of it at once, which costs about half as much and usually finishes within minutes, at most a few hours.
  • Results are collected by the next processing round: new traffic, the app itself while traces are waiting (the progress banner on Production Sync, Discovery and the Dashboard checks every 20 seconds), or the optional scheduled job (see Self-hosting). No scheduler is needed.

Give classification context

Write a sentence or two in Settings → Agent description about what your agent does. Claude uses it to assign traces, propose names for new behavior, and judge edge cases in your agent's real domain, instead of guessing from a generic default. For example, a travel assistant's "cancel" means a booking, not a subscription.

New behavior, merges and checking the map

Claude never creates a capability on its own. A trace that fits none of your registered capabilities stays unassigned, with a name Claude proposes for it. That keeps your capabilities stable: Claude can't call the same behavior "refund_request" one day and "process_refund" the next, splitting its history in two.

Three pages under Semantic Agent Map keep the map complete, stable and correct:

  • Discovery: traffic that fits none of your capabilities, grouped by what the user asked for (with semantic grouping on) or by the tools it called. Accept a group as a new capability, add it to an existing one, or ignore it. Accepting or adding moves all its past traces and fills in its scorecard history.
  • Assignment: how traces get their capability (rules, AI, a person), how often the rules and AI agree, and merging two capabilities that are really one behavior.
  • Validation: people check pairs of traces ("same capability?"), which measures how well assignment works and suggests merges or splits.

How to use them together, step by step: Work with your capability map.

Next

See how each capability is scored: The capability scorecard.