AgentLasso.dev

Capabilities and classification

A capability is a job your agent performs: process_refund, track_shipment, cancel_subscription. Every trace is classified into one, and everything else in AgentLasso (the scorecard, risk, coverage, the priorities) is measured per capability.

Each capability has:

  • a name and a plain-English description,
  • its target tools: the tools that identify it (e.g. stripe.refund, orders.lookup),
  • a business goal and a business value model, which you set in its deep dive on the Semantic Agent Map.

How traces are classified

1. Tool matching (free, instant). If a trace's tool calls overlap a capability's target tools, the capability with the most overlapping tools wins. The rules step aside when they can't decide: no tools called, no capability overlaps, or two capabilities tie. No model is involved, and it costs nothing.

2. Claude (for the rest). Traces the rules can't place are classified by Claude, which also names new capabilities and flags edge cases worth a human's attention (prompt-injection attempts, ambiguous multi-intent requests, replies that look wrong). This needs an Anthropic API key on the server (ANTHROPIC_API_KEY). Without one, those traces wait, marked as waiting for AI classification, and the Semantic Agent Map shows how many.

Two more details:

  • Free edge-case signals. Traces the rules classify are also checked for concrete signals, such as an escalation to a human (a tool named like escalate, handoff or transfer_to_human).
  • Audit sample. About 10% of rule-classified traces are still sent to Claude, chosen deterministically by trace ID. That's how routing accuracy (how often Claude's classification agrees with the capability the trace's tools point to) is measured, and how edge cases in rule-classified traffic still get noticed.

When classification runs

  • Your first traces are classified as they arrive. While a project has fewer than 10 classified traces, the telemetry endpoint classifies new traces before responding, so you see results right away.
  • After that, in the background. A scheduled job (POST /api/v1/sync-cluster) classifies new traces in batches using Anthropic's Batch API, which costs about half as much. Self-hosting: see Self-hosting for how to schedule it.

Give classification context

Write a sentence or two in Settings → Agent description about what your agent does. Claude uses it to name capabilities and judge edge cases in your agent's real domain, instead of guessing from a generic default. For example, a travel assistant's "cancel" means a booking, not a subscription.

New behavior: unmapped traffic

Traffic that doesn't belong to any registered capability shows up under Unmapped log volume on the Semantic Agent Map, grouped by what Claude named it, with example requests. Register a group to turn it into a new capability. That's how your capability map grows with what users actually ask for.

Next

See how each capability is scored: The capability scorecard.