Read test results
Every run of your golden tests appears on the Test Runs page, graded check by check: runs you start in the app, and runs from your CI.

Run tests from the app
With your agent connected (Settings → Your agent, see Connect your agent), AgentLasso calls it on each test and grades the answers, with nothing to set up in CI:
- Run all tests on Test Runs or in Dataset Studio;
- Run now on a suite (Dataset Studio → By suite, or All tests);
- Run its tests on a capability in Dataset Studio;
- Run on one test (open it in the list).
A run goes through a few tests at a time, with progress at the top of every page and the results appearing on Test Runs as each test is graded. It keeps going while the app is open in any tab; close it and the run picks up where it left off next time you open it. When it finishes, a message gives the result.
Tests are graded exactly as in CI, including AI judge checks and evaluators. If the agent can't answer a test (an error, a timeout), that test is not graded with the reason, and the rest of the run continues. If the agent is disconnected during a run, the run stops and says so. Up to 500 tests per run; for more, run a suite or a capability at a time, or use CI.
Use CI to gate pull requests, or when AgentLasso can't reach your agent (for example, it runs on a private network): Run golden tests in CI.
The run list
Each run shows what it covered (a suite, capabilities, all tests or one test), its status, where it came from (From the app or CI), and Pass X% / need Y%: the pass rate against the run's pass threshold.
- Completed: the pass rate met the threshold.
- Failed: it didn't, or nothing in the run could be graded.
- N not graded (no checks): test cases without checks. They verify nothing, so they're left out of the pass rate.
Each test case row summarizes its checks by family (trajectory, content, format), then its result and score:
- a check mark: every check in that family passed;
- a red flag: a critical check failed;
- a warning mark: only minor checks failed;
- an empty circle: the test case has no checks in that family.
A test case's detail
Click a row to see exactly what happened in that run:
- Checks: each check with pass/fail, its family and severity, and the reason, e.g. "stripe.refund was never called. The agent called: orders.lookup."
- Tool calls in this run, with any expected tool that was never called highlighted.
- Agent output in this run.
- Rubric, not graded yet: free-text items that need converting into checks.
- The production trace this test case was created from, for comparison. That's the original behavior, not this run.
Pass threshold
A run passes when its pass rate is at least its pass threshold, which you set when creating the run (default 1.0: every graded test case must pass). A run where nothing could be graded never passes. See Run golden tests in CI and the API reference.