Connect your agent
Give AgentLasso an HTTPS endpoint that runs your agent on one message, and you can grade a test case against your agent's live answer from the rubric editor.
Usually there's nothing to build: pick how AgentLasso should talk to your agent.
OpenAI-compatible chat
AgentLasso sends one user message, plus model if you set one:
It reads the reply from choices[0].message.content and tool calls from choices[0].message.tool_calls (function.name, function.arguments).
If your agent runs its tools internally and returns only the final answer, the tool calls aren't in the reply, so tool checks can't see them. Reply checks (contains, forbidden, AI judge) still work. To grade tool use too, return the tool calls in the reply, or use Custom JSON pointing at where your API lists them.
Custom JSON
- Request body: JSON with
"{{input}}"where the message goes, for example{ "query": "{{input}}", "stream": false }. The message is escaped for you;{{test_case_id}}works too. Credentials go in the auth header, not the body. - Reply text is at: a dot path into the reply, such as
answer,data.replyorchoices.0.message.content. - Tool calls at (optional): the path to the list of tool calls, with each one's name field and arguments field (default
nameandarguments).
Test connection shows the raw reply under Raw reply, so you can find the paths. A wrong path says what's in the reply instead.
AgentLasso format
AgentLasso sends:
Your endpoint replies 200 with:
output: the agent's final reply (a string, ornull).tool_calls: the tools it called, in order.argumentsmay be an object or a JSON string (common with OpenAI-style agents).- At least one of
outputortool_callsis required.
These are test calls, not real customers. Have your endpoint run tools that book, pay, cancel or send email in a dry-run mode for them, while still listing the calls in tool_calls.
Set it up
In Settings → Your agent:
- How to talk to it: OpenAI-compatible chat, Custom JSON or AgentLasso format (above), with the fields it needs.
- Endpoint URL: the final HTTPS URL. Redirects aren't followed, so use the URL your agent actually answers on.
- Auth header (optional): a header name and value your endpoint checks, e.g.
Authorization/Bearer …. The value is stored server-side and never shown again. - Test connection sends a sample message (editable) using the form as filled in, before you save. You'll see the reply and tool calls, or a specific error:
- HTTP 401/403: check the auth header;
- HTTP 404/405: check the URL (it must accept
POST); - "isn't JSON" or a missing field, with an excerpt of what came back;
- a redirect (use the final URL), a timeout, or an unreachable host.
- Save.
Use it
- Run tests from the app: Run all tests, a suite, a capability's tests or one test. AgentLasso calls your agent on each and grades the answers; see Read test results.
- Rubric editor → Try it before saving → Run my agent: sends the test case's input to your agent and grades its answer with your draft checks. Edit checks and the same answer re-grades instantly; Run again asks your agent again.
Security
AgentLasso calls your endpoint from its servers, never from the browser:
- HTTPS only; private, local and cloud-metadata addresses are refused;
- redirects aren't followed;
- 30-second timeout and a 1 MB reply limit;
- at most 20 calls a minute per user;
- the auth header value is stored where only the server can read it.
Self-hosting: set AGENT_ENDPOINT_ALLOW_PRIVATE=true to allow http:// and private addresses (e.g. an agent on your own network). Leave it unset if untrusted users can sign up.
Next
Gate every release on your golden tests: Run golden tests in CI.