Connect your agent

Give AgentLasso an HTTPS endpoint that runs your agent on one message, and you can grade a test case against your agent's live answer from the rubric editor.

Usually there's nothing to build: pick how AgentLasso should talk to your agent.

FormatUse it when
OpenAI-compatible chatYour agent has a chat completions endpoint (…/v1/chat/completions), as many agent platforms and frameworks do.
Custom JSONAny other JSON API: you say what to send and where the answer is in the reply.
AgentLasso formatYou write a small endpoint for AgentLasso (below). It's the same as the callAgent step in the CI example, so it serves both.

OpenAI-compatible chat

AgentLasso sends one user message, plus model if you set one:

json
{  "model": "<optional>",  "messages": [{ "role": "user", "content": "I want my money back for order 44182" }]}

It reads the reply from choices[0].message.content and tool calls from choices[0].message.tool_calls (function.name, function.arguments).

If your agent runs its tools internally and returns only the final answer, the tool calls aren't in the reply, so tool checks can't see them. Reply checks (contains, forbidden, AI judge) still work. To grade tool use too, return the tool calls in the reply, or use Custom JSON pointing at where your API lists them.

Custom JSON

  • Request body: JSON with "{{input}}" where the message goes, for example { "query": "{{input}}", "stream": false }. The message is escaped for you; {{test_case_id}} works too. Credentials go in the auth header, not the body.
  • Reply text is at: a dot path into the reply, such as answer, data.reply or choices.0.message.content.
  • Tool calls at (optional): the path to the list of tool calls, with each one's name field and arguments field (default name and arguments).

Test connection shows the raw reply under Raw reply, so you can find the paths. A wrong path says what's in the reply instead.

AgentLasso format

AgentLasso sends:

http
POST <your endpoint>content-type: application/json<your auth header>
{  "input": "I want my money back for order 44182",  "messages": [{ "role": "user", "content": "I want my money back for order 44182" }],  "test_case_id": "…",  "source": "agentlasso-preview"}
FieldMeaning
inputThe message to answer: a test case's input.
test_case_idWhen running a test case.
sourceagentlasso-test (connection test), agentlasso-preview (rubric editor) or agentlasso-run (a test run started in the app). Use it to tag or filter these requests, and to keep them out of your production traces.

Your endpoint replies 200 with:

json
{  "output": "Your refund of $89 is on its way.",  "tool_calls": [    { "name": "orders.lookup", "arguments": { "order_id": "44182" } },    { "name": "stripe.refund", "arguments": { "amount_cents": 8900 } }  ]}
  • output: the agent's final reply (a string, or null).
  • tool_calls: the tools it called, in order. arguments may be an object or a JSON string (common with OpenAI-style agents).
  • At least one of output or tool_calls is required.

These are test calls, not real customers. Have your endpoint run tools that book, pay, cancel or send email in a dry-run mode for them, while still listing the calls in tool_calls.

Set it up

In Settings → Your agent:

  1. How to talk to it: OpenAI-compatible chat, Custom JSON or AgentLasso format (above), with the fields it needs.
  2. Endpoint URL: the final HTTPS URL. Redirects aren't followed, so use the URL your agent actually answers on.
  3. Auth header (optional): a header name and value your endpoint checks, e.g. Authorization / Bearer …. The value is stored server-side and never shown again.
  4. Test connection sends a sample message (editable) using the form as filled in, before you save. You'll see the reply and tool calls, or a specific error:
    • HTTP 401/403: check the auth header;
    • HTTP 404/405: check the URL (it must accept POST);
    • "isn't JSON" or a missing field, with an excerpt of what came back;
    • a redirect (use the final URL), a timeout, or an unreachable host.
  5. Save.

Use it

  • Run tests from the app: Run all tests, a suite, a capability's tests or one test. AgentLasso calls your agent on each and grades the answers; see Read test results.
  • Rubric editor → Try it before saving → Run my agent: sends the test case's input to your agent and grades its answer with your draft checks. Edit checks and the same answer re-grades instantly; Run again asks your agent again.

Security

AgentLasso calls your endpoint from its servers, never from the browser:

  • HTTPS only; private, local and cloud-metadata addresses are refused;
  • redirects aren't followed;
  • 30-second timeout and a 1 MB reply limit;
  • at most 20 calls a minute per user;
  • the auth header value is stored where only the server can read it.

Self-hosting: set AGENT_ENDPOINT_ALLOW_PRIVATE=true to allow http:// and private addresses (e.g. an agent on your own network). Leave it unset if untrusted users can sign up.

Next

Gate every release on your golden tests: Run golden tests in CI.