Your Agent Has Every Tool. It Still Picks the Wrong One.

The agent chooses the export tool. Unfortunately, there are two export tools: one previews a document and one starts a background job. The names are similar, the descriptions overlap, and the arguments it supplies belong to the other one.

A longer prompt can explain the distinction. That fix gets harder to maintain with each new tool. The design I prefer gives the agent a small discovery surface and retrieves the details when they matter.

This is an architecture note from platform work. The examples below are illustrative; they do not reproduce a private tool inventory or report a measured selection-accuracy improvement.

Search the catalog, then load the schema

Treat tool metadata as a searchable collection. The permanent interface needs to answer a few questions:

The search response should help the model choose a candidate without paying for all of its parameters yet. For example:

{
  "query": "preview a report without creating a file",
  "results": [
    {
      "tool": "reports.render_preview",
      "summary": "Read-only preview of a report section; creates no export job."
    },
    {
      "tool": "reports.export_async",
      "summary": "Create an export file in a background job; returns a job id."
    }
  ]
}

The model can now fetch the preview schema and call it. Adding another tool means adding a registry entry. It should not require rewriting every agent's prompt.

flowchart LR
    task["Task"] --> search["Search metadata"]
    search --> cards["Short candidate list"]
    cards --> schema["Describe one tool"]
    schema --> call["Validate and call"]
    search -->|"No suitable match"| absent["Report missing capability"]

This moves a problem; it does not eliminate it. If search omits the right tool, the model cannot choose it. A vague description becomes a retrieval bug. A small set of obvious, frequently used tools may deserve to stay directly available, especially when an extra search and describe would dominate a short task.

The pattern is also available in provider tooling: Anthropic documents deferred tool discovery and programmatic composition in its advanced tool-use design. The useful comparison is between those facilities and the custom behavior your system actually needs.

Keep intermediate data out of the conversation

Some tasks need a chain: fetch rows, sort them, render the top results. Passing the whole table through the model just so it can copy values into the next call is unnecessary work. A bounded composition facility can run that chain and return only the result the conversation needs.

# Illustrative composition; the execution service supplies the tools.
rows = tools.call("data.query", {"source": "example_sales"})
top = sorted(rows, key=lambda row: row["revenue"], reverse=True)[:10]
doc = tools.call("docs.render_table", {"rows": top})
result = {"document_id": doc["id"], "row_count": len(top)}

The intermediate rows stay inside the execution service. That can reduce returned payload size; actual token cost and end-to-end latency still need measurement.

An AST allowlist, restricted builtins, and a timeout are useful checks, but they do not by themselves establish a secure Python sandbox. Untrusted code needs an appropriate isolation boundary and resource limits. Every tool called from that code must still pass the same authorization and input validation as a direct call. Composition is a convenience, not a new permission level.

What happens after the model makes a mistake?

The server has to answer this even if retrieval is excellent.

A retry must refer to the same logical operation. Have the application assign an operation id and retain it across retries. A newly minted id on each attempt cannot deduplicate anything. Argument hashes alone are also insufficient: a user may legitimately request the same action twice. The durable operation record and the side effect need coordinated handling so a crash between them cannot quietly repeat the action.

A long job needs a life outside the chat connection. Return a job id, persist state transitions, and reconcile unfinished jobs after a restart. Reconnecting to the conversation should recover the job's status rather than start another copy.

Confirmation must be checked by the application. A destructive action can return a preview before execution. Approval should bind the human's decision to the exact action and arguments, with expiry and single-use enforcement. Returning a token to the model is not, by itself, evidence that a person approved anything.

An error should say whether retrying helps. Give callers stable error codes, retryability, and enough context to recover. Keep the retry budget in the runtime; a helpful error message is guidance, not enforcement.

The comparison I still want

A smaller prompt is easy to demonstrate. Better tool selection is a separate claim. A useful test would run the same tasks against a flat catalog, a discovery surface, and a hybrid with common tools pinned. It would record:

• whether retrieval surfaced the required tool;
• whether the selected tool and arguments were correct;
• total task completion, including recovery from failed calls;
• latency and token use across the entire task.

Until that comparison exists, I would describe the gain as bounded disclosure and a more maintainable catalog—not a proven accuracy win. The local MCP project has a narrower measured example: compact reads against real research artifacts, with explicit limits on what those measurements establish.