How Do You Give an AI Agent 75 Tools Without Wrecking Its Judgment?

There's a moment every agent platform hits. The first five tools work great. The next ten still work. Somewhere past a few dozen, the agent starts doing something maddening: it picks a plausible tool instead of the right one, invents a parameter that belongs to a sibling tool, or gives up and asks the user a question it has the tool to answer.

Nothing broke. No exception fired. The catalog just got big enough that tool selection quietly turned into a needle-in-a-haystack retrieval problem — and we were solving it by dumping the entire haystack into the prompt.

This post is about the pattern that fixed it for us: putting 75+ tools behind a small tool-RAG discovery surface, plus the reliability engineering that had to come with it once agents could reach everything.

Why dumping N tools into context degrades selection

The naive MCP setup is beautifully simple: register every tool on the connection, let the model see every schema, let it choose. It fails for two compounding reasons.

The token cost is structural. Every tool definition — name, description, JSON Schema for parameters — rides along in every prompt, whether the task needs it or not. Back-of-envelope (this is arithmetic, not a measurement): if a well-documented tool schema runs a few hundred tokens, seventy-five of them is tens of thousands of tokens of fixed overhead before the agent has read a single word of the actual task. That's context the model can't spend on the user's problem, retrieved documents, or its own working notes. The fixed cost grows as $O(n)$ in catalog size, while the number of tools any single task actually needs stays roughly $O(1)$.

The distractor cost is worse. Tool selection is a classification problem the model solves implicitly on every turn, and every irrelevant schema is a distractor class. Real catalogs make this brutal, because real catalogs are full of near-duplicates: the search tool and the lookup tool, the sync exporter and the async exporter, three tools that all mention "report" in their description. With five tools the distinctions are obvious. With seventy-five, the model is pattern-matching on surface similarity in a sea of lookalikes, and the failure modes are exactly what you'd predict from retrieval theory: plausible-but-wrong selection, parameter bleed between sibling schemas, and — the sneakiest one — a growing reluctance to call tools at all, because nothing in context stands out as clearly correct.

The instinct to fight this with prompt engineering ("IMPORTANT: use tool X for Y!") doesn't scale. You're hand-tuning a ranking function in prose, and every new tool invalidates the tuning.

The reframe that actually worked: stop treating the catalog as prompt content and start treating it as a corpus. We already know how to let an LLM work against a corpus too large for context — retrieval. Tools are no different.

The discovery surface: five verbs instead of seventy-five schemas

What lives in the agent's context permanently is not the catalog. It's a tiny, stable meta-API — the same shape as a RAG pipeline, pointed at tool metadata instead of documents:

1. list — the coarse map. Returns the catalog's namespaces and categories with one-line summaries, so the agent can orient ("there are data tools, document tools, workflow tools…") without seeing a single schema.
2. search — the retrieval step. Free-text query over tool names, descriptions, and tags (semantic and keyword). Returns short cards — name, one-sentence purpose, category — never full schemas. Cheap enough to call speculatively.
3. describe — the disclosure step. Full parameter schema, constraints, examples, and warnings for one named tool, fetched only when the agent has a candidate.
4. call — the invocation step. Validated execution of one tool with arguments, returning a structured result envelope.
5. run-code — the escape hatch, covered below: a sandboxed script that composes several tools in one step.

A search response is deliberately schema-free — just enough to rank, nothing more:

{
  "query": "export quarterly report",
  "results": [
    { "tool": "reports.export_async", "summary": "Long-running export of a report to a file; returns a job id.", "category": "reports" },
    { "tool": "reports.render_preview", "summary": "Fast, read-only preview of a report section.", "category": "reports" },
    { "tool": "files.share_link", "summary": "Create a shareable link for an existing file.", "category": "files" }
  ]
}

The progressive-disclosure property is the whole trick. The model's implicit classification problem shrinks from "which of 75 schemas?" to "which of these three cards looks right?" — and it only pays full schema tokens for the finalist. Selection quality stops degrading as the catalog grows, because the catalog's size is no longer the model's problem; it's the retrieval index's problem. That's a problem we have fifty years of tooling for.

Two second-order wins mattered as much as the selection fix. Adding a tool became a registry write — no prompt edits, no redeploys of agent configs, no re-tuning of the "which tool when" prose. And the surface is honest about ignorance: when search returns nothing convincing, the agent can say so, instead of hallucinating a call against the least-wrong schema in context.

The failure mode this creates — and I'll be honest about it below — is that your tool descriptions are now a retrieval corpus, with all the curation burden that implies.

The escape hatch: composing tools in code

Discovery fixes selection. It does nothing for the chattiness problem: real tasks are chains. Fetch, filter, transform, submit — four tool calls, four round trips, and every intermediate result serialized into the model's context just so the model can copy it into the next call's arguments. That's slow, expensive, and a fresh opportunity to mangle data at every hop.

So the fifth verb accepts a short script instead of a single call. The server executes it in a sandbox with the tool catalog bound as ordinary functions, and only the final result returns to the model:

# agent-authored, server-executed
rows = tools.call("data.query", {"source": "sales", "period": "last_quarter"})
top = sorted(rows, key=lambda r: r["revenue"], reverse=True)[:10]
doc = tools.call("docs.render_table", {"rows": top, "title": "Top accounts"})
result = {"document_id": doc["id"], "row_count": len(top)}

Three tool invocations, one agent turn, zero intermediate payloads flowing through the context window. The intermediate rows never cost a token.

Letting an LLM submit code to your server is, obviously, the kind of sentence that should make you sit up straight. The sandbox is layered and boring on purpose:

• AST allowlist. The script is parsed before execution and every node is checked against an allowlist. No import, no attribute access into dunder internals, no exec/eval, no file or network primitives. If the walker sees a node type it doesn't recognize, the script is rejected with a structured error the model can read and fix — rejection is a teaching signal, not just a wall.
• Restricted builtins. The execution namespace contains the bound tool functions and a hand-picked set of pure builtins (len, sorted, sum, comprehension machinery). Nothing that touches the host.
• Wall-clock timeout. A hard deadline on the whole script, because an allowlisted loop is still a loop, and "the agent wrote an accidental O(n²) over a big result" is not a hypothetical.

Crucially, the sandbox doesn't grant new authority — every tools.call inside the script goes through exactly the same validation, permission checks, and guards as a direct call. Code mode changes the composition cost, not the security boundary.

Reliability is most of the work

Discovery and composition are the fun half. The unglamorous half is what happens when an agent can now reach tools that are slow, dangerous, or flaky — because the LLM is the least reliable component in the system, and it's the one holding the trigger.

Durable async jobs

Some tools outlive an agent turn. Holding the MCP request open is a losing game — the connection drops, the agent times out, the work is orphaned. So long-running tools return immediately with a job id, and the job's lifecycle is write-through persisted: every state transition hits the durable store before the worker proceeds, never after-the-fact. On server restart, a reconciliation pass walks every job marked running and re-derives the truth — re-attach, restart, or mark failed — so a deploy can't strand work in a phantom "running" state forever.

Runtime-derived idempotency keys

Agents retry. Networks retry. Models occasionally emit the same call twice in one turn with a straight face. For any tool with side effects, the execution layer derives an idempotency key at runtime — from the tool name and a canonicalized hash of the arguments plus the invocation context — rather than trusting the model to supply one. Ask an LLM to generate its own idempotency key and it will cheerfully mint a fresh one per retry, which defeats the entire point.

def idempotency_key(tool, args, ctx):
    canonical = json.dumps(args, sort_keys=True, separators=(",", ":"))
    return sha256(f"{tool}|{canonical}|{ctx.session_id}|{ctx.turn_id}")
 
# duplicate key within the dedupe window -> return the original
# result envelope, execute nothing

Same request twice → same effect once, same response twice. It's the difference between "the agent retried" and "the agent submitted the request twice."

Confirm-guards on destructive actions

Destructive tools are two-phase. The first call executes nothing — it returns a human-readable preview of what would happen plus a short-lived confirmation token. Only a second call presenting that token fires the action, and the confirmation routes through the human, not just the model:

{
  "status": "confirmation_required",
  "preview": "This will permanently delete 3 records. This cannot be undone.",
  "confirm_token": "cfg_7f3a…",
  "expires_in_seconds": 120
}

The token being short-lived and single-use matters: a stale confirmation from five turns ago — when the world may have changed — can't authorize anything now.

Structured error envelopes

Here's a reframe that changed how we write error handling: in an agent system, the error message is a prompt. The model reads it and decides what to do next. A bare stack trace teaches it nothing; worse, it invites creative misinterpretation. Every failure comes back in the same envelope, with a machine-readable retryability flag and a suggested next action:

{
  "ok": false,
  "error": {
    "code": "UPSTREAM_TIMEOUT",
    "message": "The data source did not respond within the deadline.",
    "retryable": true,
    "suggested_action": "Retry once with a narrower date range; if it fails again, report the outage to the user instead of retrying further."
  }
}

The retryable flag alone eliminated a whole class of pathology — agents hammering permanently-failed calls, and agents abandoning transiently-failed ones.

Circuit breaker on the LLM gateway

Several tools are themselves LLM-backed, which creates a fun recursion: an agent, driven by a model, calling tools that call models. When the gateway degrades, naive retries turn a brownout into a pile-on. A standard circuit breaker sits in front — trip on an error-rate threshold, fail fast while open with a retryable-later envelope, probe with a half-open trickle before closing. The agent experiences a clean, structured "this capability is temporarily down" instead of a minute of hung calls, and the gateway gets room to recover.

None of these four patterns is novel — they're distributed-systems staples. What's new is the client: a probabilistic planner that retries enthusiastically, misreads unstructured errors, and can't be trusted to generate its own idempotency keys. The patterns aren't optional hardening here. They're the interface contract.

What's hard, and what's next

Honest ledger, because this is far from finished.

Tool descriptions are the new prompt engineering. The discovery surface moved the quality burden from "tune one giant prompt" to "curate 75+ descriptions as a retrieval corpus." A tool with a vague summary is effectively invisible — search never surfaces it, so the agent behaves as if it doesn't exist, and nothing errors. That's a silent failure mode, and we don't yet have automated regression checks that catch "this tool stopped being findable for the queries that should find it." Retrieval-eval for tool cards is squarely on the list.

The latency tax is real. A task that needs one obvious tool now costs a search and a describe before the call. For a handful of high-traffic tools, always-in-context registration is probably the right hybrid; picking that pinned set principledly — rather than by anecdote — is open work.

Models fight the surface. Stronger models memorize tool names across a session and try to call directly, skipping describe — fine until a schema changes under them. Weaker models sometimes loop in search without committing. The surface needs to be forgiving in both directions: cheap schema re-fetches, and search results that gently push toward commitment. Neither behavior is fully tamed.

The sandbox walks a tightrope. Every builtin I allow makes the composer more useful; every one is surface area to reason about. The AST allowlist is conservative today, and agents occasionally write reasonable scripts it rejects. Expanding it demands adversarial review each time, and "the agent needed getattr" is exactly the request that should trigger suspicion, not sympathy.

I don't have the ablation I want. The qualitative before/after on tool selection is stark, but I haven't run the rigorous head-to-head — same tasks, flat 75-tool registration versus the discovery surface, selection accuracy measured properly. Until that harness exists, "this fixed tool-selection collapse" is an engineering judgment, not a measurement, and this blog has a rule about claims without evidence. Building that harness is the next real deliverable.

The one-sentence version, if you're staring down your own growing catalog: the moment your tool list stops fitting comfortably in the model's judgment — well before it stops fitting in the context window — stop shipping the catalog and start shipping a library card.