Tool-Use Patterns in LLM Agents

From free text to structured actions

An LLM only knows how to do one thing: predict the next token given everything before it. For an agent to actually search the web, query a database, or send an email, something outside the model has to turn its output into a real action. Tool use (function calling) is the interface that makes this reliable: instead of asking the model to describe an action in prose and hoping your code can parse it, you give the model a formal description of each available tool, and the model is trained to emit a structured, machine-parseable request when it wants to use one.

The model itself never executes anything. It outputs structured text that names a tool and supplies arguments; your application code is responsible for actually running that tool and feeding the result back in.

Anatomy of a tool schema

Every tool is described to the model with a name, a natural-language description of what it does and when to use it, and a schema for its parameters — almost always JSON Schema. The description matters more than developers often expect: the model decides whether and when to call a tool largely based on how well the tool's description matches the user's apparent intent, so vague or overlapping descriptions lead directly to wrong or missed tool calls.

How the model decides which tool to call

During training, models learn a pattern: given a conversation and a list of available tool schemas, predict whether a tool call is warranted, and if so, which tool and what arguments. At inference time this is just next-token prediction over a vocabulary that includes a structured tool-call format alongside normal text — the model is choosing, token by token, to emit either conversational text or a tool invocation.

This decision is sensitive to several things you control: how clearly each tool's description distinguishes it from the others, how many tools are offered at once (too many similar tools increase the chance of picking the wrong one), and how the system prompt frames when tool use is appropriate versus when the model should just answer directly from its own knowledge.

Free-text action parsing vs native tool calling

What makes tool use reliable in production

A few patterns consistently improve reliability. Keep tool descriptions distinct and specific — if two tools could plausibly apply to the same request, the model will sometimes pick the wrong one or alternate inconsistently between them. Validate every tool call's arguments against the schema before executing anything; models occasionally hallucinate a plausible-sounding parameter name or value that doesn't actually exist, and executing an unvalidated call against a real system is a direct path to bugs or worse. Return clear, structured error messages back to the model when a tool call fails or arguments are invalid, rather than a raw exception — the model can often self-correct and retry with fixed arguments if the error is legible.

It also helps to limit the number of tools exposed in any single turn to the ones genuinely relevant to the current task. Giving a model forty tools 'just in case' measurably increases the rate of wrong or unnecessary tool calls compared to giving it the five that are actually relevant.

What's next

Tool use lets an agent act, but most agents also need to find information they weren't given directly. Advanced RAG patterns go beyond simple vector lookup — combining keyword and semantic search, re-ranking, and query rewriting — to make that retrieval step actually reliable. That's the next lesson.