Function Calling in LLMs

9 min read

Learn

Quick Reading

Estimated 9 mins

Prereq

Intermediate

Basic ML concepts helpful

Interactive

Static Playbook

Static guide & reference tables

Giving a model a typed interface to the outside world

Function calling (also called tool use) lets an LLM request the execution of external code — a weather API, a database query, a calculator — by emitting a structured output (typically JSON) that names a function and supplies arguments, rather than only emitting free-form text. The model itself never executes anything; it decides when a tool call is appropriate and what arguments to pass, and the surrounding application is responsible for actually invoking the function and feeding the result back into the model's context.

The mechanics of a tool call round trip

A typical round trip has four steps. First, the developer provides the model with a schema of available tools — names, descriptions, and typed parameters (usually JSON Schema). Second, given a user request, the model either answers directly or emits a tool call matching one of the schemas. Third, the application layer parses that call, executes the actual function, and captures its return value. Fourth, that return value is appended to the conversation as a new message, and the model is called again, now with the tool's result available, to either call another tool or produce a final natural-language answer.

function_calling_loop.py

python

import json

tools_schema = [{
    "name": "get_weather",
    "description": "Get current weather for a city",
    "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
    },
}]

def get_weather(city):
    return {"city": city, "temp_c": 18, "condition": "cloudy"}  # mock implementation

def run_with_tools(user_message, llm_call, max_rounds=4):
    messages = [{"role": "user", "content": user_message}]
    for _ in range(max_rounds):
        response = llm_call(messages, tools=tools_schema)
        if response.get("tool_call") is None:
            return response["content"]
        call = response["tool_call"]
        args = json.loads(call["arguments"])
        result = globals()[call["name"]](**args)   # dispatch to the real function
        messages.append({"role": "assistant", "tool_call": call})
        messages.append({"role": "tool", "name": call["name"], "content": json.dumps(result)})
    return "max rounds exceeded"

Schemas are prompts in disguise

Function names, parameter descriptions, and docstrings all get formatted into the model's context under the hood — a poorly named function (do_thing) or vague parameter description leads to worse tool selection and argument quality than a precise one (search_knowledge_base(query: str, max_results: int) with a clear description of what counts as a good query). Treat tool schemas as prompt engineering surface area, not just an API contract — the wording materially affects accuracy.

Parallel calls, forced calls, and structured output overlap

Most modern APIs support parallel tool calls, where the model can request multiple independent function calls in a single turn (e.g. checking weather in three cities at once), which the application executes concurrently before responding. APIs also typically allow forcing a specific tool call (useful when you know the next step must be a particular function) or disabling tool use entirely for a turn. Function calling is also the same underlying mechanism many providers use to implement structured JSON output — asking the model to 'call' a single pseudo-function whose schema matches the desired output shape is a common trick for reliably getting JSON back.

Tool call patterns

Pattern	Behavior	Use case
Auto (model decides)	Model chooses whether and which tool to call	General agentic behavior
Forced single tool	Model must call a specific named tool	Guaranteed structured extraction
Parallel calls	Model requests multiple independent calls at once	Batched independent lookups
Disabled	Model can only produce text	Final answer turn, or non-agentic contexts

Validate before executing — never trust arguments blindly

A model can hallucinate plausible-looking but invalid arguments (a malformed date, a SQL-injection-shaped string, an out-of-range ID). Production tool-calling layers must validate arguments against the schema, sanitize anything that reaches a database or shell, and apply the same authorization checks you'd apply to a request from an untrusted user — because, functionally, the LLM's output is untrusted input to your system.

What's Next?

Function calling gives a model the ability to act, which immediately raises the question of how to keep those actions safe, bounded, and verifiable — the next layer of the stack is exactly that guardrail and validation problem.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →