Tool Use & Function Calling

Start with a plain picture. A language model is a text-in, text-out system: it reads words and predicts more words. That is all it does. So when you want it to fetch live data or take a real action, there has to be a hand-off — the model says what it wants, and something outside the model actually does it. That hand-off is tool use.

Here is the single most important idea, and the one beginners most often get wrong: the model never runs any code. When you give a model a calculator tool, it does not compute anything. It looks at the conversation, decides a calculation would help, and emits a structured message that says, in effect, "please call calculate with the argument expression='42*17'." Your application receives that request, runs the real function, and sends the answer back. The model then continues reasoning with the result in hand.

Think of the model as a brilliant analyst locked in a room with a phone. It cannot leave or touch anything itself — but it can pick up the phone and dictate precise requests to an assistant outside, then react to whatever the assistant reports back. Function calling is that phone.

This design is what lets an LLM reach beyond its training data: it can read live data, run code, and take real-world actions, all while staying a pure text-in, text-out system. "Function calling" (OpenAI's 2023 term) and "tool use" (Anthropic's term) describe the same mechanism — we'll use them interchangeably.

If the model is going to request a function, it first needs to know which functions exist and how to call them. You provide that as a list of schemas — think of each schema as the "menu entry" for one tool: its name, what it's for, and the inputs it takes. Every schema has three parts:

1. name — a short identifier (alphanumeric, 1–64 chars), e.g. get_weather.
2. description — natural language explaining when and how to use the tool. This is the single biggest factor in whether the model picks the right tool. Treat it as prompt engineering, not a code comment.
3. parameters (OpenAI) / input_schema (Anthropic) — a JSON Schema object defining the expected inputs, their types, and which are required. (JSON Schema is just a standard JSON format for describing the shape of data — types, fields, and constraints.)

{
  "name": "get_weather",
  "description": "Get the current weather for a city. Use when the user asks about temperature, rain, or conditions in a specific location.",
  "input_schema": {
    "type": "object",
    "properties": {
      "city": { "type": "string", "description": "City name, e.g. 'Paris'" },
      "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
    },
    "required": ["city"]
  }
}

The schema is sent as part of every request and counts as input tokens — the tools array, the model's tool calls, and the results you return all add to your token bill.

Now put the model and your code in conversation. A single tool call is a small back-and-forth: the model asks, your code answers, and you hand the answer back so the model can keep going. Concretely it's a five-step loop:

1. Send the conversation plus the tools array to the model.
2. Receive a response. If the model wants a tool, it stops with stop_reason: "tool_use" (Anthropic) or finish_reason: "tool_calls" (OpenAI), and the response carries a structured call with the tool name and arguments.
3. Execute the real function in your code with those arguments.
4. Return the result back in a new message — a tool_result block (Anthropic) or a role: "tool" message (OpenAI).
5. Continue: the model reads the result and either calls another tool or writes its final answer.

Steps 1–5 repeat until the model returns plain text. This is exactly the ReAct loop — reason, act, observe — implemented at the API level.

response = client.messages.create(model="claude-opus-4-8", messages=messages, tools=tools)

while response.stop_reason == "tool_use":
    tool = next(b for b in response.content if b.type == "tool_use")
    result = run_tool(tool.name, tool.input)        # your code executes
    messages.append({"role": "assistant", "content": response.content})
    messages.append({"role": "user", "content": [{
        "type": "tool_result", "tool_use_id": tool.id, "content": str(result)
    }]})
    response = client.messages.create(model="claude-opus-4-8", messages=messages, tools=tools)

Ordering is strict: the tool result must immediately follow the assistant message that requested it — interleaving other messages causes a 400 error.

Two controls let you tune how many tools run per round-trip and whether the model is allowed to skip tools at all.

Parallel tool calls

Some tasks need several tools that don't depend on each other. Rather than make one round-trip per tool, the model can emit them all in one response. Ask "what's the weather in Tokyo, London, and Cairo?" and a modern model returns three get_weather calls at once. Your code runs them concurrently (e.g. asyncio.gather) and returns all three results together. This collapses latency from the sum of the calls to the duration of the slowest one.

Note the division of labor: the model decides what can run in parallel and emits the calls; your code is responsible for actually running them concurrently. The model is stateless and runs nothing itself.

tool_choice

By default the model decides whether to use a tool at all ("auto"). You can override that:

The concepts line up, but the names differ — OpenAI's required is Anthropic's any, a classic migration bug. Forcing a tool (any/tool/required) also adds ~100 system-prompt tokens on Anthropic and, importantly, is incompatible with Claude's extended thinking — only auto/none work with reasoning mode.

How does the model know which tool to reach for? Not by running anything — it matches meaning. It reads the user's intent, compares it against each tool's description, and picks the closest fit, the same way you'd skim function docs to find the right call. Three things steer that choice:

1. Tool descriptions — the primary signal. Clear, specific descriptions win.
2. System prompt policy — explicit instructions like "always verify prices with the lookup_price tool before answering."
3. What's already in context — models skip tools for stable knowledge they already have, and reach for tools for fresh or external facts.

Two reliability levers matter in production:

• Strict mode (strict: true) guarantees the model's arguments conform to your schema. It is an OpenAI-specific option that requires additionalProperties: false, every field listed in required, and optional fields typed as e.g. ["string", "null"]. It is off by default — you must opt in. Anthropic does not expose an equivalent flag; Claude generally follows the schema but you should validate inputs in your code regardless. Without strict mode (or your own validation), arguments can partially violate the schema.
• Tool-set size — more tools is not better. Past ~30–50 tools, selection accuracy degrades. Keep the active set scoped to the task, or for large catalogs use dynamic tool pre-filtering (Anthropic ships this as Tool Search, using regex and BM25 modes) to surface relevant tools on demand before the model context fills with irrelevant schemas.

Under the hood, models are fine-tuned and RL-trained on curated tool-call datasets to recognize when a structured call beats a text answer — quality is strongly dataset-dependent.

Whatever your tool returns is the only thing the model sees of the outside world — it becomes the model's observation, the next thing it reasons about. So return results that are useful, bounded, and machine-readable. A 50,000-row dump wastes context; a summarized, structured result keeps the agent on track.

Errors deserve the same care. It's tempting to treat a failed tool call as something to hide or crash on, but to the model an error is just more information — feedback it can act on. When a tool fails, return a descriptive error rather than crashing the loop:

try:
    data = lookup_order(order_id)
    result = {"type": "tool_result", "tool_use_id": tid, "content": json.dumps(data)}
except OrderNotFound:
    result = {
        "type": "tool_result", "tool_use_id": tid, "is_error": True,
        "content": "No order found with that ID. Ask the user to re-check the number."
    }

Anthropic uses the is_error flag; OpenAI returns the error string in the role: "tool" message. Either way, a good error message ("Rate limit exceeded. Retry after 60 seconds.") lets the model self-correct. Claude will typically retry an invalid call 2–3 times with corrections before giving up gracefully. Vague errors ("Error 500") leave it stuck. Write tool errors the way you'd write them for a junior engineer who can only see that one line.

Good news first: the mechanism is identical everywhere — model asks, your code acts, you return the result. What changes between providers is only the wire format: the field names and the exact shape of the JSON. That sounds trivial, but a handful of these differences silently break code during a migration. Here are the ones that actually bite:

The arguments format is the sharpest edge: OpenAI hands you a JSON string you must parse; Anthropic and Google give you a ready object. Two more notes: OpenAI's old functions parameter is deprecated (as of late 2023) — use tools with type: "function". And above the raw APIs sits the Model Context Protocol (MCP) — an open standard released by Anthropic in November 2024, adopted by OpenAI (March 2025) and Google, and since donated to the Linux Foundation's Agentic AI Foundation (December 2025) — that standardizes tool discovery and invocation across providers, so you describe a tool once and any MCP-aware model can use it. Framework abstractions (Vercel AI SDK, LangChain, LlamaIndex) normalize these differences for you.