Day 23: Tool Use and Function Calling — Connecting LLMs to APIs, Databases and Shells

By default a language model is an isolated reasoning engine. It cannot read the current time, query your private database or run a command. Function calling, also called tool use, is the bridge that connects that engine to real systems, so the model becomes a router that decides what to call while your software does the calling.

In plain terms: The model is a manager who can write instructions but cannot touch anything. It hands a clear, structured request to an assistant (your application), who does the task and reports back.

The four-step protocol

A common misconception is that the LLM runs the code. It does not. Function calling is a deterministic, four-step exchange.

1. Declare the tools. Your application sends the model a list of tools described with JSON schemas: the name, what it is for and its typed parameters.
2. Model invocation. If the question needs outside data or an action, the model replies with a structured tool call (the function name and exact arguments) instead of ordinary text.
3. Client-side execution. Your application intercepts that reply, validates it and runs the real code: a SQL query, a REST call, a script.
4. Observation injection. Your application sends the result back as a tool message, and the model uses it to finish its answer.

Type safety and validation

Free text breaks software pipelines. Use structured arguments and validate them with JSON Schema or a library such as Pydantic, so a malformed value is rejected before it reaches your backend. Then add checks the model cannot override: an allow-list of tools and a permission check for the current user.

Here is the loop with an OpenAI-compatible API. Ollama and vLLM from Day 16 expose the same interface, so the pattern works with a local model too. It assumes a model that supports tool calling, and get_gpu_status is a stand-in for your own function.

import json
from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()                     # or OpenAI(base_url="http://localhost:11434/v1", api_key="x")

class GpuArgs(BaseModel):             # type-safe arguments
    node: str

def get_gpu_status(node: str) -> dict:
    return {"node": node, "status": "degraded"}   # replace with a real lookup

TOOLS = [{"type": "function", "function": {
    "name": "get_gpu_status",
    "description": "Returns GPU health for a node",
    "parameters": GpuArgs.model_json_schema(),
}}]
ALLOWED = {"get_gpu_status": (GpuArgs, get_gpu_status)}

messages = [{"role": "user", "content": "Is gpu-07 healthy?"}]
reply = client.chat.completions.create(model="gpt-4o-mini", messages=messages, tools=TOOLS)
msg = reply.choices[0].message

if msg.tool_calls:
    messages.append(msg)
    for call in msg.tool_calls:
        schema, fn = ALLOWED[call.function.name]                       # allow-list
        args = schema.model_validate_json(call.function.arguments)     # validate
        result = fn(**args.model_dump())                               # execute
        messages.append({"role": "tool", "tool_call_id": call.id,
                         "content": json.dumps(result)})               # observation
    final = client.chat.completions.create(model="gpt-4o-mini", messages=messages)
    print(final.choices[0].message.content)

Restricting the model to proposing structured arguments keeps execution under your control: authenticated, validated and logged inside your own application.

Coming Up Next

Day 24: Multi-agent orchestration: supervisor, worker and evaluator patterns.

#FunctionCalling #ToolUse #APIs #Pydantic #SoftwareArchitecture