Fresh 2026

Tool calling

How to define tools, send tool-calling requests, handle responses, and return results across SDKs.

Tool calling lets models invoke functions you define. Gateway passes your tool definitions to the model, the model suggests which tool to call with what arguments, and your application executes it and sends back the result. This works consistently across vendors, with capability checks applied to the exact execution route before the request is sent upstream.

Define tools

Tools are defined as function objects with a name, description, and a JSON Schema for the parameters.

{
  "type": "function",
  "name": "get_weather",
  "description": "Get the current weather for a location.",
  "parameters": {
    "type": "object",
    "properties": {
      "location": { "type": "string", "description": "City name" }
    },
    "required": ["location"]
  }
}

Send a request with tools

from merge_gateway import MergeGateway

client = MergeGateway(api_key="YOUR_API_KEY")

tools = [
    {
        "type": "function",
        "name": "get_weather",
        "description": "Get the current weather for a location.",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City name"},
            },
            "required": ["location"],
        },
    }
]

response = client.responses.create(
    model="openai/gpt-5.1",
    input=[
        {"type": "message", "role": "user", "content": "What's the weather in San Francisco?"},
    ],
    tools=tools,
    tool_choice="auto",
)

import { MergeGateway } from "merge-gateway-sdk";

const client = new MergeGateway({ apiKey: "YOUR_API_KEY" });

const tools = [
  {
    type: "function" as const,
    name: "get_weather",
    description: "Get the current weather for a location.",
    parameters: {
      type: "object",
      properties: {
        location: { type: "string", description: "City name" },
      },
      required: ["location"],
    },
  },
];

const response = await client.responses.create({
  model: "openai/gpt-5.1",
  input: [
    { type: "message", role: "user", content: "What's the weather in San Francisco?" },
  ],
  tools,
  toolChoice: "auto",
});

Handle the response

When the model calls a tool, the response contains a tool_use content block with finish_reason: "tool_use".

{
  "output": [
    {
      "role": "assistant",
      "finish_reason": "tool_use",
      "content": [
        {
          "type": "tool_use",
          "id": "call_abc123",
          "name": "get_weather",
          "input": { "location": "San Francisco" }
        }
      ]
    }
  ]
}

Send tool results

After executing the function, send the result back with a tool_result input to continue the conversation.

# 1. Extract the tool call from the response
tool_call = response.output[0].content[0]

# 2. Execute your function
weather_data = get_weather(tool_call.input["location"])

# 3. Send the result back
follow_up = client.responses.create(
    model="openai/gpt-5.1",
    input=[
        {"type": "message", "role": "user", "content": "What's the weather in San Francisco?"},
        {"type": "message", "role": "assistant", "content": [
            {"type": "tool_use", "id": tool_call.id, "name": tool_call.name, "input": tool_call.input},
        ]},
        {"type": "tool_result", "tool_use_id": tool_call.id, "content": weather_data},
    ],
    tools=tools,
)

print(follow_up.output[0].content[0].text)

Tool choice

Control whether and how the model uses tools.

Value	Behavior
"auto"	Model decides whether to call a tool (default)
"none"	Model will not call any tools
"required"	Model must call at least one tool
{"type": "function", "function": {"name": "get_weather"}}	Model must call the specified tool

OpenAI SDK

Tool calling works through the OpenAI SDK with a simple update of changing the base url and API key.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api-gateway.merge.dev/v1/openai",
)

response = client.chat.completions.create(
    model="gpt-5.1",
    messages=[
        {"role": "user", "content": "What's the weather in San Francisco?"},
    ],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the current weather for a location.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {"type": "string", "description": "City name"},
                    },
                    "required": ["location"],
                },
            },
        }
    ],
    tool_choice="auto",
)

AI SDK (Vercel)

Tool calling works with the Vercel AI SDK using the tool() helper and Zod schemas.

import { createOpenAI } from "@ai-sdk/openai";
import { generateText, tool } from "ai";
import { z } from "zod";

const gateway = createOpenAI({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api-gateway.merge.dev/v1/ai-sdk",
});

const { text, toolResults } = await generateText({
  model: gateway("openai/gpt-4o"),
  prompt: "What's the weather in San Francisco?",
  tools: {
    getWeather: tool({
      description: "Get the current weather for a location.",
      parameters: z.object({
        location: z.string().describe("City name"),
      }),
      execute: async ({ location }) => {
        return { temperature: 72, condition: "sunny" };
      },
    }),
  },
});

FAQ

Use GET /v1/models and inspect vendors..capabilities.supports_tool_calling for the exact route you plan to use.

Yes. The response may contain multiple tool_use content blocks. Send a tool_result for each one before continuing the conversation.

No. Tool parameter schemas are passed directly to the provider. Validation is handled by the model and your application.

Capability checks

Gateway validates tool support against the exact vendor route that will serve the request. Use GET /v1/models and inspect vendors..capabilities.supports_tool_calling and vendors..capabilities.supports_tool_choice when deciding whether to send tools or a specific tool_choice value.