Tools and structured outputs

Function calling and structured outputs work in the unified OpenAI format across all providers. You define tools once; whether Claude, GPT, or Gemini answers, the wire format stays the same.

Function calling

Declare tools in the request; the model decides when to call one:

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string"}
            },
            "required": ["city"]
        }
    }
}]

resp = client.chat.completions.create(
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Weather in Vienna?"}],
    tools=tools,
)

When the model wants the tool, the response contains tool_calls with the function name and JSON arguments. You execute the function yourself, append the result as a role: "tool" message, and call the API again; the model then answers using the result.

tool_choice controls the behavior: "auto" (default), "none", "required", or forcing one specific function.

Structured outputs

When you need the answer itself as guaranteed-valid JSON, use response_format:

resp = client.chat.completions.create(
    model="gpt-5",
    messages=[{"role": "user", "content": "Extract: 'Anna Berger, 34, Vienna'"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"},
                    "city": {"type": "string"}
                },
                "required": ["name", "age", "city"]
            }
        }
    },
)

The lighter variant {"type": "json_object"} guarantees valid JSON without enforcing a schema.

Cross-provider notes

• The gateway translates the unified format to each provider's native protocol; you do not handle provider differences.
• Tool definitions count as input tokens, so keep descriptions tight: they are prompt text.
• Streaming and tools combine: tool_calls arrive as deltas you accumulate. See Streaming.
• Schema support varies at the edges between providers; when a model cannot honor a constraint natively, validate on your side before trusting the shape.