Fresh 2026

SOP: Route LLM Requests with Gateway

Procedure to send LLM traffic through Merge Gateway and route it across providers with a policy.

Prerequisites

• A Merge account with Gateway enabled.
• A Gateway project and a management API key.

Procedure

1. Create a project. Projects scope keys, policies, and spend. See Projects.
2. Send a first request. Point your existing OpenAI-style client at the Gateway endpoint and call POST /responses. See Quickstart.
3. Choose a routing strategy. Pick single-provider, priority, performance, or intelligent routing based on whether you optimize for simplicity, reliability, latency, or cost-vs-quality. See Routing Policies.
4. Define the policy. Write the routing policy JSON once. Your application code keeps calling the same endpoint.
5. Add cost governance. Set budgets and spend caps. See Cost governance and savings.
6. Enable extras as needed. Tool calling, multimodal inputs, web search, and context compression are policy and request options. See Features.

Verification

• POST /responses returns a completion through the routed provider.
• Failover and cost rules behave as defined when you change the policy (not the code).

Related

• Gateway Routing Flow
• API Overview
• Security