Multi-model SDK migration should not start by replacing every client library in your app. If your production code already uses the OpenAI Python or Node SDK for chat completions, the safer first move is to keep that client surface stable, move provider choice into configuration, and prove Claude, Gemini, and DeepSeek behavior behind a compatible route before you widen traffic.
Flatkey is built for teams that want one key, an OpenAI-compatible routing layer, model access across providers, and usage and billing review in one place. That makes it useful for a multi-model SDK migration, but it does not remove the need to test endpoint families, model aliases, streaming, tool calls, and usage records. Treat every code sample here as a template until you validate it with your own Flatkey key and current model catalog.
The practical goal is simple: keep the request object your app already knows, isolate the few values that must change, and build a proof packet before cutover.
Multi-Model SDK Migration Decision Matrix
Use this matrix before you change code. It separates what can stay stable from what must become configurable.
| Layer | Keep Stable | Make Configurable | Proof To Save |
|---|---|---|---|
| SDK client | Existing OpenAI SDK import and chat completion call path | API key, base URL, timeout, retry policy | SDK version and redacted startup config |
| Request shape | messages, model, stream, basic tools where supported | Provider-specific extras and unsupported parameters | Minimal request and raw response |
| Model selection | App-level aliases such as support_chat or analysis_fast | Gateway model alias and endpoint family | Model catalog screenshot or export |
| Operations | Existing logs, request IDs, and error handling | Provider, route, usage, quota, and rollback metadata | Dashboard/readback evidence |
| Release | Current app deployment path | Percentage rollout and rollback environment variables | Rollback owner and previous config |
This is the core discipline of multi-model SDK migration: do not mix library replacement, prompt migration, provider evaluation, billing validation, and production rollout into one deploy.
What The Official Docs Say About Compatibility
OpenAI's SDKs already support the pattern you need. The current OpenAI Python README documents the OpenAI client and a base_url option, and the Node README documents the standard OpenAI client and chat completions. OpenAI's API reference says Chat Completions creates a model response for a conversation, supports streaming, and returns a chat completion object or stream chunks.
If you are splitting the rollout by runtime, pair this checklist with the OpenAI Python SDK custom base URL guide and the OpenAI Node SDK base URL guide. If the first decision is provider routing rather than client setup, use the Claude vs GPT API routing guide before you choose the test workload.
Provider compatibility differs:
| Provider | Official Compatibility Signal | Migration Implication |
|---|---|---|
| OpenAI SDK | Python supports base_url; Node uses baseURL; Chat Completions has messages, model, stream, tools, and usage fields. | Keep the OpenAI client wrapper for the first migration slice. |
| Anthropic / Claude | Anthropic publishes an OpenAI SDK compatibility layer, but says it is mainly for testing and comparison, not the long-term production path for most use cases. The native Messages API uses /v1/messages, max_tokens, messages, and a top-level system parameter. | Use compatibility for quick evaluation, but test native Claude requirements before relying on provider-specific features. |
| Google Gemini | Google says Gemini models are accessible through OpenAI Python, TypeScript/JavaScript libraries, and REST by changing the API key, base URL, and model. | Gemini is a good candidate for an OpenAI-shaped smoke test, with reasoning and model-specific behavior verified separately. |
| DeepSeek | DeepSeek says its API format is compatible with OpenAI and Anthropic; its first-call docs list OpenAI and Anthropic base URLs and OpenAI-format examples. | DeepSeek can fit the OpenAI client path, but model-name deprecations and thinking settings need explicit checks. |
| Flatkey | Flatkey's public model page exposes endpoint types such as openai, anthropic, and gemini; the homepage shows an OpenAI-compatible chat completions route. | Select models by endpoint family, not provider name alone. |
The important lesson is that "OpenAI-compatible" usually means the client can send an OpenAI-shaped request. It does not mean every parameter, tool schema, reasoning control, media input, or usage field behaves identically across providers.
Step 1: Freeze The Client Boundary
Start the multi-model SDK migration by putting the OpenAI client behind one local factory. The factory should be the only place that knows the key, base URL, timeout, and model alias.
import os
from openai import OpenAI
def make_llm_client() -> OpenAI:
return OpenAI(
api_key=os.environ["FLATKEY_API_KEY"],
base_url=os.getenv("FLATKEY_BASE_URL", "https://router.flatkey.ai/v1"),
timeout=float(os.getenv("LLM_TIMEOUT_SECONDS", "60")),
max_retries=int(os.getenv("LLM_MAX_RETRIES", "2")),
)
client = make_llm_client()
completion = client.chat.completions.create(
model=os.environ["FLATKEY_MODEL"],
messages=[
{"role": "system", "content": "You answer with short implementation notes."},
{"role": "user", "content": "Reply with exactly: migration probe ok"},
],
)
print(completion.choices[0].message.content)
For Node:
import OpenAI from "openai";
export function makeLlmClient() {
return new OpenAI({
apiKey: process.env.FLATKEY_API_KEY,
baseURL: process.env.FLATKEY_BASE_URL ?? "https://router.flatkey.ai/v1",
timeout: Number(process.env.LLM_TIMEOUT_MS ?? 60000),
maxRetries: Number(process.env.LLM_MAX_RETRIES ?? 2),
});
}
const client = makeLlmClient();
const completion = await client.chat.completions.create({
model: process.env.FLATKEY_MODEL,
messages: [
{ role: "system", content: "You answer with short implementation notes." },
{ role: "user", content: "Reply with exactly: migration probe ok" },
],
});
console.log(completion.choices[0].message.content);
The app should call make_llm_client() or makeLlmClient(), not construct clients across controllers, workers, and scripts. That single boundary is what lets a multi-model SDK migration stay reversible.
Step 2: Build A Model And Endpoint Map
Do not point every workload at one new model name. Start with a map that names the workload, endpoint family, current provider, target provider, and rollback value.
| Workload | Endpoint Family | Current Path | Target Path | Rollback |
|---|---|---|---|---|
| Support chat | OpenAI-style chat completions | Current OpenAI model | Flatkey chat alias for GPT, Claude, Gemini, or DeepSeek | Previous key, base URL, and model |
| Internal analysis | OpenAI-style chat completions | Current OpenAI model | Flatkey model alias selected for reasoning and cost | Previous model alias |
| Tool workflow | Chat completions with tools | Current tool-compatible model | Candidate model with verified tool behavior | Disable provider route |
| Streaming UI | Chat completions stream | Current stream parser | Candidate provider with raw stream proof | Switch stream traffic back |
| Native Claude feature | Anthropic Messages API | Native Claude client | Keep native if compatibility drops required feature | Do not migrate this path |
Flatkey's model page is useful here because it exposes endpoint types. If a model is listed under a native endpoint family, do not assume it is available through the OpenAI chat path until your account catalog and smoke tests prove it.
Step 3: Run The Four-Provider Smoke Test
The minimum useful multi-model SDK migration smoke test has four provider rows and the same prompt. Keep the prompt boring so response style does not hide transport failures.
| Test | Request | Pass Condition | Failure To Triage |
|---|---|---|---|
| GPT or current OpenAI model | Existing OpenAI-shaped request through Flatkey | Response returns, usage is visible, logs link to expected alias | Wrong key, wrong base URL, duplicate /v1, missing model |
| Claude | Same OpenAI-shaped request through the selected Flatkey alias | Plain response returns; no unsupported feature in the request | System-message mapping, tool schema, native Claude-only features |
| Gemini | Same OpenAI-shaped request through the selected Flatkey alias | Plain response returns; reasoning defaults are acceptable | Model alias, reasoning settings, multimodal assumptions |
| DeepSeek | Same OpenAI-shaped request through the selected Flatkey alias | Plain response returns; deprecation-sensitive alias is current | Model rename, thinking settings, stream parser |
Save the raw request body, redacted headers, response body, HTTP status, model alias, request ID if returned, and dashboard usage entry. If any provider needs a different parameter, move that parameter into a provider profile rather than scattering if provider == branches across your app.
Step 4: Test Streaming And Tool Calls Separately
A plain completion only proves the lowest-risk path. Streaming and tool calls are separate contracts.
For streaming, save:
| Evidence | Why It Matters |
|---|---|
| Response headers | Confirms whether the route returns event-stream data. |
| First event | Catches buffering or proxy parser issues. |
| Final event | Confirms end-of-stream handling and usage behavior. |
| UI parser output | Proves your frontend did not assume one provider's chunk shape. |
For tools, test a tiny schema first:
{
"type": "function",
"function": {
"name": "lookup_order_status",
"description": "Return a test order status.",
"parameters": {
"type": "object",
"properties": {
"order_id": { "type": "string" }
},
"required": ["order_id"]
}
}
}
Do not use your largest production tool schema as the first proof. Anthropic's compatibility documentation, for example, calls out limitations around OpenAI compatibility and recommends the native Claude API for the full Claude feature set. That is exactly the kind of detail a multi-model SDK migration checklist should catch before rollout.
Step 5: Keep Provider Profiles Small
Provider profiles should describe differences, not own business logic.
{
"support_chat": {
"base_url_env": "FLATKEY_BASE_URL",
"model_env": "FLATKEY_SUPPORT_CHAT_MODEL",
"endpoint_family": "openai_chat_completions",
"stream": true,
"tools": "basic",
"fallback_alias": "support_chat_previous"
},
"analysis_fast": {
"base_url_env": "FLATKEY_BASE_URL",
"model_env": "FLATKEY_ANALYSIS_FAST_MODEL",
"endpoint_family": "openai_chat_completions",
"stream": false,
"tools": "none",
"fallback_alias": "analysis_fast_previous"
}
}
Keep native provider features out of the shared OpenAI client path unless you have tests for every target provider. If Claude citations, Gemini-specific media behavior, DeepSeek thinking controls, or a provider-native batch endpoint is required, create a named native adapter and keep the OpenAI-compatible path for workloads that truly fit it.
Step 6: Prove Billing And Rollback
A migration that returns text but loses billing evidence is not done. Before increasing traffic, save:
| Operations Check | Pass Condition |
|---|---|
| Usage visibility | The request appears in Flatkey usage or logs with the expected route/model context. |
| Cost review | Pricing or usage records are visible enough for finance or operations to reconcile. |
| Quota behavior | A low test quota or budget threshold behaves as expected in a non-production key. |
| Error taxonomy | 401, 403, 404, 429, 5xx, and provider errors are mapped to clear owner actions. |
| Rollback | One environment change returns traffic to the previous key/base URL/model. |
This is where Flatkey's unified billing and usage review matters. The goal is not just access to more models. The goal of a multi-model SDK migration is access that product, operations, and finance can inspect after traffic moves.
A Cutover Workflow For Multi-Model SDK Migration
Use this order for the first production slice:
- Pick one low-risk workload with an existing OpenAI SDK client.
- Add the client factory and environment variables without changing the current provider.
- Prove the current model through the new factory.
- Prove one Claude, Gemini, or DeepSeek target with a plain completion.
- Prove streaming and tools only if that workload uses them.
- Verify usage, billing, error handling, and rollback.
- Route a small percentage of traffic to the new model alias.
- Review failures by provider and endpoint family before adding the next workload.
If the first workload needs native provider features, choose a different workload for the first multi-model SDK migration. The first slice should prove the migration pattern, not every model capability.
FAQ
Can I use one OpenAI SDK client for Claude, Gemini, and DeepSeek?
You can often keep an OpenAI SDK client for OpenAI-compatible routes, but you still need provider-specific tests. Google and DeepSeek document OpenAI-compatible access. Anthropic documents an OpenAI SDK compatibility layer, but says it is mainly for testing and comparison rather than the long-term production choice for most use cases.
Should I migrate Responses API traffic the same way?
Only if the target gateway and target model explicitly support the endpoint family you are using. Chat Completions and Responses are not the same contract. Start with the endpoint family your app already uses, then create a separate test plan for Responses.
What is the most common mistake in a multi-model SDK migration?
The most common mistake is treating the base URL change as the whole migration. A real multi-model SDK migration also verifies model names, endpoint families, streaming, tools, usage, quota, error handling, and rollback.
When should I keep a native provider SDK?
Keep a native provider SDK when your workload depends on provider-specific features that the compatibility layer does not cover. Native Claude features, Gemini-specific media paths, or DeepSeek-specific reasoning controls may justify a separate adapter.
Final Checklist
Before you call the migration complete, confirm that you have:
| Item | Complete When |
|---|---|
| Client factory | All app paths use one OpenAI SDK wrapper or a named native adapter. |
| Config split | API key, base URL, model alias, timeout, retries, and route owner are environment-driven. |
| Provider proof | GPT, Claude, Gemini, and DeepSeek candidates were tested with saved raw evidence. |
| Feature proof | Streaming and tools were tested separately from plain chat. |
| Operations proof | Usage, billing, quota, logs, and errors are visible to the right owners. |
| Rollback proof | One documented change returns the workload to the previous route. |
Multi-model SDK migration works best when it is boring: one small workload, one client boundary, one model map, and evidence for every provider you add. When you are ready to run those tests through a unified route, get a Flatkey key and start with a low-risk OpenAI-compatible chat path.
