An AI gateway SLA can look simple until the first model incident. A buyer sees an availability percentage, a support address, and a promise that the gateway routes requests. The production team still has to answer harder questions: which endpoint is covered, which upstream provider failed, whether fallback changed the model contract, what evidence support needs, and whether any remedy is automatic.
Use this AI gateway SLA checklist to turn vendor language into an evidence file. The goal is not to negotiate by slogan. The goal is to know what the SLA covers, what it excludes, how routing and fallback are proven, and what your team should review before renewal.
For Flatkey specifically, the public Service Level Agreement checked on July 6, 2026 says the covered scope is the hosted dashboard, API gateway, routing, metering, and account services that Flatkey directly operates. It targets 99.5% monthly availability for covered endpoints, excludes third-party AI model provider outages and provider-side failures, asks customers to send request IDs, timestamps, errors, endpoint, account email, and impact summary for support, and does not create automatic credits unless a separate written agreement says otherwise. Treat that as the starting evidence, then verify your account, traffic class, fallback policy, and support process before production traffic depends on it.
Quick Answer: AI Gateway SLA Questions
| SLA area | Ask this before signing or renewing | Evidence to keep | |---|---|---| | Covered service | Which dashboard, API gateway, routing, metering, account, and support functions are covered? | Current SLA URL, contract schedule, endpoint list, and dated screenshots. | | Availability math | Is availability measured by vendor monitoring, customer probes, public status, or a contract report? | Monthly availability report, probe log, incident export, and calculation method. | | Third-party providers | Are upstream model outages, rate limits, regional restrictions, model changes, and provider-side failures excluded? | Provider status links, model error bodies, route logs, and vendor exclusion text. | | Routing | Which model, provider, account, region, or group was attempted first? | Request ID, selected route, model ID, endpoint family, key owner, and timestamp. | | Fallback | Which errors trigger retry, fallback, queueing, or fail-closed behavior? | Fallback policy, route trace, retry count, final model, and accepted-output proof. | | Support | What evidence must be sent, and what severity path applies? | Ticket template, request IDs, endpoint, timestamps, errors, impact summary, and response timeline. | | Remedies | Are service credits automatic, case-by-case, or only in a separate written agreement? | SLA remedy clause, contract amendment, credit request, and finance approval. | | Renewal | What must be reviewed before the next term? | Incident register, fallback exceptions, support outcomes, quota events, pricing changes, and route approvals. |
The practical AI gateway SLA answer is this: separate gateway responsibility from upstream provider responsibility, then test the route. If the buyer cannot inspect the route, fallback, support ticket, and monthly availability evidence, the SLA is not ready for production governance.
Separate Gateway Uptime From Upstream Model Availability
The first AI gateway SLA question is whether the vendor is promising the gateway, the upstream model, or the full end-user outcome. Those are different layers.
Flatkey's current public SLA makes that separation explicit: covered Flatkey-operated services are in scope, while third-party model providers, provider outages, provider rate limits, policy changes, regional restrictions, model behavior, and provider-side failures are outside the SLA. That is common enough in gateway buying that procurement should never treat a single uptime percentage as "all model requests will succeed."
Use this split in the evidence file:
| Layer | What can fail | Buyer evidence | |---|---|---| | Client and SDK | Bad base URL, bad key, unsupported endpoint, timeout too low, parser error. | Client config, SDK version, request body, timeout, and local logs. | | Gateway | Authentication, routing, metering, gateway endpoint, dashboard, usage log, account services. | Gateway SLA, request ID, route trace, status, usage record, and support ticket. | | Upstream provider | Provider outage, provider quota, model retirement, regional block, policy rejection, model-specific behavior. | Provider status, provider quota page, upstream error body, model ID, and fallback decision. | | Business workflow | User impact, queue depth, degraded output, missed internal objective. | Incident timeline, affected traffic class, customer impact summary, rollback decision. |
This is where an AI gateway SLA becomes operational. The buyer should ask for a route-level trace, not only a monthly uptime number.
Routing And Fallback Questions
Routing is the part of an AI gateway SLA that most often gets under-specified. A gateway may support multiple models and providers, but production reliability depends on which route was allowed for each traffic class.
Official gateway documentation from other vendors shows why the details matter. Vercel's AI Gateway docs describe provider routing and model fallbacks, including control over routing order and fallback behavior. Its model fallback docs say backups can be tried in order when a primary model fails or is unavailable. Pydantic AI Gateway docs describe routing groups where priority and weight decide failover or load balancing, with fallthrough when a higher-priority member is unavailable, rate-limited, or errors. These are useful category examples, not Flatkey guarantees. They show the questions a buyer should ask of any gateway, including Flatkey.
Use this routing table before approving automatic fallback:
| Failure signal | Default buyer question | Safe evidence | |---|---|---| | Gateway 401 or 403 | Was the key, account, environment, or policy scope wrong? | Key owner, environment, auth error, and policy denial record. | | Model not found | Did the requested model exist in the current catalog for this account? | Catalog row, model ID, endpoint family, and publish-date or test-date proof. | | Provider 429 | Was this account, region, model, or deployment quota exhausted? | Quota page, upstream error body, retry-after data, and fallback policy. | | Provider 5xx or timeout | Should the gateway retry, switch same-model route, queue, or fail closed? | Route trace, retry count, timeout threshold, final result, and cost record. | | Context limit or unsupported input | Is fallback allowed if the backup model changes context, tools, vision, or media behavior? | Model contract, feature test, accepted-output check, and product owner approval. | | Policy or safety rejection | Is another provider allowed to receive the same request? | Data boundary, provider approval, prompt classification, and compliance sign-off. | | Unknown failure | What is the stop condition? | Incident runbook, max attempts, fail-closed rule, and support escalation. |
The strongest AI gateway SLA evidence is a route trace that names the starting route, the fallback rule, the final route, and the reason each transition happened.
Fallback Is Not Always A Reliability Win
Procurement often asks whether the gateway has automatic fallback. Platform teams should answer with a second question: fallback to what?
An automatic switch from one account to another account for the same approved model can be reasonable. A switch from one model family to another model family can change quality, data handling, latency, output format, tool behavior, image or video policy, and cost. Some traffic should fail closed because hidden fallback would make the incident worse.
Define fallback by traffic class:
| Traffic class | Fallback posture | Why | |---|---|---| | Customer chat | Use only approved equivalent routes, or return a controlled error. | User-facing quality and safety must stay predictable. | | Internal summarization | Retry, queue, or use a lower-cost approved route. | Latency can be traded for cost and completion. | | Evaluations and benchmarks | Fail closed. | Hidden model changes corrupt comparison data. | | Finance-sensitive batch jobs | Queue or require approval before fallback. | Retries and backup models can multiply spend. | | Agent workflows with tools | Fallback only after tool-call behavior is tested. | Tool schemas and streaming behavior may differ by provider. | | Regulated or customer-isolated traffic | Fail closed unless the backup provider and region are pre-approved. | Data boundary and procurement evidence matter more than automatic completion. |
This is the buyer-owned test: run the same prompt through the primary route and the proposed fallback route, then compare response shape, logs, cost, latency, accepted output, and support evidence. Do not treat an AI gateway SLA as complete until the fallback evidence exists.
Support Questions For The Evidence Packet
Flatkey's current SLA asks customers to contact support with the account email, affected endpoint, request IDs if available, timestamps, error messages, and impact summary. That is a useful template for any AI gateway SLA support process.
Before launch, create a support packet with these fields:
| Field | Why it matters | |---|---| | Account email and workspace | Lets support find the correct tenant, key, and entitlement. | | Affected endpoint | Separates chat, responses, messages, image, video, and dashboard incidents. | | Request IDs | Connects app logs to gateway and upstream traces. | | Timestamps with timezone | Prevents monthly availability and incident windows from drifting. | | Model ID and route class | Shows which route policy and provider family were involved. | | Error messages and status codes | Separates auth, quota, timeout, provider, and parser failures. | | Impact summary | Explains affected users, revenue, jobs, queue depth, or internal workflow. | | Customer-side retries | Shows whether duplicate attempts increased cost or load. | | Required action | Clarifies whether you need diagnosis, route disablement, credit review, or contract follow-up. |
Link this support packet to your audit logs AI API usage process. If request IDs and route evidence are not available after an incident, the support process becomes a memory exercise.
Remedy And Credit Questions
The remedy section of an AI gateway SLA matters because it often defines what the buyer can actually recover. A 99.5% target without a credit clause is different from a committed SLA with automatic service credits, written notice requirements, exclusions, and claim deadlines.
Flatkey's public SLA says there are no automatic service credits, refunds, penalties, or liquidated damages unless a separate written agreement provides a different remedy. Any goodwill adjustment, balance correction, or support remediation is handled case by case under the user agreement and applicable policies.
That does not make the SLA unusable. It means procurement should ask the right questions:
- Is the public SLA the whole agreement, or is there a separate enterprise schedule?
- If credits exist, what is the claim window?
- What evidence must the customer provide?
- Are upstream provider outages excluded even when the gateway selected that upstream?
- Are scheduled maintenance and emergency security actions excluded?
- Are latency, model quality, throttling, and fallback behavior covered or only availability?
- Does the remedy apply to prepaid balance, invoice credits, refunds, support remediation, or contract termination rights?
- Who inside finance approves the final claim?
The answer should live in the same vendor file as the AI API vendor risk assessment. Legal and finance need the same evidence that platform engineers use during incident review.
Quotas, Rate Limits, And Provider Dependencies
An AI gateway SLA can be valid while a request still fails because an upstream provider quota is exhausted. This is why quota evidence belongs in the SLA review.
Microsoft's Azure OpenAI quota documentation says token-per-minute and request-per-minute limits are scoped per region, subscription, and model or deployment type. AWS Bedrock documentation similarly directs teams to service quotas for Bedrock resources and notes that model inference is controlled by token usage quotas. These provider facts are not Flatkey promises. They are reminders that upstream account shape can decide whether fallback is available during a real incident.
Add these checks to the launch file:
| Dependency check | Evidence | |---|---| | Upstream quota scope | Region, account or subscription, model, deployment type, TPM/RPM or provider equivalent. | | Gateway quota scope | Key, team, environment, model group, budget cap, and reset window. | | Retry budget | Maximum retry count, jitter/backoff policy, and cost guardrail. | | Fallback capacity | Backup route quota, model availability, and provider approval. | | Throttling behavior | Error code, retry-after signal, support escalation, and user-facing handling. |
If quota is not measured by the same owner as routing, the AI gateway SLA review will miss the most common failure mode: the route exists, but the account cannot absorb production traffic.
Flatkey Staging Review
For Flatkey, the public product proof checked on July 6, 2026 supports one AI API gateway, one API key, the OpenAI-compatible router base URL https://router.flatkey.ai/v1, dashboard-oriented usage and billing visibility, pricing review, and routing-related positioning. The live pricing API snapshot returned success: true, pricing version a42d372ccf0b5dd13ecf71203521f9d2, 45 model rows, 48 vendors, supported endpoint metadata for Anthropic messages, Gemini generateContent, image generation, OpenAI chat completions, and video requests, plus availability states of available and unknown_failure across rows.
Use that as dated public evidence only. It does not prove your account's SLA remedy, support tier, route success, fallback eligibility, permanent model availability, latency, compliance scope, or account-specific pricing.
A Flatkey AI gateway SLA staging review should include:
- Open the current pricing page and select the exact model or route family.
- Confirm the model row and endpoint family in your account or dashboard.
- Run a low-risk request through
https://router.flatkey.ai/v1. - Capture request ID, model ID, endpoint, timestamp, status, usage units, cost evidence, and key owner.
- Run one approved failure test: bad model, quota boundary, timeout, or disabled fallback.
- Confirm where support evidence appears in logs.
- Decide which traffic classes can fallback and which must fail closed.
- Add the support packet fields to the incident runbook.
- Save the current SLA, pricing, route proof, and support path in the vendor evidence folder.
When that evidence exists, platform, procurement, and finance can evaluate the same file instead of arguing from screenshots.
Renewal Trigger Checklist
Do not review an AI gateway SLA only once. Add renewal triggers so the buyer file stays current.
| Trigger | What to re-check | |---|---| | SLA page or contract update | Scope, exclusions, remedies, support process, and contact details. | | New model family | Endpoint, provider, quota, data boundary, pricing unit, and fallback policy. | | New production traffic class | Failover posture, support severity, and owner approval. | | Incident or near miss | Route trace, support timeline, user impact, cost impact, and preventive action. | | Provider quota change | Backup capacity, throttling behavior, and traffic allocation. | | Pricing change | Request-level cost, retry budget, fallback model cost, and finance threshold. | | Security or compliance review | Logging scope, retention, provider approval, and access review. |
Tie this checklist back to the enterprise AI API gateway checklist. The SLA is one part of the enterprise evidence packet, not a standalone trust badge.
Common Mistakes
| Mistake | Why it hurts | Better check | |---|---|---| | Treating a gateway uptime target as a model-success guarantee | Upstream providers, quota, policy, and model behavior can be excluded. | Split gateway, provider, client, and business workflow evidence. | | Asking whether fallback exists, but not what it switches to | Backup models can change output, data boundary, features, and cost. | Approve fallback by traffic class and model contract. | | Keeping no request IDs | Support cannot connect customer logs to route logs reliably. | Add request ID capture to every production route. | | Ignoring quota scope | A backup route can exist but still throttle during peak traffic. | Store upstream and gateway quota evidence together. | | Assuming credits are automatic | Many SLAs require a separate agreement, claim process, or case-by-case review. | Save remedy terms and claim owner before launch. | | Reviewing only at purchase time | Model catalogs, quotas, and support paths change. | Add renewal triggers and incident-driven rechecks. |
FAQ
What is an AI gateway SLA?
An AI gateway SLA defines the availability target, scope, exclusions, support process, and remedies for the gateway services a vendor operates. It may not cover third-party model provider outages, quota limits, model behavior, customer configuration, or every fallback outcome.
Does an AI gateway SLA guarantee every model request succeeds?
No. An AI gateway SLA can cover gateway availability while excluding upstream provider failures, provider rate limits, policy changes, model behavior, customer credentials, scheduled maintenance, or customer-side issues. Check the exclusion text and route evidence.
What should procurement ask about model fallback?
Procurement should ask which errors trigger fallback, which models or providers are approved backups, whether fallback changes data handling or output behavior, how retries are billed, and where route evidence is stored after an incident.
What evidence should support receive during an SLA incident?
Support should receive account email, affected endpoint, request IDs, timestamps, error messages, model ID, route class, impact summary, and customer retry behavior. Flatkey's public SLA currently asks for several of these fields, including endpoint, request IDs, timestamps, errors, and impact summary.
Are service credits automatic?
Not always. Flatkey's public SLA says automatic service credits, refunds, penalties, or liquidated damages are not created unless a separate written agreement says otherwise. Buyers should confirm the remedy clause, claim deadline, and approval path.
How often should teams review an AI gateway SLA?
Review an AI gateway SLA at purchase, renewal, major route change, new model family, new production traffic class, pricing change, quota change, and after every material incident or near miss.
Final Recommendation
An AI gateway SLA is useful only when it becomes testable. Start with the public or contracted SLA, separate gateway scope from upstream provider scope, run route and fallback tests, save request IDs, define support evidence, and add renewal triggers.
Flatkey can help teams centralize model access, routing, pricing review, and usage evidence behind one gateway workflow. Before production rollout, verify the exact model, endpoint, route behavior, support process, and SLA evidence for your account. Then get a key when you are ready to test an AI gateway SLA workflow with real staging traffic.
