Enterprise Controls and TrustJuly 30, 2026Flatkey Team

API Key Management Best Practices for AI Products (2026 Checklist)

A practical API key management checklist for AI products: keep keys server-side, scope one key per environment, rotate without downtime, verify revocation, scan for leaks, and respond fast when a key is exposed.

API Key Management Best Practices for AI Products (2026 Checklist)

API key management best practices come down to five things: keep keys server-side and out of code, give each environment and workload its own narrowly scoped key, store keys in a secrets manager, rotate and revoke them on a tested schedule, and monitor usage without ever logging the key itself. For AI products the stakes are higher than for most APIs, because a leaked model key can run up a large bill within hours and also authorize access to prompts, files, and tool calls.

This guide turns those principles into a checklist that platform, security, and AI product teams can apply. It covers the key lifecycle, a key inventory template, zero-downtime rotation, CI/CD safeguards, an incident runbook, and code for generating, storing, and verifying keys. Sources include the OWASP Secrets Management Cheat Sheet, Google Cloud's API key best practices, and GitHub secret scanning docs, all checked October 27, 2026.

API key management best practices: the quick checklist

#	Best practice	What it prevents
1	Never hardcode keys in source code, notebooks, or client apps	Leaks through Git history, browser bundles, and mobile binaries
2	Keep provider keys server-side; give clients short-lived session tokens	Users or malware extracting your key from a device
3	Use one key per environment, application, and team member	One leak taking down everything when you revoke
4	Apply the narrowest restrictions available: APIs, models, IPs, quotas	Stolen keys being usable for anything
5	Store keys in a secrets manager, not in tickets, chat, or wikis	Copies you cannot find or rotate
6	Keep a key inventory with owner, workload, and rotation date	Rotation breaking unknown consumers
7	Rotate on a schedule with an overlap window, then revoke the old key	Long-lived keys that outlast employees and vendors
8	Delete keys you no longer use	Forgotten keys with live permissions
9	Turn on secret scanning and push protection	Keys reaching a public or shared repository
10	Never log keys or authorization headers; log a key ID or version	Secrets leaking through logs, traces, and error reports
11	Set spend limits and alert on unusual usage	Surprise bills from a stolen or runaway key
12	Have a revoke-first incident runbook	Long exposure while teams debate whether a leak was used

The rest of this guide explains how to implement each practice for AI workloads, where a key does more than authenticate: it authorizes a data path for prompts, documents, and model outputs.

The API key lifecycle

OWASP recommends managing secrets across their full lifecycle: creation, rotation, revocation, and expiration, with auditing at each step (OWASP Secrets Management Cheat Sheet). For an AI API key, that lifecycle looks like this:

Stage	Owner action	Evidence to keep
Request	Justify the workload, environment, and models needed	Ticket or inventory entry
Create	Generate in the provider console or API with a descriptive name	Key ID, creation time, creator
Restrict	Limit models, APIs, networks, and budget	Restriction settings
Store	Save directly into a secrets manager; never paste elsewhere	Secret path and access policy
Deliver	Inject at runtime into the one workload that needs it	Deployment binding
Monitor	Track requests, tokens, spend, and last-used time	Usage dashboard and alerts
Rotate	Create a replacement, cut over, verify	Rotation record
Revoke	Invalidate at the provider, confirm a 401	Revocation timestamp and test result

The five boundaries an AI key can cross

Start by mapping the places where a secret or the data it authorizes can move. Most failures occur because teams protect the provider key but overlook one of these adjacent boundaries.

Boundary	Typical failure	Required control
Client to application	A provider key is embedded in a browser bundle, mobile binary, desktop app, or extension	Keep provider keys server-side; issue short-lived user or session credentials to clients
Application to gateway	Every service shares one unrestricted key	Use workload identities, scoped gateway tokens, quotas, and explicit route policy
Gateway to provider	One credential can access every model, project, or environment	Separate keys by provider, environment, workload, and risk tier where supported
Request to logs	Authorization headers, prompts, files, or outputs appear in traces	Denylist secret fields, minimize payload logging, tokenize identifiers, and test redaction
Human operations	Keys are pasted into tickets, chat, runbooks, or support tools	Use controlled access workflows, audited retrieval, break-glass procedures, and automatic expiry

This boundary map changes the design question from “Where do we store the key?” to “Which identity can make which request, with which data, through which route, and what evidence remains afterward?”

Use a server-side key custody model

Long-lived provider credentials should not be delivered to browser, mobile, desktop, or extension clients. Anything shipped to an end-user device should be treated as recoverable by that user or by malware running with the same privileges.

A safer pattern is:

The client authenticates to your backend using a user session, device credential, or short-lived application token.
Your backend authorizes the requested feature and applies per-user or per-tenant limits.
A gateway or server-side integration chooses the approved provider and model.
The provider key is retrieved at runtime or made available to the trusted workload through the deployment platform.
The provider response returns through the same policy boundary.

The client never needs the provider secret. It only receives authority to call your product under the limits you define.

For coding agents and local developer tools, use the same principle with a deliberate exception model. A developer may need a local credential, but it should be a product or gateway credential with revocation, usage attribution, and bounded scope—not a shared organization-wide provider key copied into dotfiles across the company.

Build a key inventory before you rotate anything

Rotation projects fail when teams do not know which workload uses a key. Create a machine-readable inventory and assign an owner before changing credentials.

At minimum, record:

Field	Example	Why it matters
Secret ID	`prod-support-chat-anthropic-01`	A stable internal reference that is not the secret value
Provider and project	Provider account + project identifier	Defines the external blast radius
Environment	Development, staging, production	Prevents test systems from inheriting production authority
Workload	`support-chat-api`	Enables attribution and targeted revocation
Owner	Team and on-call rotation	Creates accountability during incidents
Storage location	Secret manager path or deployment binding	Shows where the source of truth lives
Allowed models/routes	Approved model family or gateway policy	Limits unexpected use
Spend and request limits	Workload budget, RPM, TPM, or concurrency controls	Constrains abuse and runaway automation
Created and last rotated	Timestamps	Makes stale credentials visible
Rotation method	Dual-key, version alias, or maintenance window	Prevents improvised changes
Revocation dependency	Services that must be updated first	Protects availability
Data classification	Public, internal, confidential, regulated	Connects key policy to prompt governance

Do not put the key value in the inventory. Store only metadata and a reference to the managed secret.

Separate identities by environment, workload, and risk

The most common anti-pattern is a single production key shared by every service. It is convenient until one repository, test runner, contractor laptop, or support transcript leaks it. Then the team cannot revoke the key without disrupting unrelated products.

Prefer separate identities for:

Production, staging, development, and local testing.
Customer-facing traffic, internal tools, batch jobs, and evaluation pipelines.
High-risk workflows that can call tools or process confidential data.
Different business units or tenants when contractual boundaries require separation.
Emergency or break-glass access, which should remain disabled or tightly controlled during normal operation.

Where a provider offers restrictions, apply them. Restrictions may include allowed APIs, models, source networks, projects, referrers, applications, or quotas. Google Cloud’s API key guidance, for example, recommends restricting keys, isolating them, deleting unneeded keys, avoiding repository commits, and monitoring their use.

If the provider does not offer sufficiently granular controls, enforce them at your own gateway. A central gateway can validate the calling workload, map it to an approved route, apply budgets and rate limits, and keep vendor credentials behind one reviewed server-side boundary. Review the broader AI API gateway architecture guide for routing and failover design.

Treat prompt and log governance as part of key management

An API key authorizes a data path. Protecting the key without controlling that path leaves the main AI-specific risk unresolved.

Before routing a request, classify the payload and apply a minimum-necessary rule:

Remove credentials, tokens, cookies, private keys, and connection strings.
Exclude fields the model does not need.
Tokenize or pseudonymize direct identifiers when the task can work without them.
Reject unsupported regulated or contractually restricted data.
Separate system instructions from untrusted user or retrieved content.
Validate tool arguments before an agent can call external systems.

Logs need an explicit schema. Do not rely on developers remembering not to log a request object. Define which fields are allowed, then discard or transform everything else.

A useful production event can contain:

{
  "request_id": "req_01J...",
  "tenant_id_hash": "tnt_7f2...",
  "workload": "support-chat-api",
  "route_policy": "support-low-risk-v3",
  "provider": "selected-provider",
  "model": "selected-model",
  "input_tokens": 842,
  "output_tokens": 211,
  "latency_ms": 1370,
  "status": 200,
  "key_version": "v12",
  "redaction_policy": "customer-support-v4"
}

It should not contain the authorization header, raw provider key, complete prompt, uploaded document, unredacted tool argument, or full model output by default. If payload capture is necessary for a tightly controlled evaluation or incident, make it time-bounded, access-controlled, and visibly separate from routine telemetry.

The OWASP Logging Cheat Sheet provides a useful baseline for excluding access tokens, passwords, encryption keys, and other sensitive data from application logs.

Design zero-downtime API key rotation

Rotation is only a control if it can be performed safely. A runbook that causes an outage will be deferred until an emergency.

Use a dual-key or versioned-secret sequence where the provider supports overlapping credentials:

Create a new provider key. Apply the same or narrower restrictions as the old key.
Store it as a new secret version. Do not overwrite the old value in place if your platform supports versions or aliases.
Deploy readers that accept the new version. Update the gateway or workloads to resolve the current alias or version.
Shift traffic and observe. Confirm successful requests, expected models, spend, rate limits, and error rates using the new key version.
Remove old consumers. Check deployment inventories, jobs, workers, and disaster-recovery environments.
Revoke the old key. Do not merely stop using it; invalidate it at the provider.
Verify rejection. A controlled test should confirm that the old credential no longer works.
Record evidence. Save timestamps, owners, affected workloads, validation results, and the next review date.

For a system that supports key version selection, the application should refer to a stable alias rather than hard-coding a secret version:

type ProviderCredential = {
  value: string;
  version: string;
};

async function loadProviderCredential(): Promise<ProviderCredential> {
  const activeVersion = await secretStore.resolveAlias("ai/provider/active");
  const value = await secretStore.readVersion("ai/provider", activeVersion);

  return { value, version: activeVersion };
}

Do not print value, serialize the returned object, or attach it to an error. Log only the non-secret version identifier.

The OWASP Secrets Management Cheat Sheet recommends planning the full secret lifecycle, including creation, rotation, revocation, expiration, auditing, backup, and break-glass access. It also emphasizes automating rotation where practical.

Prevent secrets from entering repositories and CI logs

Secret managers do not help after a credential is copied into source code, a fixture, a build artifact, or a CI transcript.

Use controls at three stages:

Before commit

Provide .env.example files with placeholders, never working credentials.
Keep local secret files outside version control.
Run a fast secret scanner in pre-commit hooks for common provider patterns and high-entropy values.
Teach developers that deleting a secret in a later commit does not remove it from history.

At push and pull request

Enable repository secret scanning and push protection where available.
Add custom patterns for internal gateway tokens that public scanners will not recognize.
Require a documented bypass reason and route bypasses to security review.
Scan generated files, notebooks, test snapshots, and infrastructure plans—not only application source.

GitHub documents push protection as a way to scan during the push process and block detected secrets before they enter a repository. Detection is not a reason to keep the key: treat any confirmed committed credential as exposed and rotate it.

During CI/CD

Prefer workload identity or short-lived federation over stored cloud credentials.
Expose a secret only to the job and step that need it.
Mask known secret values, but do not depend on masking as the primary control.
Disable shell tracing around secret retrieval.
Prevent untrusted forked code from accessing deployment secrets.
Review artifacts, caches, crash dumps, and test reports for accidental capture.

Monitor use without logging the secret

Good monitoring answers “who used which authority?” without recording the authority itself.

Track:

Workload, environment, tenant or project, and key version.
Provider, model, route policy, and fallback path.
Request counts, token volume, spend, latency, and error class.
Source network or deployment identity where useful.
Access to the secret manager, including denied reads.
Key creation, restriction changes, rotation, revocation, and deletion.
Sudden use from an unexpected environment, geography, model, or time window.

Set alerts around behavior, not just total spend. A stolen low-budget key can still expose prompts or probe internal workflows. Conversely, a legitimate batch job can create a spend spike without credential compromise. Correlate provider usage with application request IDs, gateway policy decisions, deployment events, and secret-manager audit logs.

For traffic controls that complement credential controls, use the LLM rate limits guide and the LLM API fallback routing playbook.

Use an exposure-to-revocation incident clock

When a key may be exposed, the first goal is containment—not proving whether an attacker used it.

Run this sequence:

Declare the credential suspect. Record when and where it may have been exposed.
Create a replacement through the normal controlled path. Do not paste a new key into chat to speed up the incident.
Shift legitimate workloads to the replacement. Use the prepared rotation procedure.
Revoke the suspect key. If immediate revocation would create unacceptable harm, isolate routes and reduce limits while completing the cutover.
Search every copy location. Check source history, CI logs, artifacts, container layers, support systems, dashboards, notebooks, local configuration, and backups.
Review authorized data paths. Determine which prompts, outputs, files, tools, or models the key could access—not just its billing scope.
Correlate activity. Compare provider usage, gateway logs, deployments, secret reads, and user activity.
Notify the right owners. Involve security, platform, legal, privacy, and customer teams according to the data and contracts involved.
Remove the root cause. Add the missing scanner, restriction, identity boundary, or log filter.
Measure the clock. Record time to detection, replacement, traffic cutover, revocation, and verification.

The most actionable metric is often exposure-to-revocation time: how long a suspected credential remains usable after the organization has credible evidence of exposure. Reducing that interval requires prepared ownership, inventory, automation, and tested rotation—not a longer policy document.

If you issue API keys to your own customers

Many AI products also hand out keys, for example to let customers call your agent or embed your feature. The same best practices apply in reverse:

Generate keys with a cryptographically secure random generator, at least 32 bytes of entropy.
Add a recognizable prefix (like sk-fk- on Flatkey keys) so secret scanners and your support team can identify leaked keys. GitHub's partner program uses known token formats to notify issuers when a key is pushed to a public repository (GitHub secret scanning).
Show the full key once, then store only a hash. If your database leaks, the keys stay safe.
Display only a masked prefix in your dashboard, plus the creation date and last-used time.
Make revocation immediate and return 401 for revoked keys.
Let customers scope keys to specific models, projects, or spend limits.

Python: generate, hash, and verify keys

import hashlib
import hmac
import secrets

PREFIX = "sk-acme-"

def new_api_key() -> tuple[str, str]:
    """Return (plaintext key shown to the user once, hash to store)."""
    key = PREFIX + secrets.token_urlsafe(32)  # 32 random bytes
    return key, hashlib.sha256(key.encode()).hexdigest()

def verify(presented: str, stored_hash: str) -> bool:
    candidate = hashlib.sha256(presented.encode()).hexdigest()
    return hmac.compare_digest(candidate, stored_hash)  # constant-time compare

key, digest = new_api_key()
print(key[:12] + "...")   # show the full key once in the UI, never log it
print(verify(key, digest))  # True

A fast hash such as SHA-256 is acceptable here because the key is long and random, unlike a human password. Store the hash, a key ID, the owner, the scope, and timestamps.

Node.js: load a key from the environment and fail closed

// Never fall back to a hardcoded default key.
const apiKey = process.env.FLATKEY_API_KEY;
if (!apiKey || !apiKey.startsWith("sk-fk-")) {
  throw new Error("FLATKEY_API_KEY is missing or malformed");
}

const res = await fetch("https://router.flatkey.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${apiKey}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "claude-haiku-5-5",
    messages: [{ role: "user", content: "ping" }],
    max_tokens: 5,
  }),
});
console.log(res.status); // log the status, never the header

Verify that a revoked key is really dead

Revocation is only proven when the old key fails. After each rotation or incident, run a check like this from a trusted machine:

# Expect 401 for the revoked key; do not echo the key itself
OLD_KEY="$(cat /run/secrets/old_flatkey_key)"
status=$(curl -s -o /dev/null -w "%{http_code}" \
  https://router.flatkey.ai/v1/models \
  -H "Authorization: Bearer $OLD_KEY")
if [ "$status" = "401" ]; then echo "revoked OK"; else echo "STILL ACTIVE: $status"; fi

Record the result and time in your rotation log. Flatkey documents that revocation is immediate and that any request with a revoked key receives 401 Unauthorized, with no grace period (Flatkey API keys docs).

How Flatkey supports these best practices

Flatkey gives you one OpenAI-compatible endpoint (https://router.flatkey.ai/v1) for many model providers, which means fewer provider keys spread across your services. According to the Flatkey API keys documentation (checked October 27, 2026):

You can create multiple keys: one per environment, application, or team member, with descriptive names like prod-customer-chatbot.
The full sk-fk- key is shown only once. The Keys page then lists the name, creation date, last-used timestamp, and a masked prefix.
Revocation is immediate, so deploy the replacement key before revoking the old one.
All keys share the account balance and can access all models by default. Use Groups in the console to restrict a key to specific models or set a spend quota.

The usage docs describe per-request logs with model, tokens, cost, and key, plus a monthly spend cap and low-balance email alerts, which cover practices 10 and 11. For larger organizations, the Flatkey enterprise docs describes tree-structured sub-keys with their own budgets and model allowlists. A gateway does not replace your own client authentication, data classification, and log hygiene, so keep the rest of this checklist in place. To get started, see the Flatkey integration starter.

Reference architecture for multi-provider AI products

A secure multi-provider setup can be organized into five layers:

Client identity layer: authenticates the user, device, agent, or application without exposing provider credentials.
Application authorization layer: verifies product entitlements, tenant boundaries, feature access, and budgets.
Data policy layer: classifies and redacts prompts, files, retrieved context, and tool arguments.
Gateway and routing layer: selects approved models, applies quotas, records non-secret attribution, and handles failover.
Provider credential layer: stores isolated vendor credentials, rotates versions, and exposes them only to the trusted routing workload.

This design limits blast radius. A compromised client token does not automatically reveal a provider key. A compromised workload identity should not grant access to every provider. A leaked provider credential should not authorize every environment. A logging failure should not expose both the secret and the full payload.

Production checklist

Use this checklist before launch and during quarterly control reviews.

Custody and identity

[ ] No long-lived provider key is shipped in browser, mobile, desktop, or extension code.
[ ] Every production workload has an identifiable owner and credential path.
[ ] Production, staging, development, evaluation, and local access are separated.
[ ] Shared human keys have been replaced with workload or gateway identities where possible.
[ ] Provider and gateway restrictions are set to the narrowest practical scope.

Storage and delivery

[ ] The source of truth is a managed secret store or controlled deployment binding.
[ ] Applications retrieve secrets only at runtime and do not print or serialize them.
[ ] CI jobs receive only the secrets required for the required step.
[ ] Secret access and administrative changes are audited.
[ ] Break-glass access is documented, time-bounded, and tested.

Data and observability

[ ] Authorization headers and key values are excluded from logs, traces, errors, and support exports.
[ ] Prompt, output, file, and tool-argument logging follows an allowlist schema.
[ ] Redaction happens before multi-provider routing.
[ ] Usage can be attributed to workload, environment, route, provider, model, and key version.
[ ] Alerts cover unusual routes and identities as well as spend.

Rotation and response

[ ] A tested dual-key or versioned-secret rotation runbook exists.
[ ] Old keys are revoked at the provider and verified unusable.
[ ] Secret scanning and push protection cover repositories and generated artifacts.
[ ] A suspected leak triggers immediate containment without waiting for proof of abuse.
[ ] Exposure-to-revocation time is measured after drills and incidents.

FAQ

What are the best practices for API key management?

Keep keys out of source code and client apps, use a separate key per environment and workload, restrict each key to the APIs, models, and budget it needs, store keys in a secrets manager, rotate them on a tested schedule, delete unused keys, enable secret scanning, never log keys, and revoke immediately when a leak is suspected.

Should an API key be stored in an environment variable?

Environment variables are a reasonable way to deliver a key to a process, but they are not a management system. The value should come from a secrets manager or your platform's secret injection, with access control, rotation, and auditing. Never commit .env files.

How often should API keys be rotated?

Pick an interval based on risk and your provider's controls; many teams rotate production AI keys every 90 days and rotate immediately after staff changes or suspected exposure. What matters most is that rotation is rehearsed, the old key is revoked, and revocation is verified.

Can a browser or mobile app call an AI provider directly with an API key?

Avoid it. Anything shipped to a device can be extracted. Have the client authenticate to your backend, and let the backend or gateway call the model with a server-side key and per-user limits.

What should I do if an API key is leaked on GitHub?

Treat it as compromised. Create a replacement, move workloads to it, revoke the leaked key, confirm it returns 401, review usage for the exposure window, and add scanning or push protection so it does not happen again. Deleting the commit does not make the key safe.

Is one gateway key safer than many provider keys?

It reduces the number of secrets in your code, but it concentrates authority. Use separate gateway keys per environment and workload, scope them with model and spend limits, and protect the provider keys behind the gateway as carefully as before.