What is an AI chatbot API?
An AI chatbot API is a programmatic interface that lets an application send a user message and receive a response for a conversational experience. The phrase can mean two different products. A model API generates the next answer from a prompt and conversation context. A chatbot platform API can also provide conversation storage, channels, human handoff, workflow rules, and analytics. Ask which meaning a provider uses before comparing prices or copying an integration example.
For a simple support assistant, the request path is usually
chat UI → your server → model or chatbot API → your server → chat UI.
Your server should own the key, validate the request, attach only the
context the bot is allowed to use, and decide what to do with the
response. The API call is one part of the product; it does not
automatically supply trustworthy knowledge, access control, or a safe
action workflow.
This guide helps a developer choose the right API layer, make a first call, estimate cost, and test a chatbot before real users rely on it. It also explains where a multi-model router such as Flatkey fits. Flatkey provides a model-facing API surface; it is not a ready-made customer-support inbox or a replacement for a conversation database.
Choose the API layer that matches the job
| What you need | Start with | You still need to own or verify |
|---|---|---|
| A custom chat interface in your app | A model API or compatible router | UI, user identity, history, retrieval, safety, evaluation |
| A support bot across website and messaging channels | A chatbot platform with channel and handoff APIs | Integrations, escalation policy, data permissions, reporting |
| Answers grounded in internal documents | A model API plus retrieval or a platform with that feature | Index quality, source citations, document permissions, freshness |
| Tool calls that change records or send messages | A model API with tool-calling support and your server-side executor | Authorization, confirmation, audit log, rollback |
| Several models or providers behind one client | A compatible router | Model selection, quality tests, usage limits, provider-specific differences |
The first decision is operational, not cosmetic. A purpose-built chatbot platform may save work on channels and handoff, but can constrain the application workflow. A direct model API gives more control over the experience but asks your team to build more of the stack. A router can simplify model switching while leaving chatbot product responsibilities with you.
How a chatbot request becomes a reply
The browser sends a message to your application server with a conversation identifier. The server verifies the user, loads the permitted conversation context, and creates a request for the chosen model. The model returns text, structured output, or a stream of chunks. The server validates the result, records the turn according to your retention policy, and sends a safe response to the browser.
Conversation state deserves an explicit design. Some APIs take the full message history on every call; others offer provider-managed state. The OpenAI Responses documentation and Gemini multi-turn documentation describe different ways to carry context. Do not assume a provider stores, forgets, or isolates turns in the same way as another. Test how the chosen endpoint handles history, truncation, deletion, and data retention.
Keep a stable internal conversation ID even when the provider offers a remote thread ID. That gives your app a place to enforce permissions and to correlate retries, messages, costs, and human handoff. Store only the history needed for the task. A long transcript increases latency and billable input, and can make the bot repeat outdated instructions.
A first AI chatbot API integration
Start with one task, such as answering questions about a small, public help center. Define the allowed topics, a response style, a fallback answer, and when to show a human contact option. Then build this minimal server-side path:
- Create a scoped API key and store it in a server-side secret store or protected environment variable. Never ship it in browser JavaScript, a mobile bundle, or a public repository.
- Choose a documented chat-capable model and endpoint. Check the current model catalog for context limits, tool support, streaming, and pricing.
- Accept the user’s message at your own authenticated endpoint. Limit length and rate, validate the conversation ID, and reject messages from users who cannot access that conversation.
- Send a small system instruction plus recent relevant context to the API. If you use documents, retrieve a few permitted passages rather than putting the entire knowledge base into every prompt.
- Handle success, timeouts, rate limits, and provider errors separately. Return an app-safe message and keep the request ID for diagnosis.
- Validate the response before showing it or executing any tool. Save the turn and emit latency, error, token or unit usage, and model ID to your own telemetry.
The Flatkey API documentation shows its OpenAI-compatible router and current model catalog. The Flatkey quickstart covers the first request through its router. Use the live docs for the exact model ID and endpoint shape; a code sample copied from a blog may outlive a model. If you need a chatbot platform’s channel or handoff features, verify that platform’s API separately instead of assuming a model router supplies them.
A small request contract
Your own endpoint might accept conversation_id and
message, then return reply,
request_id, and a status. Do not let the client choose an
arbitrary provider URL or pass an API key. The server selects a
supported model and attaches the policy and context it controls.
For streaming, forward only validated output chunks to the user and handle disconnects. A browser closing its connection does not necessarily cancel the provider call. Define whether to save a partial reply, retry, or mark the turn incomplete. For structured tool calls, treat arguments as untrusted input and let your application decide whether an action is allowed.
Retrieval, tools, and human handoff
A chatbot that answers company-specific questions needs a source of truth. Retrieval can select passages from a help center, policy library, or product catalog and pass them to the model. The hard part is not vector search alone: the retrieved passages must be current, relevant, and visible to that user. Test with questions that have no answer in the source; the bot should say so or escalate, not invent a policy.
Tool use raises the stakes. A model may propose a ticket update, refund, booking, or outbound message. Your server must check the actor’s permissions, validate the tool arguments, apply business rules, and record what happened. For irreversible actions, use a confirmation step or human review. A conversational interface is not a security boundary.
Define handoff before launch. State which questions a bot should decline, what context a human agent receives, and how a user knows the handoff occurred. A chatbot platform may provide this workflow; if you use only a model API, you will need to build it. Measure how often handoff happens and whether the conversation context helped the human resolve the issue.
What an AI chatbot API actually costs
For a model API, monthly model cost is roughly
accepted conversations × average turns × billable input and output units per turn × current unit price.
This is a starting estimate, not a quote. The real bill may also include
cached input, retrieval, search or other tools, retries, safety checks,
image or audio input, and longer conversation histories. A chatbot
platform can add seats, channels, workflow runs, or conversation
charges.
Make a small workload table from your own test data. Record messages per conversation, context length by turn, average and long-tail output size, and the fraction of replies that users accept. Price the resolved conversation, not only a single model response. A cheap call that produces many retries or handoffs may cost more overall.
Rate limits are a separate constraint. Estimate peak concurrent users and turns per minute, then test what happens when the provider returns a limit. Queueing a short retry can be reasonable; an unbounded retry storm can multiply cost and make an outage worse. If you add a second model as a fallback, compare answer quality, tool behavior, and response format before routing real users to it.
Flatkey’s pricing page is the source for current model rates through its router. Verify the exact model and request units there before using a router in a cost estimate. Do the same for any direct provider or chatbot platform; published rates and free allowances change.
Security and privacy checks before launch
Protect the key on your server and scope it where the provider supports that. Restrict who can read a conversation or a retrieved document. Do not place private data in traces merely because the prompt was convenient to log. Decide how long conversations, requests, and model outputs are retained, and compare that policy with the provider’s current processing terms.
Treat a retrieved webpage or user message as data, not as an instruction with authority over your system. A malicious page can include prompt-injection text asking the bot to reveal secrets or ignore its task. Keep system policy outside retrieved content, restrict tools to the minimum needed, and validate every proposed action. The OWASP Top 10 for LLM Applications offers a practical threat checklist for this review.
Test unsafe and ambiguous cases with real examples: a request for another customer’s information, a request to change an order, conflicting retrieved passages, and an unavailable source. Include multilingual questions if the bot will serve multiple languages. Make sure escalation and error messages are useful without exposing credentials or raw provider responses.
How to compare candidates in one afternoon
Use the same 30–50 representative conversations against each candidate. Include simple FAQs, multi-turn follow-ups, questions that require retrieval, questions outside scope, and a few adversarial inputs. Score grounded accuracy, refusal when evidence is missing, response latency, handoff quality, cost per accepted result, and ease of diagnosing an error. Keep prompts and expected outcomes in a small evaluation set so you can repeat it when a model changes.
If you need one model and an existing direct-provider account meets your requirements, begin there. If you need a consistent client shape across multiple supported models, evaluate a router with the same conversation set. If channels, ticketing, or human handoff are the dominant requirement, evaluate chatbot platforms first. The best AI chatbot API is the one whose responsibilities match the product you are actually building.
Frequently asked questions
Is a chatbot API the same as an LLM API?
Sometimes people use the terms interchangeably. An LLM API generates or processes model output. A chatbot API may include that model call plus conversation storage, channels, workflow, and handoff. Inspect the product’s documented endpoints and responsibilities.
Can I put the API key in my website?
Do not expose a provider or router key in public client code. Send messages to your own server, which authenticates the user and calls the API with a protected key.
Do I need a vector database for a chatbot?
No. A small bot for public, static information can begin with curated context or deterministic answers. Add retrieval when the source set is large or changes often, and test permissions and freshness before relying on it.
Can a model API automatically hand a user to a person?
A model can suggest escalation, but your application or chatbot platform must implement the handoff, route the conversation, and show the user what happened.
What should I measure after launch?
Track successful resolutions, handoffs, grounded-answer accuracy, unsafe or unsupported answers, latency, error rate, model and tool usage, and cost per accepted conversation. Review the data by use case rather than treating total message count as success.
