Searching for the Open AI embeddings API? OpenAI's embeddings endpoint converts text into numeric vectors so an application can retrieve related passages. The request is only the first step: a useful search feature also needs stable document IDs, a consistent vector space and a test that tells you whether retrieval improved.
This guide builds a small Python search example, shows what to store alongside each vector, and gives you a checklist for taking the experiment into a product. It uses OpenAI directly. Gateway compatibility and model availability must be verified separately for each provider.
Open AI embeddings API: the request you need
Send text to POST https://api.openai.com/v1/embeddings with an embedding model and an API key. Choose encoding_format="float" when your application expects numeric arrays. For a batch, use each response item's index to reconnect the vector to its input. The response also includes token usage. The dimensions option is supported by the third-generation embedding models. OpenAI API reference.
An embedding is not a generated answer. In a retrieval-augmented application, search first selects source passages; a separate generation call can then answer using those passages. Keeping the stages separate lets you diagnose a bad result: did search retrieve the wrong evidence, or did the answering model misuse good evidence?
Pick a model and keep its configuration fixed
The documented default vector lengths are 1,536 for text-embedding-3-small and 3,072 for text-embedding-3-large. These models support shorter output vectors through dimensions. A shorter vector changes storage and retrieval trade-offs; it is not a reason to skip evaluation. OpenAI embeddings guide.
Start with one configuration and a small set of representative queries. Record the model, requested dimension count and text-processing version with the index. Two vectors having the same length does not establish that they came from a compatible model. When changing models or processing rules, build a separate index and compare it before switching production traffic.
A minimal Python semantic-search example
Install openai in your Python environment and make OPENAI_API_KEY available through your normal secret manager or environment configuration. Keep the key out of source control and browser code. The official Python SDK reads that variable when constructing OpenAI().
Save the following as search_demo.py. It uses three invented support documents, embeds the corpus and a query, and ranks the documents with cosine similarity. Running it makes billable API requests. The output depends on the service response; the example does not promise a particular score or ranking.
import math
from openai import OpenAI
client = OpenAI(timeout=30.0, max_retries=2)
model = "text-embedding-3-small"
dimensions = 1536
documents = [
{"id": "key-rotation", "text": "Replace an exposed API key and revoke the old credential."},
{"id": "invoice-export", "text": "Download a monthly invoice from the billing workspace."},
{"id": "request-timeout", "text": "Record the request ID and retry temporary upstream timeouts."},
]
query = "What should I do after accidentally sharing a secret key?"
def embed(texts):
response = client.embeddings.create(
model=model, input=texts,
dimensions=dimensions, encoding_format="float",
)
ordered = sorted(response.data, key=lambda item: item.index)
if [item.index for item in ordered] != list(range(len(texts))):
raise ValueError("Unexpected embedding response indices")
vectors = [item.embedding for item in ordered]
if any(len(vector) != dimensions for vector in vectors):
raise ValueError("Unexpected vector dimensions")
print("Input tokens:", response.usage.prompt_tokens)
return vectors
def cosine(a, b):
if len(a) != len(b):
raise ValueError("Vector dimensions do not match")
denominator = math.sqrt(sum(x*x for x in a) * sum(y*y for y in b))
if not denominator:
raise ValueError("Cannot compare a zero vector")
return sum(x*y for x, y in zip(a, b)) / denominator
vectors = embed([document["text"] for document in documents])
query_vector = embed([query])[0]
ranked = sorted(
((cosine(query_vector, vector), document["id"])
for document, vector in zip(documents, vectors)),
reverse=True,
)
for score, document_id in ranked:
print(document_id, round(score, 4))
Run python search_demo.py. Confirm that all three IDs appear once and that each embedding has the configured length. Then change the query to ask about invoices. This is a smoke test of the data flow, not proof that the system is ready for real support traffic. The example embeds the corpus again on each execution for clarity; a deployed service should normally persist document vectors and embed only new or changed content.
Turn the demo into a reliable document index
The following record layout is a practical starting point. It is an application design example, not a field schema required by OpenAI.
| Store with each chunk | Why it matters |
|---|---|
document_id and chunk_id | Update or delete the right content without relying on array order. |
| Source URL, section title and source revision | Display a useful citation and trace stale answers. |
| Content hash and processing version | Identify which chunks actually need to be embedded again. |
| Model and dimension count | Prevent an accidental mix of incompatible vector configurations. |
| Tenant and access-control metadata | Filter retrieval to documents the requesting user may access. |
| Text and embedding | Return the source passage after selecting its vector. |
Split documents at meaningful boundaries, then check token length. A section containing the question's answer should remain understandable when retrieved on its own. Evaluate overlap deliberately: excessive overlap can fill the result set with repeated versions of the same passage.
For the first experiment, the in-memory loop above is enough. As the corpus grows, use an index or vector store that supports your required filtering, deletion and retrieval behavior. Test those operations before relying on similarity scores. A highly similar passage from the wrong customer must never be returned merely because it ranks first.
Check input and batch limits separately
OpenAI's current endpoint reference specifies an 8,192-token maximum per input and a 300,000-token aggregate maximum across inputs in one request; it also documents a maximum of 2,048 batch inputs. Empty text is invalid. Count tokens rather than characters, and leave room below the limits when constructing batches. Endpoint constraints.
If you use Azure OpenAI, endpoint configuration, authentication and deployment naming have their own setup. Follow the Azure embeddings instructions for that service instead of assuming a direct OpenAI request can be copied unchanged. Apply the same separation when using a gateway: check its endpoint, model access, supported parameters and billing account first.
Estimate cost without hiding index rebuilds
Embedding cost follows input tokens, so count document ingestion, changed chunks and query embeddings separately. Use the current price from the relevant model page, such as text-embedding-3-small, and confirm whether the quoted rate applies to your request mode.
For a planning example, let P be your verified price per million input tokens:
Initial ingestion: 50,000 chunks × 400 tokens = 20 million tokens
Monthly changes: 5,000 chunks × 400 tokens = 2 million tokens
Monthly queries: 200,000 queries × 20 tokens = 4 million tokens
Initial embedding cost = 20 × P
Recurring embedding cost = (2 + 4) × P = 6 × P
Those are illustrative workload assumptions, not Flatkey or OpenAI prices. Add vector storage, search infrastructure and any answer-generation calls separately. If you change the embedding model, include a full re-embedding pass in the migration budget. A pipeline that re-embeds unchanged documents can spend more than the same workload with a larger but correctly cached model.
Diagnose errors before adding retries
| Symptom | First useful check |
|---|---|
| Authentication error | Confirm the key is valid for the endpoint and account you are calling. |
| Invalid request | Inspect the error body, empty strings, token counts and supported parameters. |
| Rate limit or quota error | Distinguish request-rate pressure from exhausted account quota. |
| Temporary server error or timeout | Preserve request identifiers and use bounded retry behavior. |
| Vector-store dimension error | Compare the collection configuration with the actual returned vector length. |
| Plausible but irrelevant results | Check chunk text, model consistency and query examples before blaming the network. |
The first four checks follow the OpenAI error guide. The last two concern your application and index. Respect server retry guidance and account for SDK retries before adding another retry loop. Log operational metadata without recording credentials or unnecessary private document text.
Evaluate retrieval before choosing an answering model
Create a small labeled set from real user questions, including paraphrases, exact product names, ambiguous questions and questions your corpus cannot answer. Write down the expected source document for each answerable query before tuning the system.
For each candidate configuration, record whether an acceptable source appears in the first few results, how much duplicate material is returned, retrieval latency and cost per query. Review failures individually. If a query asks for an exact error code, a keyword match may complement semantic similarity. If the source is missing, changing embedding dimensions cannot invent it.
Keep a held-out portion of the questions for the final comparison. Do not pick a similarity threshold from the three-document demo and treat it as a universal confidence score. A useful acceptance test reflects the consequences of wrong retrieval in your own application.
Where Flatkey fits in the application
Flatkey provides an OpenAI-compatible model router and a current model directory. Use the Flatkey integration documentation to evaluate the model-generation side of your application. The examples above call OpenAI directly; they do not establish that an embedding model or every embeddings parameter is available through Flatkey.
For a multi-provider application, keep retrieval configuration independent from your answering model configuration. That lets you compare answer models without rebuilding the document index for every experiment. Review Flatkey's API architecture guides when planning routing, account controls and operational checks.
Frequently asked questions
Does the embeddings API answer questions by itself?
It returns vectors. Your application uses them to select related content; an answering model is a separate component if you need a written response.
Can I compare vectors from different embedding models?
Do not assume they share a meaningful coordinate space, even if their dimensions match. Keep model-specific indexes and evaluate migrations using the same questions and source documents.
Should I choose the largest embedding model immediately?
Start from a configuration you can measure. Compare retrieval quality, latency, storage and cost on your own queries before accepting a larger index or a different model.
What should I build first?
Use the Open AI embeddings API to embed a small, permission-safe corpus, verify document-to-vector mapping, and test retrieval with labeled questions. Expand the corpus after that loop produces reliable evidence.
