OpenAI-Compatible APIs: Meaning, Providers and Examples

An OpenAI-compatible API accepts OpenAI-style requests and returns responses that OpenAI clients can parse, most often for Chat Completions. Reuse the OpenAI SDK by changing the base URL, API key and model ID. Tools, JSON schemas, streaming metadata and Responses support vary by endpoint and model. Compatibility describes the interface, as the Modular handbook explains.

The useful question is: “Does this endpoint support the requests my application sends?”

What OpenAI compatibility means

For a basic chat request, your client sends a model identifier and a messages array to a Chat Completions route. It expects an assistant reply under choices[].message. Streaming returns Server-Sent Events (SSE) containing choices[].delta. These are parts of the OpenAI Chat Completions contract.

A compatible provider implements that contract for the features it supports. Embeddings, model discovery, images, audio and Responses are separate API capabilities.

Base URL means the prefix, not the full operation URL. With Requesty, set base_url="https://router.requesty.ai/v1"; the SDK appends /chat/completions. Gemini's prefix ends in /v1beta/openai/. DeepSeek uses https://api.deepseek.com. Follow the Requesty quickstart, Gemini setup or DeepSeek setup rather than adding /v1 everywhere.

The destination determines credentials, billing and data policies. Compatibility does not transfer an OpenAI API balance or ChatGPT subscription to another provider. A shared payload format does not imply the same answer quality. Even OpenAI documents different parameter support between its own models.

Which providers support OpenAI-compatible APIs?

Search the matrix by provider, filter by category or require the features your application uses. Expand a row for its base URL, restrictions and sources. Tools means developer-defined function calling; provider-hosted search and code execution need separate checks.

Perplexity Agent's custom functions, schema output and image inputs use its Agent surface. Its Sonar migration notice explains the September 27 transition. Keep that surface separate from Router when configuring a client.

Use an OpenAI-compatible endpoint

These examples use Requesty and openai/gpt-6-luna. Replace the URL, key and model to use another provider. Check current identifiers in the Models API reference.

Export these values; the examples read the LLM_* variables explicitly:

export LLM_BASE_URL="https://router.requesty.ai/v1"
export LLM_API_KEY="your-requesty-key"
export LLM_MODEL="openai/gpt-6-luna"

python -m pip install openai

import os
from openai import OpenAI
 
client = OpenAI(
    base_url=os.environ["LLM_BASE_URL"],
    api_key=os.environ["LLM_API_KEY"],
)
response = client.chat.completions.create(
    model=os.environ["LLM_MODEL"],
    messages=[{"role": "user", "content": "Explain an API in one sentence."}],
)
print(response.choices[0].message.content)

npm install openai

import OpenAI from 'openai';
 
const baseURL = process.env.LLM_BASE_URL;
const apiKey = process.env.LLM_API_KEY;
const model = process.env.LLM_MODEL;
if (!baseURL || !apiKey || !model) {
  throw new Error('Set LLM_BASE_URL, LLM_API_KEY, and LLM_MODEL');
}
 
const client = new OpenAI({ baseURL, apiKey });
const response = await client.chat.completions.create({
  model,
  messages: [{ role: 'user', content: 'Explain an API in one sentence.' }],
});
console.log(response.choices[0].message.content);

curl https://router.requesty.ai/v1/chat/completions \
  -H "Authorization: Bearer $LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-6-luna",
    "messages": [{"role": "user", "content": "Explain an API in one sentence."}]
  }'

Run the TypeScript files with npx tsx; they use top-level await, so the project needs "type": "module" in package.json.

Use @ai-sdk/openai-compatible

Vercel's OpenAI Compatible provider wraps an OpenAI-shaped API for the AI SDK. Supply name, baseURL and credentials. Save this as ai-sdk-example.ts and run npx tsx ./ai-sdk-example.ts:

npm install ai @ai-sdk/openai-compatible

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { generateText } from 'ai';
 
const baseURL = process.env.LLM_BASE_URL;
const apiKey = process.env.LLM_API_KEY;
const model = process.env.LLM_MODEL;
if (!baseURL || !apiKey || !model) {
  throw new Error('Set LLM_BASE_URL, LLM_API_KEY, and LLM_MODEL');
}
 
const provider = createOpenAICompatible({ name: 'custom', baseURL, apiKey });
const { text } = await generateText({
  model: provider(model),
  prompt: 'Explain an API in one sentence.',
});
console.log(text);

The provider interface requires name and baseURL. Choosing the adapter also chooses an API surface:

See the generic adapter source, OpenAI provider guide and Open Responses provider guide. The generic adapter has no .responses() language-model factory.

Set supportsStructuredOutputs: true for a schema-capable model. It tells the adapter what to request; server support must already exist. For streaming, includeUsage: true requests stream_options.include_usage. Both settings are covered in Vercel's provider reference.

Where compatibility breaks

Tools: test the return trip

Your application validates the function's JSON arguments, executes an authorized operation and returns the result with the matching tool-call ID. The provider then needs to accept that conversation on the next request. OpenAI documents the tool-message shape and streamed argument fields.

Test auto, required, named selection, strict arguments and parallel calls. Groq separates parallel support by model. DeepSeek rejects forced choices in thinking mode. Anthropic's direct layer ignores strictness.

For self-hosting, vLLM automatic selection needs a matching parser/template; named and required selection use structured decoding. LM Studio describes malformed calls appearing in message content. Rich tool results need another check: Vercel's compatible-message converter stringifies content-style tool results, even when user image inputs are supported.

JSON mode, JSON Schema and strict tools are different

These distinctions follow the OpenAI Chat reference and Azure structured-output guide.

HTTP 200 does not establish enforcement. Anthropic ignores response_format. DeepSeek's Chat enum permits only text and json_object; its strict tool schemas use a beta base.

Also test combinations and keywords. Cerebras disallows JSON object mode with streaming. Fireworks documents schema restrictions. Validate output locally and handle refusal or truncation before using it. Our structured-output testing article covers the gap between valid shape and correct content.

Streaming: guard empty choices and missing usage

OpenAI permits a final usage chunk with an empty choices array. Interrupted streams might never deliver final usage. Both behaviors appear in the streaming reference.

Reuse the Python client above for this text-only loop, following our streaming guide:

stream = client.chat.completions.create(
    model=os.environ["LLM_MODEL"],
    messages=[{"role": "user", "content": "Write two sentences about HTTP."}],
    stream=True,
    stream_options={"include_usage": True},
)
usage = None
for chunk in stream:
    if chunk.choices:
        text = chunk.choices[0].delta.content
        if text:
            print(text, end="", flush=True)
    if chunk.usage is not None:
        usage = chunk.usage
print()
print("Usage:", usage)

Keep missing usage separate from zero tokens. DeepSeek and Fireworks report usage on a final finish chunk. Cached-token and reasoning fields also differ, as Together documents.

For streamed tools, accumulate argument fragments by index and ID before parsing. The text loop above does not collect tool arguments.

Vision and reasoning: test the exact format

Cerebras accepts base64 PNG/JPEG on selected models, but rejects external HTTPS image URLs. Ollama's local checklist checks base64 support and leaves remote URLs unchecked. Test your intended transport, MIME type and content shape.

Reasoning settings also differ. Gemini prohibits combining reasoning_effort with its thinking controls. Anthropic ignores reasoning_effort in its direct layer. Preserve provider-required reasoning or signature fields when replaying tool conversations.

Choose a native SDK when you need native content types, caching, reasoning or hosted tools outside the compatible surface.

Chat Completions does not imply Responses support

Responses changes the request and response contract. OpenAI recommends trying Responses for new projects; Chat Completions remains supported in its Node SDK.

See the OpenAI Chat contract, Requesty Responses reference and Azure schema examples.

With the Python client above, a basic Requesty Responses call is:

response = client.responses.create(
    model=os.environ["LLM_MODEL"],
    input="Explain an API in one sentence.",
)
print(response.output_text)

Stored continuation needs its own test. DeepSeek and Ollama implement stateless Responses. Fireworks supports previous_response_id. Check storage semantics and hosted tools on the selected route as well.

Open Responses defines an open specification based on Responses and provides compliance tests. A generic compatibility label does not establish compliance with that specification.

Test compatibility before switching providers

Start with the requests your application sends. A drafting tool needs text. An agent needs tool replay. A document extractor needs validated output and image handling. Their acceptance tests should reflect those jobs.

Run a smoke test

Use the exported variables and Python installation above. Create a synthetic PNG, such as a red circle on white, save it as probe.png, and export its path:

export LLM_IMAGE_PATH="./probe.png"

Save the script as compatibility_probe.py and run python compatibility_probe.py. It makes billable requests for chat, streaming, JSON object output, JSON Schema, a tool round trip and vision. Run without Python's optimization flag so assertions remain enabled.

import base64
import json
import os
import secrets
from pathlib import Path
from openai import OpenAI
 
client = OpenAI(base_url=os.environ["LLM_BASE_URL"],
                api_key=os.environ["LLM_API_KEY"], max_retries=0, timeout=60)
model = os.environ["LLM_MODEL"]
 
 
def request(messages, **options):
    return client.chat.completions.create(model=model, messages=messages, **options)
 
 
def prompt(text, **options):
    return request([{"role": "user", "content": text}], **options)
 
 
def chat():
    r = prompt("Say hello.")
    assert r.choices and r.choices[0].message.role == "assistant"
    assert r.choices[0].message.content
 
 
def streaming():
    chunks = prompt("Say hello.", stream=True,
                    stream_options={"include_usage": True})
    text, usage, finished = [], None, False
    for chunk in chunks:
        for choice in chunk.choices:
            if choice.delta.content:
                text.append(choice.delta.content)
            finished |= choice.finish_reason is not None
        if chunk.usage is not None:
            usage = chunk.usage
    assert text and finished and usage is not None
 
 
def json_object():
    r = prompt("Return a JSON object with value 7.",
               response_format={"type": "json_object"})
    assert r.choices[0].finish_reason != "length"
    assert isinstance(json.loads(r.choices[0].message.content or ""), dict)
 
 
def schema_output():
    schema = {"type": "object", "properties": {"value": {"type": "integer"}},
              "required": ["value"], "additionalProperties": False}
    r = prompt("Return the integer 7 as JSON.", response_format={
        "type": "json_schema", "json_schema": {
            "name": "probe", "strict": True, "schema": schema}})
    assert r.choices[0].finish_reason != "length"
    value = json.loads(r.choices[0].message.content or "")
    assert isinstance(value, dict) and set(value) == {"value"}
    assert type(value["value"]) is int
 
 
def tools():
    definitions = [{"type": "function", "function": {
        "name": "get_marker", "description": "Get the marker needed to answer.",
        "parameters": {"type": "object", "properties": {}, "required": [],
                       "additionalProperties": False}}}]
    messages = [{"role": "user", "content": "Use get_marker; return its exact marker."}]
    assistant = request(messages, tools=definitions).choices[0].message
    assert assistant.tool_calls, "No tool call under automatic selection"
    messages.append(assistant.model_dump(exclude_none=True))
    marker = "probe-" + secrets.token_hex(8)
    for call in assistant.tool_calls:
        assert call.type == "function" and call.id
        assert call.function.name == "get_marker"
        assert json.loads(call.function.arguments) == {}
        messages.append({"role": "tool", "tool_call_id": call.id,
                         "content": json.dumps({"marker": marker})})
    assert marker in (request(messages).choices[0].message.content or "")
 
 
def vision():
    image = base64.b64encode(Path(os.environ["LLM_IMAGE_PATH"]).read_bytes()).decode()
    r = request([{"role": "user", "content": [
        {"type": "text", "text": "Describe the image in one sentence."},
        {"type": "image_url", "image_url": {"url": "data:image/png;base64," + image}}
    ]}])
    assert r.choices[0].message.content
    print("Image description:", r.choices[0].message.content)
 
 
for test in [chat, streaming, json_object, schema_output, tools, vision]:
    try:
        test()
        print("PASS:", test.__name__)
    except Exception as error:
        print("FAIL:", test.__name__, type(error).__name__, str(error))

We ran this probe through Requesty on September 28, 2026 against openai/gpt-6-luna, google/gemini-2.5-flash and mistral/mistral-small-latest, and all six checks passed. In the same session, openai/gpt-4.1-mini named the wrong color for a solid-color test image in two of three tries, which is why you should read the image description rather than trust a non-empty answer.

Read PASS as the result of that probe. One valid object does not prove strict schema enforcement. Automatic tool selection also tests model behavior; add required and named-selection probes where supported. Inspect the image description against the synthetic image, then use a JSON Schema validator and representative task evaluations. A failed advanced-feature check leaves basic Chat support as a separate result.

Check the production contract

Troubleshoot by symptom

Match errors by HTTP status and documented codes, as Together recommends. For quota failures, use our 429 causes and retry guide. Protocol checks and provider-switching evaluations answer different questions.

When a gateway helps

We provide one OpenAI-compatible endpoint and an Anthropic-compatible endpoint for 600+ models. The same client code reaches providers through one Requesty key. You also get fallbacks when a provider fails, monthly spend caps and per-request logs alongside unified usage analytics.

Use https://router.requesty.ai/v1 for the OpenAI SDK or https://router.requesty.ai for the Anthropic SDK. For the coding-agent setup, see Claude Code with an API key.

Inspect the catalog capability fields before selecting models: supports_tool_calling, supports_vision, supports_reasoning, supports_output_json_object and supports_output_json_schema. This read-only query lists approved chat models advertising schema output and requires jq:

curl https://router.requesty.ai/v1/models \
  -H "Authorization: Bearer $LLM_API_KEY" \
  | jq '.data[] | select(.supports_output_json_schema == true) | .id'

Authenticated discovery reflects organization and key approvals; /v1/models covers chat models. Choose a fallback chain and its order, then call policy/your-policy-name. Our routing-policy guide covers that setup.

Provider-specific features still vary by model and route. Test the features you depend on across every model in your fallback chain.

Keep one integration across providers

Follow our quickstart to configure the endpoint and key, then run your acceptance tests. Create a Requesty account to add models, fallbacks and spend controls.

Sources

Provider sources are linked in the matrix and beside each limitation.

• OpenAI Chat Completions contract
• OpenAI streaming event contract
• Vercel compatible-provider reference
• Open Responses specification
• Requesty Responses endpoint
• Requesty model capability reference