Responses API vs Chat Completions: what changes?

OpenAI recommends the Responses API for new projects and still supports Chat Completions. Chat Completions takes and returns messages; Responses takes a string or typed input items and returns an output array of typed items. Requesty translates web search across providers, while OpenAI's stored state, file search, and code interpreter stay tied to OpenAI.

What changes in the API?

OpenAI's migration guide breaks the move into three jobs: call /v1/responses, read typed output items, and decide how to carry state between turns. Most of the migration work sits in the fields.

The migration guide says the output array can contain reasoning, tool-call, and message items, so a parser cannot assume the first item is the message. A single Responses request can also call several built-in tools alongside custom functions.

Check function schemas during the move too. A function that ran in non-strict mode in Chat Completions may be handled differently when strict is omitted in Responses. Our guide to Structured Outputs across providers covers compatibility across models.

There is also a model-specific reason to consider switching. Starting with GPT-5.4, OpenAI says Chat Completions does not support tool calling with reasoning_effort values other than none.

OpenAI reports a 3% SWE-bench improvement using Responses with the same prompt and setup, plus a 40% to 80% improvement in cache utilization in internal tests. Measure the difference on your own workload before turning either figure into a savings estimate.

Which features survive a provider switch?

Endpoint support, stored continuation, and hosted tools vary by route, so check each one before switching. Our OpenAI-compatible API guide covers the checks to make.

Sources: Requesty inference APIs, Requesty web search, OpenAI reasoning guide, and Open Responses specification.

The OpenAI Agents SDK drops previous_response_id and conversation_id when a model is routed through Chat Completions. That behavior belongs in your fallback tests.

A hosted tool runs on the provider's side. A developer-hosted function runs in your application, and Open Responses uses that distinction in its multi-provider specification.

A gateway can translate a tool definition, but a fallback still needs a backend that can do the job. In our routing platform comparison, TrueFoundry keeps a follow-up using previous_response_id on the original backend and disables cross-target fallback for that follow-up.

Web search can use one tool definition

Requesty's web search docs list OpenAI, Azure OpenAI, Anthropic, Vertex AI, xAI, and Perplexity as native search providers. Requesty translates one tool definition to each provider's format across Chat Completions, Responses, and Messages.

The example below uses the documented Responses model ID and reads the key from REQUESTY_API_KEY. The response can include web_search_call items and url_citation annotations.

import os
from openai import OpenAI
 
client = OpenAI(
    base_url="https://router.requesty.ai/v1",
    api_key=os.environ["REQUESTY_API_KEY"],
)
 
response = client.responses.create(
    model="openai-responses/gpt-5.6-sol",
    input="What are the latest releases from OpenAI?",
    tools=[{"type": "web_search"}],
)
 
print(response.output_text)

import OpenAI from "openai";
 
const client = new OpenAI({
  baseURL: "https://router.requesty.ai/v1",
  apiKey: process.env.REQUESTY_API_KEY,
});
 
const response = await client.responses.create({
  model: "openai-responses/gpt-5.6-sol",
  input: "What are the latest releases from OpenAI?",
  tools: [{ type: "web_search" }],
});
 
console.log(response.output_text);

Search shows what a gateway can normalize. Our post on provider-native tools through a gateway examines that boundary in more detail.

Choose where conversation state lives

With OpenAI Responses, previous_response_id links one response to the next, so your app does not have to send the full history itself. OpenAI says prior input tokens in the chain are still billed, and the previous response's top-level instructions do not carry over. Send those instructions again on the next turn.

Storage changes the integration too. OpenAI stores Responses by default, and its data guide gives a 30 day application state retention period by default or when store is true. With Zero Data Retention enabled, store is treated as false.

OpenAI also stores Chat Completions by default for new accounts. According to its migration guide, setting store: false disables storage in either API.

For a flow that needs provider fallback, keep history in your app, set store: false, and send the needed context on each turn. For stateless reasoning flows, OpenAI's migration guide says to replay encrypted reasoning items when reasoning must continue; the model-family limit still applies when switching models.

Replaying history can increase input volume. On Requesty, cache_control breakpoints on Responses input items are translated for Anthropic models, and cache read and write tokens appear in usage. Measure cache behavior on the flow you intend to route.

What does replaying context cost?

Take a flow with 100,000 requests per month. Assume each request includes 3,000 tokens of replayed context, with no cache discount, on GPT-6.1 Sol at its November 2026 input price of $2 per million tokens (Requesty's coding benchmark).

Context length drives this number. Doubling it adds $600 per month at the same input rate, before any cache discount and other request costs. Requesty's quickstart states that a 5% service fee is added when you top up credits, and that each request's exact cost appears in usage.cost.