OpenAI is testing pay-per-outcome: what changes when evals become billing infrastructure

On 31 August The Information reported that OpenAI has started letting some of its largest customers pay only when the AI completes the job. The example given was a customer support interaction handled end to end. The arrangement is limited to select accounts. OpenAI has not announced it, and the terms, prices and customers are not public.

That is a thin fact base for a trend piece, so I want to be careful about what is on the record and what follows from it. What is on the record: the largest model vendor has begun, selectively, to sell results instead of capacity. What follows from it is the interesting part, because it changes what the important number in your stack is.

Support solved this a year ago, on purpose

Outcome pricing is not new. It is new at the model layer.

Intercom charges $0.99 for each conversation its Fin agent resolves and nothing for the ones it does not. Zendesk went further in May, billing only what it calls Verified Resolutions, confirmed by an LLM evaluation within 72 hours, at roughly $1.20 to $1.50 on committed volume. Salesforce launched Agentforce at $2 per conversation regardless of outcome, found customers hated the unpredictability, and moved to per-action Flex Credits at about 10 cents each.

Notice where this worked and where it did not. It worked in support because a resolution is one of the few AI outputs anyone can define: the ticket closed, no human touched it, the customer did not reopen it within a window. It stalled at Salesforce because "conversation" and "action" are consumption units wearing an outcome costume. An action that fails still bills.

The application vendors could do this because they own the whole loop: the agent, the tools, the ticket system, and the evaluation that decides whether the ticket is resolved. OpenAI owns the model. When the model vendor starts charging per outcome, someone has to define the outcome, and that someone is now negotiating with you.

Evals just became billing infrastructure

The New Stack put it well: outcome pricing turns evals from quality checks into billing infrastructure. That sentence deserves to be read slowly.

Today, your eval harness answers "is this model good enough for this task." It runs offline, it tolerates noise, and nobody's invoice depends on it. Under outcome pricing, the same kind of evaluation answers "does this run get billed." It runs on every task, it has to be precise, and both parties have money on the result.

Three problems arrive immediately.

Partial completion. An agent that finishes nine of ten steps and fails the last one consumed 90% of the compute and delivered 0% of the outcome. Under token pricing you pay for all of it. Under outcome pricing you pay nothing, and the vendor eats the run. Which means the vendor prices that risk into the per-success fee, which is why per-resolution rates cluster around a dollar rather than a cent.

Semantic success. A coding agent that rewrites the auth module, passes every test, and ships a regression completed the task by any automated definition. The customer disagrees. Any evaluation that can catch that is a judgement, and an LLM-as-judge with a false positive rate bills the customer for unfinished work while a false negative bills the vendor for finished work.

Who grades. When the vendor runs the agent, sets the success criteria, and runs the evaluation, it is grading its own work and invoicing on the grade. Zendesk gets away with a 72-hour verification window because the customer can see the ticket. What is the equivalent for a research agent that ran for three hours?

None of this makes outcome pricing wrong. It makes the definition of success a contract term, and contract terms get negotiated by whoever has the better data.

Why this matters even if you never sign one

Most teams reading this will not be offered outcome pricing by OpenAI this year. It is going to select enterprise accounts with well-defined workloads. So why care?

Because the data that lets you evaluate an outcome deal is the same data that should already be driving your routing, and most teams do not have it.

Consider what you need to know to price a pay-per-resolution offer at, say, $1.00 per resolved conversation. You need your current cost per attempt on the model you use. You need your success rate per attempt. Divide one by the other and you have your cost per successful outcome today. If it is $0.60, the offer is a 67% premium for risk transfer. If it is $1.40, the vendor is subsidising you and you should ask what they know that you do not.

Now notice that the same two numbers, cost per attempt and success rate per attempt, broken out by model, are exactly the inputs to the routing decision we described in cost per task, not cost per token. A cheaper model with a lower success rate can cost more per outcome than an expensive one. A model that finishes in fewer turns beats one that is cheaper per token. Teams that measure this already route on it. Teams that do not are about to be asked to negotiate a contract on numbers they have never seen.

OpenAI's own framing points the same way. Greg Brockman told VentureBeat during the Astra launch that what the market wants is the price per task: "can you get the thing done for an appropriate cost at appropriate speed?" When the vendor's president says the unit of account is the task, it is a good moment to make sure you can measure tasks.

The risk transfer runs both ways

There is a version of this story where outcome pricing is straightforwardly good for buyers. A bill that arrives only when something worked is easier to defend to a finance team than one that arrives regardless, and Futurum's May survey found 27% of buyers already prefer outcome-based models, with vendors offering seats alone being disqualified before evaluation starts.

There is another version where it entrenches the vendor. If success is defined in the vendor's evaluation, measured by the vendor's judge, on the vendor's model, then switching models means renegotiating the definition of success, not just changing a model string. Token pricing is portable. A million tokens on one provider is comparable to a million tokens on another. An outcome contract is not portable unless you own the evaluation.

That is the design principle. Own the definition of success. Run your own evaluation, on your own task set, with your own pass criteria, and log it against every model you route to. We argued in private eval harness beats public benchmarks that your own harness is the only benchmark that predicts your production. Under outcome pricing it is also the only thing that lets you audit the invoice.

What to do this week

Instrument success, not just spend. Tag every agent run with a task identifier and a terminal outcome using request metadata, and reconstruct multi-call tasks with session reconstruction. If you cannot say what share of runs succeeded last week, you cannot evaluate any outcome offer.

Compute cost per successful task per model. Join cost tracking to your success labels. This one table is your negotiating position with every vendor and your routing policy in one artifact.

Route on it. Once you have cost per outcome by model, put the cheapest-per-outcome model first in a routing policy and the higher-success model as the escalation path. That is outcome pricing you implement yourself, without a contract.

Keep the judge independent of the model. If you use an LLM to grade task completion, do not grade a model with itself. Route the judge call to a different provider through the same gateway so the evaluation survives a model switch.

Track attempts per outcome as a first-class metric. Retries, fallbacks and tool loops all inflate attempts. Tool call analytics show where a model burns turns without converging. Under any pricing model, that is where the money goes.

The takeaway

OpenAI selling results instead of tokens to a handful of enterprises is a small fact. The direction it sets is not. The unit of account for AI is moving from the token to the task, and the vendor with the best task-level data will set the terms. That vendor should be you. The routing layer already sees every attempt, every retry, every model and every dollar; add the outcome label and you have the dataset that both prices the next contract and picks the next model.