What happens on turn forty

A single-turn voice benchmark cannot tell you what a long session costs, whether latency drifts, or what the agent does while it thinks. So I measured those three things across Gemini Live, OpenAI Realtime, and our own stack.

Everything here is reproducible: github.com/virevolai/voice-benchmarks.

Almost every voice-AI benchmark measures one turn. You say something, the model answers, someone records the milliseconds. It is a real number and it decides almost nothing, because nobody ships a product where the user speaks once.

The interesting question is what happens on turn forty. Ten minutes into an onboarding call. Deep in a debugging session. Somewhere in the middle of a support conversation that went long. That is where products break, and it is where nobody publishes numbers.

So I ran the same forty-turn conversation against Gemini Live, OpenAI’s Realtime API, and our own stack, and recorded every turn.

Three findings. We are the slowest to first audio. One vendor’s cost grows seventeen-fold across a single conversation. And two of the three stacks go completely silent the moment they have to think — which turned out to be the result worth caring about.

Method

Every vendor got an identical scripted conversation: a codename planted on turn one, thirty-eight middle turns, and a recall question at the end that checks the session still holds context. All three passed. A cheap session that quietly forgot the conversation is not a cheap session.

The latency boundary, used everywhere below: from the instant the final speech sample leaves the client to the first byte of response audio arriving. Connection setup is excluded and reported separately.

Each vendor’s own endpointing decides when a turn ends, so part of every figure is that vendor’s judgment about when you stopped talking. That is deliberate — it is what a user experiences — and it means these are not time-to-first-token numbers.

Cost uses vendor-reported token counts against published list rates on the run date. Estimates, not invoices.

Two things are not comparable, and it matters. OpenAI’s Realtime API takes 24 kHz input where the others take 16 kHz, and tokenisation is rate-dependent, so absolute token counts across vendors are meaningless — only the shape of each curve can be compared. And our stack bills occupied time rather than tokens, so it has no per-turn token cost to put on the same axis. The question that survives all of this: does a later turn cost more than an earlier one?

Latency: we lose

Gemini is fastest. We are slowest, by about 330 ms against Gemini and 140 ms against OpenAI. That gap is audible.

These are settled-state figures: they exclude turn one, which on our stack costs 1,805 ms while the session starts up. Across all forty turns including that one, our p95 is 1,626 ms rather than 1,586 ms. Both numbers are in the published run data.

I am putting this near the top because everything below depends on these numbers being trustworthy, and a benchmark published by a vendor who wins every row is not worth reading. Hosted realtime models have serious inference engineering behind them. We do not beat them on raw first-audio latency.

What we do have is a curve that does not move. Mean latency over the first five turns was 1,554 ms; over the last five, 1,358 ms. It got slightly faster as the session settled. Whatever the number is, turn thirty-eight looks like turn three.

Cost: the seventeen-fold problem

The same forty turns, priced per turn:

On Gemini, the fortieth turn cost seventeen times the first. Same fixture, same amount of speech, same work performed. Context had grown from 559 to 13,231 tokens, and every turn was billed for all of it. Seven minutes of audio came to $0.80.

The mechanism is visible in the response: zero cached tokens on all forty turns. No discounted-context accounting, so the full history was re-billed on every exchange. Cost is roughly quadratic in turn count rather than linear in minutes.

OpenAI has largely solved this. Prefix caching is real — by turn forty, 3,648 of 3,740 input tokens were cached and per-turn cost was essentially flat.

The caching is documented as best-effort, and the run shows what that means in practice. It did not engage at all for nine turns. Cost climbed normally to $0.025, collapsed to $0.0067 on turn ten when 896 tokens finally cached, then stayed low. Nine of the forty turns took no cache hit, and two cost roughly five times the median.

So the fair criticism of OpenAI is not that cost grows. It does not. It is that individual turns are unpredictable, which is a harder thing to budget than a number that is merely high.

A shorter benchmark would have concluded the opposite. I ran a six-turn probe first, saw a single cache hit, and wrote down that caching does not work in live sessions. It stopped three turns before it started working. Measure past the point where the behaviour begins.

Our own curve is flat by construction, not by optimisation. We bill occupied session time, so there is no conversation history to re-bill. That is not cleverness, it is a different unit of account, and it is the whole reason the number is 1.0.

The pause

Here is the finding I did not go looking for.

Ask a voice agent something genuinely hard — something it should look up, or reason through properly instead of answering from reflex — and it has to go away and do that. The question is what happens to the conversation while it does.

Two seconds of silence on a call is long. At four you say “hello?” At ten you assume the line dropped. This is not latency in the usual sense. It is the difference between talking to something and submitting a request to it. People do not go silent while they think. They say hm, let me think about that, and if you ask them something else in the meantime, they answer you.

So: is the session still alive while the tool runs?

Getting the measurement wrong

The obvious test fails, and I fell into it.

I dispatched a slow tool, streamed silence, and watched to see whether the agent would keep talking. Both hosted vendors sat silent for the full tool duration. I nearly wrote that down.

It was nonsense. A model with nothing to respond to correctly says nothing. I had measured my own fixture.

The test that means something is to speak again while the tool is pending. The agent has asked for time; you interrupt it. Does it answer, or is the session frozen until the tool returns?

That is the probe: one hard question that provokes a real delegation, then a short follow-up — “how long will that take?” — a beat after dispatch, while the first request is still outstanding. It fires on turn eight rather than turn one, because an agent needs some conversational evidence before it reaches for a heavy tool.

The result

Three runs each, holding to within a tenth of a second.

Gemini does not answer. The follow-up is heard and nothing comes back until the original call resolves. On a twenty-second tool, that is twenty seconds of a session that appears to have died.

OpenAI answers, and this is a documented feature rather than something I uncovered: the gpt-realtime release notes say long-running function calls “will no longer disrupt the flow of a session.” It works as advertised, and it is native — you do not have to build it.

We answer in roughly half OpenAI’s time. On the one axis where we are not last, we are first by a wide margin.

Why this is the number that matters

Because it changes what the product can be, rather than what it costs.

Latency and price are dials. Everyone’s move in the same direction and these gaps will narrow, possibly before you read this. An agent that can think without abandoning the conversation is a different kind of thing. It can look something up mid-sentence. It can start a slow job and keep talking to you about something else. It can say I’m still working on that and mean it.

None of that appears in a single-turn benchmark, which is a large part of why I think single-turn benchmarks have been misleading about what these systems are actually like to use.

The bug that nearly buried it

Our first three runs said we were blocked, exactly like Gemini. I believed them and was preparing to publish it.

The follow-up sent exactly 1,200 ms of trailing silence against a 1,200 ms endpointing threshold. Voice activity detection advances in 32 ms frames, so the first boundary at or above the threshold is 1,216 ms. We were one frame short. The turn was never committed, no transcript was ever published, and from outside it looked precisely like a session that had stopped listening.

The internal trace showed the opposite. The follow-up had been transcribed, the model had run, and speech synthesis had started — all while the tool was outstanding. Everything worked. A single missing frame withheld the audio.

This is in the post because the failure was indistinguishable from the finding. A benchmark that reports “blocked” cannot tell you whether the system refused to answer or was never properly asked, and those are entirely different claims. The harness now separates them: a follow-up that never endpointed is reported as inconclusive rather than as a block.

If you run this yourself, send three seconds of trailing silence.

If you are choosing

Short sessions, instant start: use a hosted API. They are faster to first audio, far simpler to integrate, and at five or ten turns the cost differences are rounding errors. If people talk to your product for two minutes, none of this applies to you.

If your agent uses tools that take real time — search, retrieval, anything that reasons before answering — check the pause before you check the price. An agent that goes silent for ten seconds is a product problem you cannot fix with a faster model, and you will not find it in a single-turn benchmark.

If sessions run long, model the cost curve before you commit. At forty turns Gemini’s per-turn cost had risen seventeen-fold and was still climbing; the run never reached the context-compression trigger that would eventually bound it. An hour-long meeting is roughly two hundred turns.

Rapid back-and-forth is punished hardest. Cost tracks turn count, not minutes, so a debugging session of short exchanges accumulates faster than a meeting where people speak in paragraphs. Your most engaged users become your most expensive ones.

What I am not claiming

That we are better. We are slower to first audio than both hosted options, by a margin you can hear.

That the cost problem is universal. It is dramatic on Gemini and largely absent on OpenAI. “Hosted APIs are expensive for long sessions” does not survive contact with the data.

That the pause result is a clean three-way comparison. Ours runs a real reasoning tool; the hosted vendors were given a stub resolving on a timer, so tool duration is fixed for them and self-determined for us. What is compared is each stack’s behaviour around the call, not the call itself.

That this is the last word. These are single runs, one fixture, one conversation shape. Prices and caching behaviour move monthly, every figure here is dated, and the harness is public so you can check it instead of trusting me.

Caveats

Gemini’s model is a preview build (gemini-2.5-flash-native-audio-preview), and its long-session figures are several weeks older than the others. Preview behaviour changes; the pause result in particular could differ next month.

All three are single sessions rather than distributions. The right next step is repetition across times of day and network conditions, plus a second conversation shape — long paragraph turns against rapid exchanges — since those stress per-turn billing very differently.

Harness, fixtures, and raw per-turn data: github.com/virevolai/voice-benchmarks. About $1.50 to reproduce.

If you build a realtime voice stack, add it. A vendor is one driver function following the existing shape. Numbers that disagree with these are welcome, particularly if the reason turns out to be a bug in the harness.