What Latency Means on a Phone Call
In a voice AI system, latency usually refers to one specific gap: the silence between the moment a caller stops talking and the moment the AI's response audio begins. The independent benchmarking project Openbenchmarks calls this metric TTFAB, time to first audio byte, and defines it precisely: the interval from the caller's speech ending to the agent's reply starting, measured from a saved recording of the actual call (Openbenchmarks, 2026).
That distinction matters. A platform's own dashboard can report a fast number because it is timing only its internal processing, not the full round trip a caller experiences, including network hops between separate speech-to-text, language model and text-to-speech vendors.
Why 200 to 300 Milliseconds Is the Reference Point
The industry did not invent its latency target from scratch. Human conversation has a measured baseline. A 2009 study in the Proceedings of the National Academy of Sciences analyzed turn-taking across ten typologically diverse languages and found a modal gap of roughly 0 to 200 milliseconds between one speaker finishing and the next beginning (Stivers et al., PNAS, 2009). People do not tolerate long silences in conversation, and they do not make an exception for a machine.
Telecom engineering set its own bar earlier. The ITU-T G.114 standard recommends a one-way delay under 150 milliseconds for transparent conversational quality, and treats 400 milliseconds as the outer limit for general network planning before interactivity degrades (ITU-T, G.114, 2003). Voice AI vendors that market "sub-500ms" latency are citing a number close to this reference range, not an arbitrary marketing figure.
Where the Time Actually Goes
A typical voice AI call is not one system responding. It is three systems handing off in sequence: speech-to-text, a language model generating the reply, and text-to-speech turning that reply back into audio. Telnyx, a vendor that publishes its own latency breakdown, puts typical stitched-stack timing at 100 to 300 milliseconds for transcription, 350 to 1,000 milliseconds for the language model, 90 to 200 milliseconds for speech synthesis, and another 50 to 200 milliseconds in network round trips between vendors when those three components are not hosted together (Telnyx, 2026, vendor source). Add those ranges up and a stitched pipeline can consume well over a second before the caller hears anything, regardless of how fast any single component is in isolation.
What Independent Measurement Found
Openbenchmarks built a caller robot that dials each platform over a real phone call, reads a fixed script, and records both sides of the conversation on a single clock. It then locates the caller's speech-end and the agent's reply-start in that recording using a speech detector with a cross-check, discarding any turn where the two sides talked over each other or the detector could not get a clean read (Openbenchmarks, 2026). The published median and 95th-percentile results across five platforms, in milliseconds:
| Platform | Median TTFAB | P95 TTFAB | |---|---|---| | Telnyx | 1,296 | 1,856 | | ElevenLabs | 1,424 | 1,768 | | Bland AI | 1,520 | 2,248 | | Vapi | 1,558 | 2,008 | | Retell AI | 1,740 | 2,259 |
Every platform in that test landed well above the 500-millisecond figure that appears across voice AI marketing pages, and three of the five crossed two seconds at the 95th percentile, the point at which a caller notices the conversation has broken down rather than just slowed.
The Gap Between the Number on the Page and the Number on the Call
Vapi's own site advertises latency under 500 milliseconds at scale. Vapi's own engineering blog is more specific about what that requires: a well-tuned stack using a fast transcription provider and a low-latency text-to-speech tier, with the company's stated production range landing between 500 and 900 milliseconds even under favorable configuration (Vapi, 2026, vendor source). The independent test measured a 1,558 millisecond median for the same platform. Neither number is necessarily false. One measures a best-case configuration under controlled conditions. The other measures a fixed script read over a real phone call with no tuning applied on the buyer's behalf.
What to Ask Before Buying
A latency claim on a vendor's homepage is not evidence of what a deployed system will feel like on a support line. Three questions separate a marketing figure from an operational one: is the number measured from recorded call audio or from the platform's own internal timestamps, what is the 95th percentile and not just the median, since tail latency is what callers notice and complain about, and are the speech-to-text, language model and text-to-speech components hosted on the same network as the call itself, or stitched together across separate vendors.
What Commonly Goes Wrong
Buyers frequently benchmark a vendor on a controlled demo call, a short scripted exchange with no tool calls and no background noise, then deploy the same system into a support flow that requires a database lookup mid-conversation. That lookup adds a network round trip the demo never tested. Latency also degrades under concurrency, when a platform is fielding many simultaneous calls rather than one clean test call, a condition rarely reflected in a published benchmark number. The fix is not to distrust every vendor claim outright. It is to ask which conditions produced the number before treating it as a performance guarantee.
Sources: Universals and cultural variation in turn-taking in conversation — Stivers et al., PNAS 2009 · Recommendation G.114: One-way transmission time — ITU-T · Voice agent latency benchmark (2026) — Openbenchmarks · Voice agent latency comparison benchmark (2026) — Openbenchmarks · Voice AI agents compared on latency — Telnyx · Speech Latency Solutions: Complete Guide to Sub-500ms Voice AI — Vapi