Where does voicebot latency actually come from? Let’s walk through the call
Index
You call a company, a voicebot answers, and you ask your question. Then, complete silence. It’s not long, maybe a second, but it’s long enough that you glance at your phone screen to check if the call is still connected. It’s long enough that you start talking again, just as the bot starts answering, and then you both trip over each other.
In text chat, a few seconds of delay barely register. People are used to typing indicators, to waiting for a reply while doing something else. But on the phone, every “gap” in the conversation is noticeable. In natural conversation, people expect a response within a fraction of a second. Anything slower starts to feel like hesitation or a broken system.
As voicebot technology continues to gain popularity, you’re seeing more and more discussions about latency and the importance of keeping it as low as possible. But little is said about what actually causes latency and what steps you can take to keep it as low as possible.
A voice call with an AI agent isn't one system doing one thing. It's a chain of steps, each with its own processing time, and delay can creep in at almost any link. In this blogpost, we look at where latency can occur across that chain and what you can do to reduce it.
Latency can start before the voicebot does anything
Let’s start at the beginning: the initial call from the user. He starts the conversation and from there, the audio travels through a mobile or fixed-line network before it reaches the infrastructure connected to the voicebot.
Along the way, the call may pass through multiple telecom providers, gateways and network components. The longer or more complex that route is, the more delay it introduces. This also means the voicebot platform itself isn’t always the main cause of latency. If audio is routed through several international network connections before it reaches the voice platform, additional latency has already been introduced before any speech recognition or AI processing begins.
So, geography matters. If the caller, telecom infrastructure and voice platform are located relatively close to each other, the audio has less distance to travel. And this helps reduce latency.
The media path matters a lot
Once the call “travels” further, the audio still has to be processed and forwarded. Components such as media gateways, Session Border Controllers (SBCs) and RTP proxies may be involved.
We will not go into detail about these terms, but it helps to think of these components as traffic controllers for phone calls. They make sure audio reaches the right destination and that different systems can communicate with each other. Every additional step adds processing time.
The same applies when audio formats need to be converted. If one system uses a different audio codec from the next, the audio has to be transcoded along the way.
A single conversion may have little impact, but if the audio is converted several times during one call, you'll understand that all of this adds up and can increase latency.
Some latency is introduced on purpose
Audio is delivered in packages, which don’t always arrive over a network at regular intervals. Some arrive slightly earlier or later than others. This variation is called jitter. To prevent the conversation from sounding choppy, voice platforms use a so-called jitter buffer. The system holds incoming audio packets and then plays them back at a more consistent rate. This improves audio quality, but it also adds a little latency.
You have to find some balance here. If the buffer is too small, the audio becomes unstable. But if it’s too large, the bot starts to feel sluggish. A stable network connection allows the system to work with smaller buffers and keep the delay lower. Now let’s take a closer look at the actual call.
How does a voicebot know you have finished speaking?
One of the most important sources of perceived latency appears at the exact moment the caller stops talking.
Imagine someone says:
“Can you move my appointment for tomorrow?”
After the word “tomorrow”, the voicebot cannot immediately know with complete certainty that the sentence is finished. The caller might continue with:
“...to Friday afternoon.”
The system therefore waits briefly to determine whether the person has really finished speaking. This process is called Voice Activity Detection (VAD) or endpointing.
If the system waits too long, the voicebot feels slow. But if it responds too quickly, it interrupts the caller before they have finished.
This means trying to reduce latency is not simply a matter of making the waiting period as short as possible. The settings need to suit the type of conversation. Short, predictable answers may allow more aggressive endpointing. But open-ended questions require a little more breathing room.
The bot has to decide what to say
Once the text is available, the conversational logic takes over.
A simple, deterministic bot may only need to recognise an intent and choose a predefined response. More advanced use cases might require a CRM lookup, a database request or address a large language model, if the bot is probabilistic and uses AI.
All those steps take time.
Take a voicebot that helps someone reschedule an appointment. It first identifies the customer, then retrieves the existing booking and finally checks which alternatives are available. If those requests all happen one after another, their response times stack up.
This flow deserves a closer look. Which information is really needed before the bot can respond? Are there requests that can run at the same time? Which data can be reused instead of fetched again?
Streaming can keep the conversation moving
In some voicebot architectures, processing happens more sequentially. Speech-to-Text waits for more complete audio before finalising a transcript, the bot or LLM generates most or all of its response before passing it on, and Text-to-Speech only starts once enough text is available.
So-called “streaming” can reduce some of that waiting. Speech-to-Text can process audio while the caller is still speaking. A large language model can start passing on the first part of its response before the full answer is complete. And Text-to-Speech can then begin generating audio as soon as it has enough text to pass on.
The exact behaviour depends on the technologies and configuration being used, but the principle is the same: the more these steps can overlap, the less time the caller perceives latency.
It has to say that streaming doesn’t remove the processing time of STT, the language model or TTS. But it can reduce the amount of sequential waiting in the chain and make the conversation feel way more responsive and “human”.
The answer still has to reach the caller
A lot of steps that add latency, that’s for sure. And the audio hasn't even reached the caller yet. Once the audio is “ready”, it still has to travel back through the voice infrastructure and telecom network. It sounds like a lengthy process, but it all happens in milliseconds.
The same factors come into play again: routing, network distance, buffering, transcoding and connection quality. So even a very fast AI stack can still feel slow if the audio takes an inefficient route back to the caller.
Reducing latency means looking at the entire path the audio travels, in both directions.
Look at the chain as a whole
It’s tempting to describe latency as one number. And it's hard to trust voicebot providers who say, “We can reduce your latency by X seconds”, because, quite simply, a lot of providers only manage part of the call, and therefore have no control over all of its components.
What’s for sure is that it's important to look at the entire process, which in most common cases proceeds as follows:
Incoming call → telecom network → Speech-to-Text → bot logic or LLM → Text-to-Speech → telecom network → caller
STT, LLM and TTS are most commonly part of the voice orchestration platform, like Seamly.
The most useful moment to focus on is usually the pause between the caller finishing their sentence and the voicebot starting its reply (the Voice Activity Detection we discussed earlier).
If that pause is too long, break it down. But, also check out the other components. Is endpointing taking too much time? Is the Speech-to-Text provider slow? Are external APIs holding things up? Does the language model take too long to produce its first words?
Once you know where the time goes, latency stops being a vague problem and becomes something you can actually improve. However, it often means that the solution doesn’t lie with a single link in the chain.
So, why does a voicebot sometimes feel slow?
By now we know that latency usually builds up across several parts of the voice chain. A little delay in endpointing, a slightly slower API call or extra processing time in Speech-to-Text. We also know that reducing latency often means improving several parts of the chain.
But there are also ways to make waiting feel less noticeable. Filler audio is one example. While the bot is processing a request, you can play a short sound or phrase, such as the sound of typing or “Let me look that up for you.”
It doesn’t reduce the actual latency, but it can make a pause feel more natural and reassure the caller that the conversation is still moving. Used carefully, techniques like this can improve the experience without pretending the underlying delay has disappeared.
The end goal is not to chase zero latency. Some processing time will always be part of a voice interaction, and people know that. What matters is understanding where that time comes from and designing the conversation around the moments where a short pause is unavoidable.
When those pieces come together, the technology starts to fade into the background and the conversation feels much closer to what people expect from a voicebot: responsive, clear and easy to continue.
If you are exploring voice for your own platform or customer proposition and want to understand what the right setup could look like, get in touch. We would be happy to help you take the first steps in your voice journey!
