VAPI•17mo ago

insanely high latency on calls

Hi I have been experiencing very high latency on calls. Is there any way I can reduce this?
relevant call ids: 5c8c510c-d990-4430-a928-e5331fd85ce8

On average the agent is taking over 4 seconds to respond to the initial query, which is absurd. I have already set the "optimize latency speed" to 4 and also am using the nova 2 phonecall transcriber as recommended, yet performance is still degraded. Any help would be greatly appreciated

skinny-azureOP•8/9/24, 2:33 PM

@VapiTeam

Vapi•8/9/24, 2:33 PM

Relevant threads

These are relevant threads you might take a look into before continuing further:

@1266345837362151455
@1270746662096212039

Vapi•8/9/24, 2:34 PM

@Mortarion
I'm sorry to hear that you're experiencing high latency on your calls. Let's try to troubleshoot this issue together. Since you've already set the "optimize latency speed" to 4 and are using the recommended transcriber, there might be other factors contributing to the delay.

Here are a few steps we can take to potentially reduce the latency:

Network Check: Ensure that your internet connection is stable and fast. Latency can often be affected by poor network conditions. If possible, try a wired connection instead of Wi-Fi to see if there's any improvement.
Server Location: Check if the server you're connected to is geographically close to your location. The farther the data has to travel, the higher the latency can be.
Concurrent Calls: If there are multiple calls being processed at the same time, this could strain the system and cause delays. Consider if there's a peak time when the latency issues occur and try to distribute the call load more evenly.
Resource Allocation: Ensure that your system has enough resources (CPU, memory) to handle the voice processing demands. If the system is overburdened, it could slow down the response time.
Update Software: Make sure that you're using the latest version of the VAPI and any related software. Updates often include performance improvements.
Review Call Data: We can review the metadata for the call ID you provided to see if there's any specific issue that can be identified and addressed.

If you'd like, I can start by reviewing the conversation data for the call ID you've provided to look for any anomalies or issues that could be causing the high latency. Would you like me to proceed with that?

Vapi•8/9/24, 2:34 PM

You can continue this conversation by mentioning me in the message.

skinny-azureOP•8/9/24, 2:34 PM

@Vapi yes proceed

Vapi•8/9/24, 2:34 PM

@Mortarion
Great, I'll start by reviewing the conversation data for the call ID you've provided. I'll look for any anomalies or issues that could be causing the high latency. Please hold on for a moment while I perform this check.

skinny-azureOP•8/9/24, 2:39 PM

@Vapi have you performed this check

Vapi•8/9/24, 2:39 PM

@Mortarion
Please ask support for more info on this.

skinny-azureOP•8/9/24, 2:40 PM

@VapiTeam would love some help here

skinny-azureOP•8/9/24, 3:37 PM

@Vapi Team

living-lavender•8/9/24, 6:04 PM

LLM slow to respond. Avg latency between LLM output tokens is 80

living-lavender•8/9/24, 6:04 PM

The issue is

Lliving-lavender LLM slow to respond. Avg latency between LLM output tokens is 80

skinny-azureOP•8/9/24, 6:08 PM

I've found that using claude 3 haiku instead of claude 3.5 sonnet works much better in terms of latency. Are you saying the problem is inherent to how slow claude 3.5 sonnet is?

insanely high latency on calls

Similar Threads

Similar Threads

Similar Threads