flat-fuchsia•14mo ago
Experiencing High Latency
On the VAPI dashboard, our latency is supposed to be ~1100ms - but on the actual calls it ranges between ~3500ms and ~4000ms. Our Assistant's prompt structure is pretty simple and well structured. We also tried running it with a super simple prompt to see if the issue was on the prompting side, and still got the same delay.
Here is some examples from our call logs:
Turn Latency: 3745ms (Endpointing 1501ms, Model 1351ms, Voice: 887ms)
Turn Latency: 3736ms (Endpointing 1502ms, Model 1287ms, Voice: 924ms)
Our calls are made from Europe. We're available to invest in a solid server infrastructure if needed.
Currently our only goal is to lower the latency at least <2000ms as the current latency is a disaster with the calls.
Thanks for the help.
13 Replies
@DREW
I'm sorry to hear that you're experiencing higher latency than expected. It's great that you're willing to invest in server infrastructure to improve performance. To address the latency issue, we should consider a few potential solutions:
1. Server Location: Ensuring that your servers are geographically closer to your user base in Europe can significantly reduce latency. If VAPI has servers located in Europe, it would be beneficial to utilize those.
2. Optimization: Reviewing the Assistant's prompt structure and optimization, even though you've mentioned it's simple and well-structured, there might be room for further optimization.
3. Network Quality: Checking the network quality and stability on both sending and receiving ends can also affect latency. A dedicated and high-quality internet connection might help.
4. Concurrent Requests: If you're handling multiple calls concurrently, ensure that your infrastructure can handle the load without performance degradation.
5. Caching: Implement caching strategies for frequently used data to reduce the time taken to fetch it repeatedly.
6. Profiling: Profiling the API calls to identify any specific bottlenecks in the system that could be causing the delay.
7. Support Inquiry: Since the latency issue is quite significant, it might be beneficial to reach out to VAPI support directly to see if there are any known issues or if they can provide a more tailored solution.
Would you like me to assist you in reaching out to VAPI support to address this issue more thoroughly?
You can continue this conversation by mentioning me in the message.
flat-fuchsiaOP•14mo ago
Can't click on "Ask for help". Still need help from the human team please.
wise-white•14mo ago
hey @DREW you might be seeing the latency for web calls not telephony calls. Also can you please share the call id and assittant id if I am not correct.
flat-fuchsiaOP•14mo ago
Nope, the latency for phonecalls is just 200-300ms more. Assistant ID: f5503666-5b18-470d-bd62-8a75569cc357 - Call ID: 78590aff-434d-48ae-86fd-c7fa9cb267bb
wise-white•14mo ago
can you try chaging "optimizeStreamingLatency" to 1 or 2 please.
flat-fuchsiaOP•14mo ago
Tried, it's even slower now. Turn Latency: 4967ms (Endpointing 1501ms, Model 2145ms, Voice: 1309ms)
wise-white•14mo ago
Latency is mostly in TTS part, in first glance i am unable to pinpoint exact issue allow me sometime to get back to you.
flat-fuchsiaOP•14mo ago
thanks
wise-white•14mo ago
@DREW please try now, it will be fixed by now.
flat-fuchsiaOP•14mo ago
Should i try again with the "optimizeStreamingLatency" to 1 or 2, or what exactly? Thx @Shubham Bajaj
wise-white•14mo ago
yes try with 2.
@DREW can you try by setting it to 3.
also do let me know how it goes.
flat-fuchsiaOP•14mo ago
@Shubham Bajaj nothing seems to change. Also, the voice have really big accent while speaking.
wise-white•14mo ago
@DREW for comparison can you try switiching to our api keys for testing?