Huge Latency ~5s

generous-coffee · 2025-11-05T02:36:29.456Z

Hello, recently I have experienced a huge latency in my voice agent with following stack: - Groq Llama 4 Maverick - Vapi Voice - Azure STT Before yesterday the latency was super low, could you please take a look at following call id's and see what has happened? - 019a5083-e530-7ff9-b893-07aed41e4bdf - 019a507d-8283-7eeb-9934-3765b64b5ed1 Timestamp is 9:10pm CET

V

Vapi Support BotAPP•11/5/25, 2:38 AM

Thank you for sharing the call IDs and timestamp. To help pinpoint the cause of the ~5s latency, could you let me know if you noticed any specific pattern—such as latency occurring at the start of the call, during user input processing, or only on certain steps? This will help narrow down whether the delay is in STT, LLM, or TTS processing.

Sources:

⤵️ Mention @Vapi Support Bot for follow-ups 🐥

G

generous-coffeeOP•11/5/25, 6:51 AM

In this call 019a507d-8283-7eeb-9934-3765b64b5ed1 the biggest latency is after user's first and second message and then after his 6th, 7th and 8th message, so here there was no rule to it.

In this call 019a5083-e530-7ff9-b893-07aed41e4bdf it was a complete failure, after each message there was very long delay, it has not recognized user's speech apparently.

I remember some time ago I had some issues with Groq, because every once in a while it would have very long latencies even like 10-15 seconds, so maybe that is the issue?

In the call logs I see multiple Warnings with Endpointing timeouts:
{
"id": "log-266",
"level": 40,
"time": 1762287477433,
"timestamp": 1762287477433000000,
"severityText": "WARN",
"severityNumber": 13,
"body": "Endpointing timeout 100ms (rule:

heuristic

heuristic

heuristic

heuristic)",
"attributes": {
"category": "endpointing",
"callId": "019a5083-e530-7ff9-b893-07aed41e4bdf",
"orgId": "25e97c69-9a46-454f-a28c-1a20f5a08c43"
}
}

@Vapi Support Bot

V

Vapi Support BotAPP•11/5/25, 6:52 AM

Thank you for the detailed info and call logs. Given the "Endpointing timeout 100ms (rule:

heuristic

heuristic

heuristic

heuristic)" warnings and your stack (

Groq Llama 4 Maverick

Groq Llama 4 Maverick

Groq Llama 4 Maverick

Groq Llama 4 Maverick,

Vapi Voice

Vapi Voice

Vapi Voice

Vapi Voice,

Azure STT

Azure STT

Azure STT

Azure STT), could you confirm if your endpointing configuration recently changed (e.g., switching from smart to heuristic or adjusting timeout values)? Even small changes here can cause unpredictable latency spikes.

If possible, please check your current endpointing settings in the Vapi dashboard and let me know if any recent adjustments were made.

Sources:

G

generous-coffeeOP•11/5/25, 8:48 AM

No, I have not changed anything since the last time it was working well. I attach the photo of my configuration, so the Smart Endpointing is set to Vapi, as it was previously.

G

generous-coffeeOP•11/5/25, 8:48 AM

@Duckie

V

Vapi Support BotAPP•11/5/25, 8:49 AM

To address the latency caused by endpointing timeouts, try switching your smart endpointing provider to LiveKit (recommended for English calls). Update your configuration as follows:

{
  "startSpeakingPlan": {
    "smartEndpointingPlan": {
      "provider": "livekit",
      "waitFunction": "2000 / (1 + exp(-10 * (x - 0.5)))"
    },
    "waitSeconds": 0.4
  }
}

{
  "startSpeakingPlan": {
    "smartEndpointingPlan": {
      "provider": "livekit",
      "waitFunction": "2000 / (1 + exp(-10 * (x - 0.5)))"
    },
    "waitSeconds": 0.4
  }
}

{
  "startSpeakingPlan": {
    "smartEndpointingPlan": {
      "provider": "livekit",
      "waitFunction": "2000 / (1 + exp(-10 * (x - 0.5)))"
    },
    "waitSeconds": 0.4
  }
}

{
  "startSpeakingPlan": {
    "smartEndpointingPlan": {
      "provider": "livekit",
      "waitFunction": "2000 / (1 + exp(-10 * (x - 0.5)))"
    },
    "waitSeconds": 0.4
  }
}

This should restore fast and accurate endpointing, reducing latency.
For more details, see the Voice pipeline configuration documentation.

Source:

Voice pipeline configuration

G

generous-coffeeOP•11/5/25, 9:49 AM

But I am not using English, this assistant speaks in Polish so I can't use LiveKit @Duckie

G

generous-coffeeOP•11/5/25, 9:49 AM

@Sahil can you please help?

V

Vapi Support BotAPP•11/5/25, 9:51 AM

Since your assistant speaks Polish and LiveKit is not supported, try switching to transcription-based endpointing for better latency and reliability. Update your configuration as follows:

{
  "startSpeakingPlan": {
    "waitSeconds": 0.4,
    "transcriptionEndpointingPlan": {
      "onPunctuationSeconds": 0.1,
      "onNoPunctuationSeconds": 1.5,
      "onNumberSeconds": 0.5
    }
  }
}

{
  "startSpeakingPlan": {
    "waitSeconds": 0.4,
    "transcriptionEndpointingPlan": {
      "onPunctuationSeconds": 0.1,
      "onNoPunctuationSeconds": 1.5,
      "onNumberSeconds": 0.5
    }
  }
}

{
  "startSpeakingPlan": {
    "waitSeconds": 0.4,
    "transcriptionEndpointingPlan": {
      "onPunctuationSeconds": 0.1,
      "onNoPunctuationSeconds": 1.5,
      "onNumberSeconds": 0.5
    }
  }
}

{
  "startSpeakingPlan": {
    "waitSeconds": 0.4,
    "transcriptionEndpointingPlan": {
      "onPunctuationSeconds": 0.1,
      "onNoPunctuationSeconds": 1.5,
      "onNumberSeconds": 0.5
    }
  }
}

This setup is optimized for non-English conversations and should help reduce endpointing timeouts and latency. For more details, see the Voice pipeline configuration documentation.

Source:

Voice pipeline configuration

G

generous-coffeeOP•11/5/25, 10:34 AM

I did exactly that, the latency seems to be smaller now, but in call logs I sill see the same warnings, so I assume it wasn't the case of endpointing. Maybe it is groq's fault? The call id 019a537e-88c8-7006-984c-ae9ef98921bf @Duckie

V

Vapi Support BotAPP•11/5/25, 10:35 AM

To check if Groq is causing the latency, use the Vapi dashboard to review API logs for your call (e.g.,

019a537e-88c8-7006-984c-ae9ef98921bf

019a537e-88c8-7006-984c-ae9ef98921bf

019a537e-88c8-7006-984c-ae9ef98921bf

019a537e-88c8-7006-984c-ae9ef98921bf). Look for slow response times from the LLM provider. If you see delays, visit Groq's status page to check for ongoing issues. This will help you confirm if the latency is related to Groq or another part of your stack.
For more details, see the debugging guide.

Source:

Debugging voice agents

G

generous-coffeeOP•11/5/25, 11:17 AM

there are no api logs for the time of this call. The call 019a5083-e530-7ff9-b893-07aed41e4bdf happened at 3:15am ICT on 5th of Nov. Can someone else help me besides @Duckie ?
Long latency is annoying and unacceptable and if there is a way I can fix that, or models that cause that, then I want to know about that, please help me @Sahil

V

Vapi Support BotAPP•11/5/25, 11:19 AM

Since there are no API logs for your call, the best next step is to report this issue directly to Vapi support with the call ID, timestamp, and a description of the missing logs and latency. Use this format:

Call ID: 019a5083-e530-7ff9-b893-07aed41e4bdf
Timestamp: 3:15am ICT, 5th Nov
Issue: Long latency (~5s) and missing API logs
Expected: Fast response and visible API logs for LLM calls

Call ID: 019a5083-e530-7ff9-b893-07aed41e4bdf
Timestamp: 3:15am ICT, 5th Nov
Issue: Long latency (~5s) and missing API logs
Expected: Fast response and visible API logs for LLM calls

Call ID: 019a5083-e530-7ff9-b893-07aed41e4bdf
Timestamp: 3:15am ICT, 5th Nov
Issue: Long latency (~5s) and missing API logs
Expected: Fast response and visible API logs for LLM calls

Call ID: 019a5083-e530-7ff9-b893-07aed41e4bdf
Timestamp: 3:15am ICT, 5th Nov
Issue: Long latency (~5s) and missing API logs
Expected: Fast response and visible API logs for LLM calls

This will help the support team investigate backend or provider issues that aren't visible in the dashboard. For more, see the issue reporting guide.

Sources:

G

generous-coffeeOP•11/5/25, 3:04 PM

What in the world is happening with the latency? It's getting worse and worse.
Please take a look at this call id: 019a5487-2510-7885-ba45-68e21d07d695
Last messages it's honestly horrible latency. Like around 10s. I've emailed support but it will take forever for them to reply.
Can anyone beside bot help me? @Vapi Team

G

generous-coffeeOP•11/6/25, 6:20 AM

@Vapi Moderator ?

K

KyleAPP•11/6/25, 11:29 PM

Hi Marcello,

Thanks for your patience while we dug into the logs.

After reviewing the metrics for your recent calls, here’s what we found:

Latency Breakdown (Averages)

**Turn latency:** 5023.71 ms
**Model (LLM) latency:** **1862.14 ms**
**Voice latency:** 225.29 ms
**Endpointing latency:** 300.86 ms
**Transcriber latency:** 136 ms

Root Cause

The largest contributor to your delays is the LLM itself. When we dug deeper into the call logs, we found multiple warnings like this:

🔵 20:18:27:016 [user WARN] Model request failed (attempt #1, reason: meta-llama/llama-4-maverick-17b-128e-instruct is currently over capacity. Please try again and back off exponentially. Visit https://groqstatus.com to see if there is an active incident.) (provider: groq, model: meta-llama/llama-4-maverick-17b-128e-instruct, region: unknown, credential: true)

🔵 20:18:27:016 [user WARN] Model request failed (attempt #1, reason: meta-llama/llama-4-maverick-17b-128e-instruct is currently over capacity. Please try again and back off exponentially. Visit https://groqstatus.com to see if there is an active incident.) (provider: groq, model: meta-llama/llama-4-maverick-17b-128e-instruct, region: unknown, credential: true)

🔵 20:18:27:016 [user WARN] Model request failed (attempt #1, reason: meta-llama/llama-4-maverick-17b-128e-instruct is currently over capacity. Please try again and back off exponentially. Visit https://groqstatus.com to see if there is an active incident.) (provider: groq, model: meta-llama/llama-4-maverick-17b-128e-instruct, region: unknown, credential: true)

🔵 20:18:27:016 [user WARN] Model request failed (attempt #1, reason: meta-llama/llama-4-maverick-17b-128e-instruct is currently over capacity. Please try again and back off exponentially. Visit https://groqstatus.com to see if there is an active incident.) (provider: groq, model: meta-llama/llama-4-maverick-17b-128e-instruct, region: unknown, credential: true)

This means Groq was over capacity during your calls. When this happens, requests will retry and back off, which directly causes the ~5s+ delays you observed.

Summary

The issue is **not** with Vapi, endpointing, STT, or TTS.
The latency is **almost entirely from Groq’s LLM having capacity issues** at that time.
Everything else in your pipeline is running with very low latency.

If you have any other questions or want recommendations for more stable LLM providers, feel free to let us know—we’re happy to help.

Best,
Kyle at Vapi

Huge Latency ~5s

Similar Threads

Similar Threads

Similar Threads