VAPI•2y ago

Multiple responses being given at the same time, returning giberish in the output.

Hello, It seems I have been having a weird issue when on calls with the agent. Sometimes it generates random as a output giberish, and it seems like there is two LLMs giving two different outputs at the same time, and it just mashes them together for the output

I have temperature at 0, and the optimize streaming latency at 0 as well but it still happens, it was especially bad yesterday, any ideas what might be happening?
For example here are some logs from some calss:

AI: Hey there. Our training package costs 3900 dollars. Hey there. It Our includes up PTSD training package costs 3 dollars to 10 in 900. It's home 1 on includes up to 1 training sessions. 10 in Do You Home already have 1 on 1 training sessions? Or are you, there anything planning to get 1? Else you'd like to know about it?

This happens with multiple models, open AI, Claude, both gtp4 and gpt4o.

Ask Inkeep•5/31/24, 8:25 AM

Hi @Suzanna, I'm an AI assistant for Vapi. I'll analyze available sources and get back to you shortly.

Ask Inkeep•5/31/24, 8:25 AM

-----------------------------
While I generate an answer, here are some sources that might be helpful:

Orchestration Models
Web Calling
Inbound Calling
Billing Examples
-----------------------------

Ask Inkeep•5/31/24, 8:25 AM

Answer

It sounds like you're encountering issues with overlapping or garbled responses from the AI, which can be indicative of several potential issues in the configuration or integration of your voice AI setup with Vapi.

Model Configuration Issues: If multiple models are active or the system prompt within the models isn't correctly isolating its context, it could result in overlapping responses. Ensuring that the configuration aligns with a single task per call might mitigate this issue.
Streaming or Latency Issues: Even though you've optimized streaming latency, it's possible that network issues or server-side processing delays could be affecting performance. Ensuring that your server is properly scaled and your network is stable might help.
Audio Processing Anomalies: Issues with the audio input, such as noise or poor quality, could be misinterpreted by the transcriber, leading to erroneous text input to the models and thus confusing outputs. Checking the audio input quality could be a step towards a solution.
Software Bugs or Integration Issues: There could be a bug in the integration layer between Vapi and the underlying models, or potentially in how responses are handled and delivered back to the end-user. Reviewing logs and system metrics around the times these issues occur could provide insights.

Since this behavior could be due to a complex interaction of components, consider isolating each part of the system (transcription, LLM processing, and text-to-speech) to diagnose where the issue might be more pronounced. This can be done by logging the outputs at each stage to see where the anomalies start.

If the issue persists and is reproducible, contacting Vapi support with detailed logs and system configurations would be advisable to get more targeted assistance.

AAsk Inkeep ### Answer It sounds like you're encountering issues with overlapping or garbled...

Ask InkeepAPP•5/31/24, 8:26 AM

Got it. Tagging @Vapi Team for additional help.

wealthy-blueOP•5/31/24, 8:27 AM

This call log, clearly showcases this issue:
95575e3b-4968-428c-afcc-c74ce9545979

living-lavender•5/31/24, 8:31 AM

Can you remove your KB and try it once?

wealthy-blueOP•5/31/24, 8:32 AM

Sure, the thing is it doesnt happena all the time, but will give it a try.

living-lavender•5/31/24, 8:58 AM

Thanks!

wealthy-blueOP•6/5/24, 5:17 PM

I have given a test for the last few days, and removing rag does solve the issue. Is there anything we can do to still use rag and not get this issue?

Wwealthy-blue I have given a test for the last few days, and removing rag does solve the issue...

living-lavender•6/5/24, 5:33 PM

We are looking into it. So, just provide us some time until we fix it.

wealthy-blueOP•6/5/24, 5:34 PM

Sounds good thank u.

unlikely-moccasin•6/6/24, 12:15 AM

I am having the same issue. Removing the KB fixed it.

unlikely-moccasin•6/6/24, 12:20 AM

I had to create an entire new assistant though. Couldn't duplicate it either. Had to copy and paste all settings and prompt from scratch. I had a KB here named "knowledge base". Then after I deleted it, it turned into these random numbers and letters. I can't remove it even though no KB exist.

I tested this broken assistant with the kb and it started speaking gibberish. Then my new assistant I created (same prompt+settings but no kb) stopped doing that. I won't even be touching the kb button.