VAPI•9mo ago

Assistant keeps talking over itself and isn't using the KB

We have an assistant that has a knowledge base using the query tool but it keeps talking over itself and doesn't seem to be using the KB unless we explicitly tell it to use the kb.

Call ID:
e3491c86-8635-4286-bb1b-dba033814e47

Vapi Support BotAPP•4/22/25, 5:38 PM

Hi there!

I see you're having issues with your assistant talking over itself and not using the knowledge base properly.

Unfortunately, I can't diagnose the exact issue with your specific call ID without access to the detailed logs. However, I can definitely help with general troubleshooting steps for both issues you're experiencing.

Why your assistant might not be using the KB automatically

According to the knowledge base documentation, there are several reasons why your assistant might not be using the KB unless explicitly told to:

System Prompt Configuration: Your assistant may need explicit instructions in the system prompt to utilize the KB. Try customizing your system prompt with something like: "You are a smart assistant who responds to user queries using the information from your KnowledgeBase. If the answer is not in the KnowledgeBase say 'I do not know'."
Knowledge Base Association: After the migration from Canonical to Trieve, you need to attach the Knowledge Base using the API. Check if your KB is properly linked to the assistant with the correct knowledgeBaseIdknowledgeBaseId.
API Key Scope: The issue might be related to an incorrect API key scope. Try creating a new API key with the proper role and scope for the dataset in context.
Configuration Parameters: Verify your KB configuration parameters like topKtopK, fileIdsfileIds, and maxTokensmaxTokens are correctly set.

## For the "talking over itself" issue

Based on the troubleshooting information I found, your assistant might be talking over itself due to:

Voice Echos: If there are voice echos in the audio background, the AI might keep speaking to itself.
Endpointing Settings: Aggressive endpointing settings can cause repetition. Try increasing the endpointing value to 300 and setting waitSecondswaitSeconds to 0.8.

Vapi Support BotAPP•4/22/25, 5:38 PM

Model Temperature: A high model temperature (close to 1) can lead to hallucinations. Consider adjusting your model temperature to a range between 0 and 0.3.
Assistant Hearing Itself: The assistant might be picking up its own audio output and re-processing it as input, especially if the end user has their phone on speakerphone mode at high volume.
System Messages Timing: System messages sent to the LLM immediately can interrupt the current output. Try adding system messages to the conversation context but wait until there's a new turn before sending them to the LLM.

You might want to try implementing the assistantOverrides.maxDurationSecondsassistantOverrides.maxDurationSeconds parameter or custom endpointing rules to prevent interruptions.

Would you like me to provide more specific configuration examples for either of these issues?

Sources:

⤵️ Mention @Vapi Support Bot for follow-ups 🐥

faint-whiteOP•4/22/25, 5:42 PM

@Shubham Bajaj

extreme-purple•4/22/25, 5:48 PM

To help track down this issue, could you share one or two:

When exactly this happened (the timestamp)
What response you expected to get
What response you actually got instead
This would really help us figure out what went wrong!

faint-whiteOP•4/22/25, 6:09 PM

Timestamp of When the Call Started: 1745343200907
What Response I Expected: An answer saying the email of our client
What Respone I Got: The assistant talking over itself so we didn't understand its answer at all

Here's how it usually responds to queries during our latest calls:

faint-whiteOP•4/22/25, 6:10 PM

I also sent over the call id in my post but just sending this again in case you need it @Kings_big

:
Call ID:
e3491c86-8635-4286-bb1b-dba033814e47

Please let me know if you need any other information, thanks!

extreme-purple•4/22/25, 6:47 PM

Looking into it

faint-whiteOP•4/23/25, 12:31 PM

Hi @Kings_big

please let us know if there are any updates. Thank you!

faint-whiteOP•4/23/25, 5:57 PM

Hello @Kings_big

just following up on this since we're still encoutering the issue. Thanks!

faint-whiteOP•4/23/25, 6:03 PM

faint-whiteOP•4/23/25, 6:03 PM

Call Id: 231611cc-1a6d-426f-922c-c29332699263

extreme-purple•4/23/25, 8:30 PM

You're using a tool of type function, you will have to use the query tool; also check the Knowledge Base docs and recreate the tool, and it will work

faint-whiteOP•4/24/25, 1:17 PM

Hi @Kings_big

we have tried that and we followed the knowledge base docs and it is still doing the issue of it being very confusing. It answers the user's query then proceeds to say something else completely different. Call ID: 920fd58e-7724-44d2-bb33-463e40e65d2a

faint-whiteOP•4/24/25, 1:19 PM

faint-whiteOP•4/24/25, 1:20 PM

It already gave the email address but then proceeds to say that it can't give it to me

extreme-purple•4/24/25, 3:57 PM

There is an ongoing issue on our side where llm1, which invokes the KB tool, also generates the response, and llm2, which generates the response based on KB chunks. In your case, the llm1 has responded with a made-up email, and the llm2, based on the KB chunks retrieved, says it's after office hours. The second answer came based on your KB chunks. For your reference, I am sharing the request sent.

You need to reduce the number of files related to the KB because of huge chunks of file, the response is not being generated properly.

extreme-purple•4/24/25, 3:57 PM

https://cdn.discordapp.com/attachments/1359772599940550798/1364992395997085777/message.txt?ex=680bafbb&is=680a5e3b&hm=0cc597adb4d8c8119cc1e10bf712f584ea87b47b0161752cc17fcfda54ca178d&

faint-whiteOP•4/24/25, 4:13 PM

Thanks, got it. When you fix the ongoing issue on your end, will all of this work correctly again without us having to reduce the number of files related to the kb? @Kings_big

Ffaint-white Thanks, got it. When you fix the ongoing issue on your end, will all of this wor...

extreme-purple•4/24/25, 4:31 PM

Sure

faint-whiteOP•4/25/25, 12:56 PM

Thanks @Kings_big

. Please let us know when the issue has been resolved.

faint-whiteOP•4/29/25, 1:01 PM

Hi @Kings_big

@Shubham Bajaj @Sahil are there updates about this issue? Thank you!

extreme-purple•4/29/25, 2:17 PM

Not resolve yet

faint-whiteOP•5/6/25, 5:06 PM

Has this been fixed yet? @Kings_big

dangerous-fuchsia•5/6/25, 7:29 PM

@sj The issue has been resolved. Can you try one more call now?

faint-whiteOP•5/7/25, 5:48 PM

Seems fixed, thank you!

VapiAPP•5/8/25, 2:41 AM

Marking this ticket as Solved

Assistant keeps talking over itself and isn't using the KB

Why your assistant might not be using the KB automatically

Similar Threads

Similar Threads

Why your assistant might not be using the KB automatically

Similar Threads