After uploading a document to the knowledge base, the AI voice chat does not recognize

ltd-scarlet · 2025-05-05T12:03:59.432Z

Call ID : 4be474af-a9ec-4ef2-897f-4b9a20102a64 Timestamp: March 25, 2025, 05:20 Expected Response: I expected the assistant to provide accurate information based on the content I had already uploaded to the knowledge base. Actual Response: The AI voice chat provided answers that were not based on the uploaded document. Issue Summary: After uploading a document to the knowledge base, the AI voice chat does not reference or retrieve information from it. The document uploads successfully, but the AI seems unable to recognize or use its contents. The AI voice chat continues to give responses that are not aligned with the information in the uploaded document.

V

Vapi Support BotAPP•5/5/25, 12:05 PM

Vapi has migrated from Canonical to Trieve, so you need to attach your knowledge base using the API instead of the dashboard. Here's how to connect your document:

curl --request PATCH 'https://api.vapi.ai/assistant/<ASSISTANT_ID>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <YOUR_API_KEY>' \
--data '{
  "model": {
    "knowledgeBaseId": "<KNOWLEDGE_BASE_ID>",
    "provider": "openai"
  }
}'

curl --request PATCH 'https://api.vapi.ai/assistant/<ASSISTANT_ID>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <YOUR_API_KEY>' \
--data '{
  "model": {
    "knowledgeBaseId": "<KNOWLEDGE_BASE_ID>",
    "provider": "openai"
  }
}'

curl --request PATCH 'https://api.vapi.ai/assistant/<ASSISTANT_ID>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <YOUR_API_KEY>' \
--data '{
  "model": {
    "knowledgeBaseId": "<KNOWLEDGE_BASE_ID>",
    "provider": "openai"
  }
}'

curl --request PATCH 'https://api.vapi.ai/assistant/<ASSISTANT_ID>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <YOUR_API_KEY>' \
--data '{
  "model": {
    "knowledgeBaseId": "<KNOWLEDGE_BASE_ID>",
    "provider": "openai"
  }
}'

For detailed guidance, check the documentation at https://docs.vapi.ai/knowledgebase.

Sources:

⤵️ Mention @Vapi Support Bot for follow-ups 🐥

E

extreme-purple•5/5/25, 12:11 PM

Ensure the document is uploaded and linked to the assistant, publish the changes, check the file format and size, review the assistant’s configuration, and test by asking relevant questions.

L

ltd-scarletOP•5/6/25, 7:00 AM

@Kings_big

still not working

E

extreme-purple•5/6/25, 3:39 PM

Uploaded document is processed, under 300KB

https://docs.vapi.ai/api-reference/files/create

Upload File | Vapi

E

efficient-indigo•5/8/25, 10:16 AM

@Kings_big

Hello, please let me know if I should be creating a new thread instead of messaging here

Can I verify if the support bot is correct - Uploading the file to the dashboard doesn't work, and it needs to be uploaded via the API?

V

VapiAPP•5/8/25, 11:10 AM

Make sure you've linked the document to your assistant by navigating to Build > Assistant in your dashboard, and then selecting the knowledge base files you uploaded. After that, publish your assistant to activate changes. If it still doesn't work, double-check the format of your document and that it’s under the recommended size. If you're still having issues, let me know, and we can look into it further.

E

efficient-indigo•5/8/25, 12:50 PM

thanks @Kings_big

I'm using the web sdk with a dynamic assistant

o I'm uploading the file to the Dashboard, then getting the file id

And then vapiClient.start(assistantConfig)

With this format:

      {
        "type": "query",
        "function": {
          "name": "knowledge-base-query"
        },
        "knowledgeBases": [
          {
            "provider": "google",
            "model": "gemini-2.5-flash-preview-04-17",
            "name": "jp-knowledge-base",
            "description": "Use this knowledge base to draw from my knowledge, teaching, and stories",
            "fileIds": [
              "9db9757d-9d72-4120-a488-9031e6df7d7c"
            ]
          }
        ]
      }

      {
        "type": "query",
        "function": {
          "name": "knowledge-base-query"
        },
        "knowledgeBases": [
          {
            "provider": "google",
            "model": "gemini-2.5-flash-preview-04-17",
            "name": "jp-knowledge-base",
            "description": "Use this knowledge base to draw from my knowledge, teaching, and stories",
            "fileIds": [
              "9db9757d-9d72-4120-a488-9031e6df7d7c"
            ]
          }
        ]
      }

      {
        "type": "query",
        "function": {
          "name": "knowledge-base-query"
        },
        "knowledgeBases": [
          {
            "provider": "google",
            "model": "gemini-2.5-flash-preview-04-17",
            "name": "jp-knowledge-base",
            "description": "Use this knowledge base to draw from my knowledge, teaching, and stories",
            "fileIds": [
              "9db9757d-9d72-4120-a488-9031e6df7d7c"
            ]
          }
        ]
      }

      {
        "type": "query",
        "function": {
          "name": "knowledge-base-query"
        },
        "knowledgeBases": [
          {
            "provider": "google",
            "model": "gemini-2.5-flash-preview-04-17",
            "name": "jp-knowledge-base",
            "description": "Use this knowledge base to draw from my knowledge, teaching, and stories",
            "fileIds": [
              "9db9757d-9d72-4120-a488-9031e6df7d7c"
            ]
          }
        ]
      }

It worked when I tested it with a single line file, but when I uploaded a 40kb text file with about 7k words, it just kept hallucinating the answers. When I looked at the logs, I saw that it made a tool call, but I can't see what's returned in it so I can't tell if it's searching or returning effectively, or where the problem is

Call ID: 3cedaf89-7600-42b4-9a7c-5a6c2b2e7566

Can you assist please?

Thank you

V

VapiAPP•5/10/25, 11:40 PM

Hey Ethan! To help track down this issue, could you share:

When exactly this happened (the timestamp)
What response you expected to get
What response you actually got instead
This would really help us figure out what went wrong!

The KB is being used, as evidenced by the logs. Can you pinpoint the unexpected actions?

extract-2025-05-10T23_39_37.559Z.csv143KB

E

efficient-indigo•5/12/25, 10:37 PM

Thanks @Shubham Bajaj

Ok after more testing I see that it sometimes fetches info accurately, but it's very unclear to me what it's fetching, how much, etc

And often after fetching it doesn't speak properly, it gets cut off

Call ID: 79ac6228-3c10-4c17-b670-ab5a837ab3d8

{
  "role": "user",
  "time": 1747059043395,
  "endTime": 1747059046195,
  "message": "But is there something specific he says about judgments?",
  "duration": 2800,
  "secondsFromStart": 182.826
}
{
  "role": "tool_calls",
  "time": 1747059049770,
  "message": "",
  "toolCalls": [
    {
      "id": "call_7AXxJ1VxITmvsrCyHNitYc0z",
      "type": "function",
      "function": {
        "name": "knowledge-base-query",
        "arguments": "{\"knowledgeBaseNames\": [\"jp-knowledge-base\"]}"
      }
    }
  ],
  "secondsFromStart": 184.32
}
{
  "name": "knowledge-base-query",
  "role": "tool_call_result",
  "time": 1747059049773,
  "result": "Success.",
  "toolCallId": "call_7AXxJ1VxITmvsrCyHNitYc0z",
  "secondsFromStart": 184.323
}

{
  "role": "user",
  "time": 1747059043395,
  "endTime": 1747059046195,
  "message": "But is there something specific he says about judgments?",
  "duration": 2800,
  "secondsFromStart": 182.826
}
{
  "role": "tool_calls",
  "time": 1747059049770,
  "message": "",
  "toolCalls": [
    {
      "id": "call_7AXxJ1VxITmvsrCyHNitYc0z",
      "type": "function",
      "function": {
        "name": "knowledge-base-query",
        "arguments": "{\"knowledgeBaseNames\": [\"jp-knowledge-base\"]}"
      }
    }
  ],
  "secondsFromStart": 184.32
}
{
  "name": "knowledge-base-query",
  "role": "tool_call_result",
  "time": 1747059049773,
  "result": "Success.",
  "toolCallId": "call_7AXxJ1VxITmvsrCyHNitYc0z",
  "secondsFromStart": 184.323
}

{
  "role": "user",
  "time": 1747059043395,
  "endTime": 1747059046195,
  "message": "But is there something specific he says about judgments?",
  "duration": 2800,
  "secondsFromStart": 182.826
}
{
  "role": "tool_calls",
  "time": 1747059049770,
  "message": "",
  "toolCalls": [
    {
      "id": "call_7AXxJ1VxITmvsrCyHNitYc0z",
      "type": "function",
      "function": {
        "name": "knowledge-base-query",
        "arguments": "{\"knowledgeBaseNames\": [\"jp-knowledge-base\"]}"
      }
    }
  ],
  "secondsFromStart": 184.32
}
{
  "name": "knowledge-base-query",
  "role": "tool_call_result",
  "time": 1747059049773,
  "result": "Success.",
  "toolCallId": "call_7AXxJ1VxITmvsrCyHNitYc0z",
  "secondsFromStart": 184.323
}

{
  "role": "user",
  "time": 1747059043395,
  "endTime": 1747059046195,
  "message": "But is there something specific he says about judgments?",
  "duration": 2800,
  "secondsFromStart": 182.826
}
{
  "role": "tool_calls",
  "time": 1747059049770,
  "message": "",
  "toolCalls": [
    {
      "id": "call_7AXxJ1VxITmvsrCyHNitYc0z",
      "type": "function",
      "function": {
        "name": "knowledge-base-query",
        "arguments": "{\"knowledgeBaseNames\": [\"jp-knowledge-base\"]}"
      }
    }
  ],
  "secondsFromStart": 184.32
}
{
  "name": "knowledge-base-query",
  "role": "tool_call_result",
  "time": 1747059049773,
  "result": "Success.",
  "toolCallId": "call_7AXxJ1VxITmvsrCyHNitYc0z",
  "secondsFromStart": 184.323
}

PARTIAL RESPONSE

{
  "role": "bot",
  "time": 1747059054527,
  "source": "model",
  "endTime": 1747059055212,
  "message": "C17 (\"Upgrade Your InnerAh, judgements. Yeah. J P talks about judgements as the very bricks of our",
  "duration": 7988,
  "secondsFromStart": 189.107
}

{
  "role": "bot",
  "time": 1747059054527,
  "source": "model",
  "endTime": 1747059055212,
  "message": "C17 (\"Upgrade Your InnerAh, judgements. Yeah. J P talks about judgements as the very bricks of our",
  "duration": 7988,
  "secondsFromStart": 189.107
}

{
  "role": "bot",
  "time": 1747059054527,
  "source": "model",
  "endTime": 1747059055212,
  "message": "C17 (\"Upgrade Your InnerAh, judgements. Yeah. J P talks about judgements as the very bricks of our",
  "duration": 7988,
  "secondsFromStart": 189.107
}

{
  "role": "bot",
  "time": 1747059054527,
  "source": "model",
  "endTime": 1747059055212,
  "message": "C17 (\"Upgrade Your InnerAh, judgements. Yeah. J P talks about judgements as the very bricks of our",
  "duration": 7988,
  "secondsFromStart": 189.107
}

Then, a hallucinated response, doesn't use the KB's response

{
  "role": "user",
  "time": 1747059068595,
  "endTime": 1747059068995,
  "message": "Hello?",
  "duration": 400,
  "secondsFromStart": 205.698
}
{
  "role": "bot",
  "time": 1747059075376,
  "source": "model",
  "endTime": 1747059075550,
  "message": "I'm here, Ethan. Let me give you something straight from J P’s way of speaking:. He also says, “Judgments are like clouds in the sky—they pass. Freedom is remembering you’re the sky, not the clouds.”. How does that land for you? ",
  "duration": 37756,
  "secondsFromStart": 209.959
}

{
  "role": "user",
  "time": 1747059068595,
  "endTime": 1747059068995,
  "message": "Hello?",
  "duration": 400,
  "secondsFromStart": 205.698
}
{
  "role": "bot",
  "time": 1747059075376,
  "source": "model",
  "endTime": 1747059075550,
  "message": "I'm here, Ethan. Let me give you something straight from J P’s way of speaking:. He also says, “Judgments are like clouds in the sky—they pass. Freedom is remembering you’re the sky, not the clouds.”. How does that land for you? ",
  "duration": 37756,
  "secondsFromStart": 209.959
}

{
  "role": "user",
  "time": 1747059068595,
  "endTime": 1747059068995,
  "message": "Hello?",
  "duration": 400,
  "secondsFromStart": 205.698
}
{
  "role": "bot",
  "time": 1747059075376,
  "source": "model",
  "endTime": 1747059075550,
  "message": "I'm here, Ethan. Let me give you something straight from J P’s way of speaking:. He also says, “Judgments are like clouds in the sky—they pass. Freedom is remembering you’re the sky, not the clouds.”. How does that land for you? ",
  "duration": 37756,
  "secondsFromStart": 209.959
}

{
  "role": "user",
  "time": 1747059068595,
  "endTime": 1747059068995,
  "message": "Hello?",
  "duration": 400,
  "secondsFromStart": 205.698
}
{
  "role": "bot",
  "time": 1747059075376,
  "source": "model",
  "endTime": 1747059075550,
  "message": "I'm here, Ethan. Let me give you something straight from J P’s way of speaking:. He also says, “Judgments are like clouds in the sky—they pass. Freedom is remembering you’re the sky, not the clouds.”. How does that land for you? ",
  "duration": 37756,
  "secondsFromStart": 209.959
}

E

efficient-indigo•5/14/25, 10:21 PM

hi @Shubham Bajaj , following up on this

Can you shed any light on why it has partial responses, or responds without using the KB info, and is there any way to see what and how much info is returned from the KB? That will also help me to prompt it in the best way

V

VapiAPP•5/15/25, 2:27 AM

Hey,

Could you please schedule a brief call at your convenience so we can discuss this matter in detail?

Kindly use the following link to book a suitable time: https://calendly.com/shubham-vapi/chat-30m.

E

efficient-indigo•5/15/25, 8:36 AM

Kyle and @Shubham Bajaj , a call would be great, but June is the next availability, which is a long way away

In the meantime can you share a bit more about exactly how the KB function works?

I'm guessing the KB is sent to Gemini 2.5 Flash, but what else is sent, and what is the Gemini prompt please?

if you can share how it works, that will help me to structure the kb data and my assistant prompt to have it work better

V

VapiAPP•5/17/25, 4:02 AM

please share the call id and we will investigate the issue

After uploading a document to the knowledge base, the AI voice chat does not recognize

Similar Threads

Similar Threads

Similar Threads