metropolitan-bronze•6mo ago

Best LLM for using query tool?

Hi guys! Trying to pick the best LLM for my assistance, balancing between productivity and low latency, I've understood, that I need to pick GPT 4o (latest) or gpt-4o-mini (and the latest date). But I'd like to ask: What is the best model to use tools? I need only one for now (query tool, knowledge base). When I use 4o-mini, it seems that it doesn't check the knowledge base. Check this call id, messages page (2acc3409-11b8-4283-a898-2cdd11918f4a). There is no tool calls there. When I use 4o latest, I can see this: { "role": "tool_calls", "time": 1749747361028, "message": "", "toolCalls": [ { "id": "call_Lm4A5kF9FLDVi1Zsg7NHmZGX", "type": "function", "function": { "name": "defaultQueryTool", "arguments": "{\n "knowledgeBaseNames": ["Dental_Supply_information"]\n}" } } ], "secondsFromStart": 20.704 } but there is a problem, that after calling this tool, very often he stoppes and doesn't talk until I'll say something. So, I like more 4o-mini, and I'll use it, but only if he can call tools and read in knowledge base Thanks!!

5 Replies

Vapi Support Bot•6mo ago

For reliable Query Tool (knowledge base) usage in Vapi, the recommended model is gpt-4o (latest).
To ensure tool calls work, configure your assistant like this:

"model": {
  "provider": "openai",
  "model": "gpt-4o",
  "toolIds": ["<your-query-tool-id>"]
}

"model": {
  "provider": "openai",
  "model": "gpt-4o",
  "toolIds": ["<your-query-tool-id>"]
}

Using gpt-4o-mini is not documented to support tool calls or knowledge base access at this time.
For best results, stick with gpt-4o for tool/knowledge base integration. Sources: - Using the Query Tool for Knowledge Bases - Vapi API Reference: Assistants

⤵️ Mention @Vapi Support Bot for follow-ups 🐥

metropolitan-bronzeOP•6mo ago

Sorry, forgot to mention, that maybe there is a problem with retrieving info from knowledge base, because I've just seen a post about "success" posted by @jack-lynq.ai. I have this thing too

metropolitan-bronzeOP•6mo ago

metropolitan-bronzeOP•6mo ago

@Vapi Team

Kyle•6mo ago

Hey! To help track down this issue, could you share: - The call ID - When exactly this happened (the timestamp) - What response you expected to get - What response you actually got instead
This would really help us figure out what went wrong!

Best LLM for using query tool?

Did you find this page helpful?