VAPI•3mo ago

Unexpected fallback to OpenAI model instead of Groq (llama3-70b-8192)

Hello,
We’re running multiple tests with different models, and so far the best results come from the Groq integration using the llama3-70b-8192 model.
The main issue is that sometimes the model works very well, but occasionally VAPI falls back to OpenAI’s gpt-4o-2024-08-06, which is significantly slower.
My questions are:
• Why is it calling OpenAI if this assistant is not configured for it?
• How can we define which model to fall back to?
• How can we set our Groq API key? There’s no option for that in the integration dashboard.
Call ID: 0199ced8-78e2-7661-8f2c-e5e56627f09a

Thanks in advance for your help.

Vapi Support BotAPP•10/13/25, 9:46 AM

To ensure your assistant uses the Groq model, specify the provider and model explicitly in your assistant configuration. For example:

{
  "model": {
    "provider": "groq",
    "model": "llama-3.3-70b-versatile"
  }
}

{
  "model": {
    "provider": "groq",
    "model": "llama-3.3-70b-versatile"
  }
}

Tip: Double-check your assistant's configuration to avoid unintended fallbacks. If you need to set an API key for Groq, please consult the latest dashboard or support, as the documentation does not specify this step.

Source:

Changelog: January 5, 2025

⤵️ Mention @Vapi Support Bot for follow-ups 🐥

scattered-tealOP•10/13/25, 9:52 AM

I did the suggested change by the bot, and still the same issue:
Call ID: 0199dcf9-948c-7eeb-a092-6f45ccfeb41c

scattered-tealOP•10/14/25, 6:47 AM

@VAPI Tech can you help with this?

scattered-tealOP•10/14/25, 6:47 AM

@Vapi can you help?

VapiAPP•10/15/25, 7:46 AM

Use the /credential API endpoint to add your groq API key:

curl -X POST https://api.vapi.ai/credential \
  -H "Authorization: Bearer YOUR_VAPI_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "groq",
    "apiKey": "gsk_YOUR_GROQ_API_KEY_HERE"
  }'

curl -X POST https://api.vapi.ai/credential \
  -H "Authorization: Bearer YOUR_VAPI_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "groq",
    "apiKey": "gsk_YOUR_GROQ_API_KEY_HERE"
  }'

The reason for the fallback is because the model request failed with groq for the tool call:

🔵 15:58:57:979 [user WARN] Model request failed (attempt #1, reason: tool call validation failed: parameters for tool get_next_step_alex did not match schema: errors: [`/provided_password`: expected string, but got null]) (provider: groq, model: llama-3.3-70b-versatile, region: unknown, credential: true)

🔵 15:58:57:979 [user WARN] Model request failed (attempt #1, reason: tool call validation failed: parameters for tool get_next_step_alex did not match schema: errors: [`/provided_password`: expected string, but got null]) (provider: groq, model: llama-3.3-70b-versatile, region: unknown, credential: true)

curl -X POST https://api.vapi.ai/credential \ -H "Authorization: Bearer YOUR_VAPI_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "provider": "groq", "apiKey": "gsk_YOUR_GROQ_API_KEY_HERE" }'

🔵 15:58:57:979 [user WARN] Model request failed (attempt #1, reason: tool call validation failed: parameters for tool get_next_step_alex did not match schema: errors: [`/provided_password`: expected string, but got null]) (provider: groq, model: llama-3.3-70b-versatile, region: unknown, credential: true)

Unexpected fallback to OpenAI model instead of Groq (llama3-70b-8192)

Similar Threads

Similar Threads

Similar Threads