VAPI•2y ago

Supporting Multiple Languages

Hi is is possible to have a agent that supports both english and spanish simultaniously? The docs seem to indicate so but the transcriber requires that we hardcode the input language and i cant seem to get it to switch between languages . Any insights on this are appreciated .

Ask Inkeep•5/16/24, 1:56 AM

Hi @Papi, I'm an AI assistant for Vapi. I'll analyze available sources and get back to you shortly.

Ask Inkeep•5/16/24, 1:56 AM

-----------------------------
While I generate an answer, here are some sources that might be helpful:

Introduction
Outbound Calling
Inbound Calling
Web Calling
-----------------------------

Ask Inkeep•5/16/24, 1:56 AM

Answer

Yes, Vapi supports multilingual capabilities, allowing you to set up an agent that can understand and respond in multiple languages. Multilingual support is primarily facilitated through the transcribers and text-to-speech (TTS) providers used in the voice pipeline.

Here's how you can set up multilingual support in your assistant:

Transcriber (Speech-to-Text): Vapi supports providers like Deepgram and Talkscriber, which offer models for various languages. You can configure different transcribers for different languages as needed (1) .
Voice (Text-to-Speech): For TTS, you can choose from providers like PlayHT, 11labs, Azure, and others that support multiple languages. For example, Azure supports approximately 400 prebuilt voices across 140 languages and variants.

Here is an example configuration for an assistant that supports both English and Spanish:

{
  "voice": {
    "provider": "azure",
    "voiceId": "en-US-JennyNeural"
  },
  "transcriber": {
    "provider": "deepgram",
    "model": "nova-2",
    "language": "en"
  },
  "model": {
    "provider": "openai",
    "model": "gpt-4"
  }
}

{
  "voice": {
    "provider": "azure",
    "voiceId": "es-ES-ElviraNeural"
  },
  "transcriber": {
    "provider": "deepgram",
    "model": "nova-2",
    "language": "es"
  },
  "model": {
    "provider": "openai",
    "model": "gpt-4"
  }
}

{
  "voice": {
    "provider": "azure",
    "voiceId": "en-US-JennyNeural"
  },
  "transcriber": {
    "provider": "deepgram",
    "model": "nova-2",
    "language": "en"
  },
  "model": {
    "provider": "openai",
    "model": "gpt-4"
  }
}

{
  "voice": {
    "provider": "azure",
    "voiceId": "es-ES-ElviraNeural"
  },
  "transcriber": {
    "provider": "deepgram",
    "model": "nova-2",
    "language": "es"
  },
  "model": {
    "provider": "openai",
    "model": "gpt-4"
  }
}

Ask Inkeep•5/16/24, 1:56 AM

For dynamically switching languages based on user input, ensure your application logic can handle checking for the user's language preference and then selecting the appropriate assistant configuration.

If you face issues with transcribers reverting to English after publishing, it might be a known issue that has been addressed. Ensure your settings are saved correctly and try refreshing your setup supportTranscriber not working for languages other than English , supportTranscriber reverting back to EN after change and publish .

Sources
[1] Introduction
[2] Transcriber not working for languages other than English
[3] Transcriber reverting back to EN after change and publish

AAsk Inkeep For dynamically switching languages based on user input, ensure your application...

Ask InkeepAPP•5/16/24, 1:57 AM

Got it. Tagging @Vapi Team for additional help.

progressive-amaranthOP•5/16/24, 1:57 AM

I'd be using vapi with a twilio number so its not as simple as choosing betweent eh right assientant

living-lavender•5/16/24, 10:27 AM

Currently the thing that you required which is multilingual is in WIP.

technological-jade•5/16/24, 2:34 PM

@Sahil What is the WIP? I am also trying to get this to work. I have a call 7aa8e599-1cfa-47b5-87aa-e545b307c9ae where a user starts speaking in Spanish but the assistent does not respond or transcibe.

living-lavender•5/16/24, 3:06 PM

Work in Progress.

Lliving-lavender Work in Progress.

technological-jade•5/16/24, 3:49 PM

Ah thank you and good to know

Lliving-lavender Currently the thing that you required which is multilingual is in WIP.

progressive-amaranthOP•5/16/24, 5:29 PM

FYI Google speech to text supports language auto detect

{ "voice": { "provider": "azure", "voiceId": "en-US-JennyNeural" }, "transcriber": { "provider": "deepgram", "model": "nova-2", "language": "en" }, "model": { "provider": "openai", "model": "gpt-4" } } { "voice": { "provider": "azure", "voiceId": "es-ES-ElviraNeural" }, "transcriber": { "provider": "deepgram", "model": "nova-2", "language": "es" }, "model": { "provider": "openai", "model": "gpt-4" } }

Supporting Multiple Languages

Answer

Similar Threads

Similar Threads

Answer

Similar Threads