eager-peach•17mo ago

Supporting Multiple Languages

Hi is is possible to have a agent that supports both english and spanish simultaniously? The docs seem to indicate so but the transcriber requires that we hardcode the input language and i cant seem to get it to switch between languages . Any insights on this are appreciated .

9 Replies

Ask Inkeep•17mo ago

Hi @Papi, I'm an AI assistant for Vapi. I'll analyze available sources and get back to you shortly. ----------------------------- While I generate an answer, here are some sources that might be helpful: * Introduction * Outbound Calling * Inbound Calling * Web Calling ----------------------------- Answer Yes, Vapi supports multilingual capabilities, allowing you to set up an agent that can understand and respond in multiple languages. Multilingual support is primarily facilitated through the transcribers and text-to-speech (TTS) providers used in the voice pipeline. Here's how you can set up multilingual support in your assistant: 1. Transcriber (Speech-to-Text): Vapi supports providers like Deepgram and Talkscriber, which offer models for various languages. You can configure different transcribers for different languages as needed (1) . 2. Voice (Text-to-Speech): For TTS, you can choose from providers like PlayHT, 11labs, Azure, and others that support multiple languages. For example, Azure supports approximately 400 prebuilt voices across 140 languages and variants. Here is an example configuration for an assistant that supports both English and Spanish:

{
  "voice": {
    "provider": "azure",
    "voiceId": "en-US-JennyNeural"
  },
  "transcriber": {
    "provider": "deepgram",
    "model": "nova-2",
    "language": "en"
  },
  "model": {
    "provider": "openai",
    "model": "gpt-4"
  }
}

{
  "voice": {
    "provider": "azure",
    "voiceId": "es-ES-ElviraNeural"
  },
  "transcriber": {
    "provider": "deepgram",
    "model": "nova-2",
    "language": "es"
  },
  "model": {
    "provider": "openai",
    "model": "gpt-4"
  }
}

{
  "voice": {
    "provider": "azure",
    "voiceId": "en-US-JennyNeural"
  },
  "transcriber": {
    "provider": "deepgram",
    "model": "nova-2",
    "language": "en"
  },
  "model": {
    "provider": "openai",
    "model": "gpt-4"
  }
}

{
  "voice": {
    "provider": "azure",
    "voiceId": "es-ES-ElviraNeural"
  },
  "transcriber": {
    "provider": "deepgram",
    "model": "nova-2",
    "language": "es"
  },
  "model": {
    "provider": "openai",
    "model": "gpt-4"
  }
}

Ask Inkeep•17mo ago

For dynamically switching languages based on user input, ensure your application logic can handle checking for the user's language preference and then selecting the appropriate assistant configuration. If you face issues with transcribers reverting to English after publishing, it might be a known issue that has been addressed. Ensure your settings are saved correctly and try refreshing your setup (2) , (3) .

Sources [1] Introduction [2] Transcriber not working for languages other than English [3] Transcriber reverting back to EN after change and publish

Ask Inkeep•17mo ago

Got it. Tagging @Vapi Team for additional help.

eager-peachOP•17mo ago

I'd be using vapi with a twilio number so its not as simple as choosing betweent eh right assientant

relaxed-coral•16mo ago

Currently the thing that you required which is multilingual is in WIP.

adverse-sapphire•16mo ago

@Sahil What is the WIP? I am also trying to get this to work. I have a call 7aa8e599-1cfa-47b5-87aa-e545b307c9ae where a user starts speaking in Spanish but the assistent does not respond or transcibe.

relaxed-coral•16mo ago

Work in Progress.

adverse-sapphire•16mo ago

Ah thank you and good to know

eager-peachOP•16mo ago

FYI Google speech to text supports language auto detect

Supporting Multiple Languages

Did you find this page helpful?