VAPI•2y ago

Azure multilingual voices switching to random languages

Hi, i'm building a bot in French, for that purpose, I tried to use Azure multilingual voice. The voice is speaking pretty well, however, during the call, the bot has to enumerate a code, composed with digits and letters, and when it has to enumerate this code, the voice randomly switch to english, spanish, italian...

any idea about how could i fix it ? is there any better options for french voice than azure ? multilingual ?

Ask Inkeep•5/29/24, 12:23 PM

Hi @hugopdvn, I'm an AI assistant for Vapi. I'll analyze available sources and get back to you shortly.

Ask Inkeep•5/29/24, 12:23 PM

-----------------------------
While I generate an answer, here are some sources that might be helpful:

Introduction
Inbound Calling
Outbound Calling
Web Calling
-----------------------------

Ask Inkeep•5/29/24, 12:24 PM

Answer

The issue you're experiencing with Azure multilingual voices randomly switching languages during a call can sometimes occur due to how the speech synthesis model interprets the context or specific triggers in the text. This could be particularly noticeable when mixing languages or when there are numbers and letters involved that might be pronounced differently in different languages.

As for alternatives, you have a few options for French voice synthesis that might offer more consistent performance in multilingual settings:

Vapi with Deepgram and PlayHT: Vapi supports integration with multiple speech-to-text and text-to-speech providers. You could configure Vapi to use Deepgram for speech recognition and PlayHT for generating the spoken output. PlayHT offers a variety of voices and might provide a more consistent language output in French (1) .
Using a Different Provider in Vapi: You could also consider using other text-to-speech providers supported by Vapi such as 11labs or Rime-ai, which may have different capabilities and support for French. Each provider offers varying degrees of language support, and you might find one that handles mixed-language scenarios better .
Custom Configuration: Ensure that your language settings are explicitly configured to maintain French across all aspects of speech processing. This includes setting the transcriber, language model, and voice settings strictly to French. Here is an example of how you might set this configuration in Vapi:

{
  "transcriber": {
    "provider": "Deepgram",
    "model": "nova-2",
    "language": "fr-FR"
  },
  "voice": {
   "provider": "PlayHT",
    "voiceId": "some-french-voice-id"
  },
  "model": {
    "provider": "openai",
    "model": "gpt-4"
  }
}

{
  "transcriber": {
    "provider": "Deepgram",
    "model": "nova-2",
    "language": "fr-FR"
  },
  "voice": {
   "provider": "PlayHT",
    "voiceId": "some-french-voice-id"
  },
  "model": {
    "provider": "openai",
    "model": "gpt-4"
  }
}

Ask Inkeep•5/29/24, 12:24 PM

By explicitly setting each component to French, the likelihood of random switches to other languages may be reduced. Adjusting these configurations can often resolve issues with inconsistent language output during voice interactions.

Sources
[1] Introduction

living-lavender•5/29/24, 1:26 PM

11lab with multilingual is better

living-lavender•5/29/24, 1:26 PM

https://docs.vapi.ai/community/videos

Vapi

Community Videos - Vapi

Videos showcasing Vapi out in the wild.

living-lavender•5/29/24, 1:26 PM

Check community video you will get a video related to it.

Lliving-lavender 11lab with multilingual is better

military-pinkOP•5/29/24, 2:52 PM

i'll give it a try, thank you for your daily support @Sahil

AAsk Inkeep By explicitly setting each component to French, the likelihood of random switche...

Ask InkeepAPP•5/29/24, 3:19 PM

Glad I could be helpful. Feel free to create a new thread with any new questions you may have.

{ "transcriber": { "provider": "Deepgram", "model": "nova-2", "language": "fr-FR" }, "voice": { "provider": "PlayHT", "voiceId": "some-french-voice-id" }, "model": { "provider": "openai", "model": "gpt-4" } }

Azure multilingual voices switching to random languages

Answer

Similar Threads

Similar Threads

Answer

Similar Threads