like-gold•4mo ago

Diacritic Sign Support for Improved Voice Conversations

I am creating an assistant that speaks Hebrew. Hebrew is one of the languages that uses diacritic signs to vocalise words correctly. A well-known example of diacritic signs can be found in German, such as the umlaut letters. The most common umlauts are ä, ö, and ü, which represent modified vowel sounds compared to their non-umlaut counterparts (a, o, and u). Another notable language using diacritics is Arabic. We have tried instructing the LLM to return its results with diacritic signs in Hebrew so that the TTS produces better results. However, even when instructing the LLM to include full diacritics, I am unable to see them in the call log (neither in the messages tab nor the logs). Please see Call ID 5427f3af-25a6-496a-8f31-4c47ce89239f as an example (you can check this call’s system prompt on your side, which generates answers with diacritics). We also tried providing instructions to the LLM in Hebrew, but observed similar results, except for the system message inside the messages tab, which does show diacritic signs. See Call IDs ff27397f-0af8-4c51-a0cc-6f0f9fadcda7 and 50261828-071c-4a67-bca9-fb865596bfe5 with short and long prompts in Hebrew only. Importantly, when we instruct the LLM (e.g., via the ChatGPT Playground, official website, Azure Foundry Playground, etc.) directly, without Vapi involved, it returns answers with full diacritics. Could you please clarify if this is a limitation on the Vapi side—that is, are diacritic signs from the LLM not supported or passed through as expected? Or is there anything I can do to ensure the LLM passes TTS words with diacritic signs? Thanks in advance for your assistance.

13 Replies

Kyle Brunker•4mo ago

Disable voice formatting in your assistant config or use a custom TTS integration..

like-goldOP•4mo ago

I am not sure which voice formatting you mean to disable. I think the problem is that Vapi doesn't attach the diacritic signs to the LLM response as i don't see them inside the call log.

Kyle Brunker•3mo ago

Set voice.chunkPlan.formatPlan.enabled = false in your vapi assistant config to preserve Hebrew diacritics in both TTS and logs..

like-goldOP•3mo ago

I tried to set voice.chunkPlan.formatPlan.enabled = false (and also voice.chunkPlan.enabled = false, and even tried flase for both), but still the same! another idea?

Kyle Brunker•3mo ago

Set voice.inputPreprocessingEnabled = false and voice.chunkPlan.formatPlan.enabled = false then run a test call

like-goldOP•3mo ago

Hi Kyle. I tried with the settings you suggested, still the same results. see Call id 31c01290-2580-487c-9264-ab6beda364a3 as an example.

Kyle Brunker•3mo ago

Hey Amir76, You can make use of pronunciation dictionary to add certain pronunciations for certain key words. - The first link below teaches you how to create pronunciation dictionary using 11Labs - The second one guides you on how to add it to the VAPI Let me know if you require further help with this.

https://elevenlabs.io/docs/api-reference/pronunciation-dictionaries/create-from-rules

https://docs.vapi.ai/api-reference/provider/provider-resource-controller-create-provider-resource?explorer=true step-by-step instructions for using pronunciation dictionaries

1. Create a pronunciation dictionary using the API endpoint: POST /provider/11labs/pronunciation-dictionary This endpoint allows you to create a custom pronunciation dictionary with rules that define how specific words should be pronounced. 2. Add pronunciation rules to your dictionary: You can create rules that specify alternative pronunciations for words, either by providing phonetic spellings or by defining substitutions. 3. Get your dictionary ID: After creating the dictionary, note the returned dictionary ID - you'll need this to attach the dictionary to your assistant. 4. Attach the dictionary to your ElevenLabs voice configuration: When configuring your assistant's voice, add the dictionary ID to the pronunciationDictionaryIds array in your ElevenLabs voice settings. 5. Test your assistant: Make a test call to verify that your assistant now pronounces the specified words according to your dictionary rules. Important Notes - ElevenLabs Only: This feature is specifically for ElevenLabs voice provider (`"11labs"`). - Dictionary Management: You can create, update, get, and delete pronunciation dictionaries through the API endpoints. - Multiple Dictionaries: You can attach multiple pronunciation dictionaries to a single assistant. For detailed API documentation and examples, refer to the ElevenLabs pronunciation dictionary documentation at: https://elevenlabs.io/docs/speech-synthesis/pronunciation-dictionaries

like-goldOP•3mo ago

this is not what i am looking for. I am looking for something else - the option to ues discritic signs with Vapi

Kyle Brunker•3mo ago

Hey Amir! We're currently out at an offsite, but I'd love to circle back on this with you first thing Monday morning. Would that work for you?

like-goldOP•3mo ago

Yes. Tnx

Kyle Brunker•3mo ago

Hi Amir76, I've identified the issue with your Hebrew diacritics! You're very close - you set 3 out of 4 required flags, but there's one missing. The Issue: Our text formatting system requires all four flags to be disabled, but our support team only provided you with three. You're missing inputReformattingEnabled: false. The Complete Fix:

{
  "voice": {
    "inputPreprocessingEnabled": false,
    "inputReformattingEnabled": false,  // ← This was missing!
    "chunkPlan": {
      "enabled": false,
      "formatPlan": {
        "enabled": false
      }
    }
  }
}

{
  "voice": {
    "inputPreprocessingEnabled": false,
    "inputReformattingEnabled": false,  // ← This was missing!
    "chunkPlan": {
      "enabled": false,
      "formatPlan": {
        "enabled": false
      }
    }
  }
}

Without all four flags set to false, your Hebrew text still went through our text formatters, which stripped the diacritics. Your system message kept them because it bypasses this formatting pipeline, but assistant responses go through TTS formatting.

like-goldOP•3mo ago

Hi, i set all 4 flags to false as you suggested and i changed well-tested assistant system prompt to always response with diacritic signs. i still don't see the signs inside the call log messages tab and transcribe tab. i see few signs inside the logs tab but no together with the word itself but each sign seperated so i can't really use it for anything. see call ID 51f12b55-f856-478f-90ac-1bd06f81b5f0 as an example. Can u confirm if this is a bug on the VAPI dashboard only that i don't see the signs inside the messages and transcripts call log screens, and you see on your side that the LLM returned Hebrew with signs correctly on each bot response? I also noticed that during this call, once i heard those words in English :"What was that" for a reason i can't see and i never had such problems before the changes. see this line from the call log: "21:01:48:444 [LOG] Model output: What was that?". i couldn't find such bot response message inside the messages log nor in the transcripts of this call. I assume this is another problem that related to the flags i set or to the system prompt that i instructed to return all responses from the bot using diacritic signs. first let me know if you see the signs correctly on this call example, and then check the other small issue. Tnx I started to use Langfuse, and i see inside Langfuse logs that the LLM return diacritic signs even when i don't tell him to do it, and i see the signs have impact on the TTS voice so i don't see the reason to use your suggested flags. now i see another problem that sometimes happen mainly (or only) when the LLM return words with signs - the TTS voice add extra words that was not part of the LLM text response. for example see call ID 4dd34dad-3b23-452c-9891-50cdc555dee7 with the following bot message taken from Vapi call log: { "role": "bot", "time": 1753054278114, "source": "", "endTime": 1753054288590, "message": "מעולה לשמוע שאת מרגישה טוב היום האם תרצי לעשות ביחד אימון קצר למוח אתה הכימו?", "duration": 15900, "secondsFromStart": 15.9 }, but looking inside Langfuse the LLM correct message was: "מְעֻולֶּה לשמוע שאת מרגישה טוב היום. הַאִם תִּרְצִי לַעֲשׂוֹת בְּיַחַד אִמּוּן קָצָר לַמּוֹחַ?" for some unknown reason, VAPI added extra 2 words at the end of this message. in call ID 787e48b0-62de-4be8-b5fb-3687592aaf9c Vapi added extra word in the middle of this same sentence. Recap help i got from Arthur during his OH: we saw the extra words inside the voice are not arrived from Vapi but most likely the STT provider added them. i will continue investigating this problem with other providers if happen as well.

Kyle Brunker•3mo ago

Hi, First off, we want to sincerely apologize for the delay in getting back to you. We understand how frustrating it is to wait - especially when you're counting on us - and we owe you a clear explanation of what’s been happening and how we’re addressing it. Over the past few weeks, we've seen a significant increase in support requests. While this reflects exciting growth, it has also stretched our small team and exposed some real challenges in scaling our support operations. To improve your experience, we’ve taken a step back to reassess our approach. Here’s what we’re implementing: - Smarter Support Through Automation: We’re investing in our AI support systems to help you resolve issues more efficiently. Soon, our support bot will offer expanded capabilities, making it easier to access accurate, instant help—particularly for common or repetitive queries. - Expanding the Support Team: To meet growing demand, we’re adding 2–3 new team members focused on managing support volume and improving response times. - Prioritized SLAs for High-Usage Accounts: We’re introducing service level improvements for users who are growing with us: - Accounts with usage over 1,000 minutes/month will receive prioritized support. - For all general inquiries, we’re establishing a standardized 48-hour response time. We’re confident these steps will lead to faster, more reliable support and help us better serve you as you grow with us. Also, in case you still need help with this ongoing ticket, do let us know, and we will help you get this resolved as soon as possible. Thank you for your continued patience and for being part of our journey. Warm regards, Vapi Team

Diacritic Sign Support for Improved Voice Conversations

Did you find this page helpful?