Regarding Transcript, Pronunciation, Listening, and Key Terms
Hi Team , please check to below mentiioned queries .
1) Issue Passing Key Terms in VAPI for Multilingual Models
We are using the assistance feature in VAPI with multiple languages (English, Spanish, Hindi, etc.). However, we’re facing an issue while passing key terms:
The key terms work correctly only with model Nova 3.
Unfortunately, Spanish and Hindi are not supported in Nova 3.
When we switch to Nova 2 (to support these languages) and try to pass the key terms, we receive a 400 Bad Request error.
2) Transcript Accuracy Issue
The transcripts are being returned inconsistently. When the AI agent speaks, the generated transcript does not accurately match the spoken words.
Example:
Actual spoken: “I am working as an Associate Developer in this company.”
Transcript generated: “Working associate in company.”
3) Agent Not Responding to Single-Word Inputs
When short responses such as “Yes,” “No,” or “Okay” are spoken, the AI agent often fails to recognize or respond to them on the first attempt. We have to repeat the word louder or multiple times before the agent responds.
4) Agent Mispronounces and Misspells Spelled-Out Words
When we spell out words (for example, P R A D E E P), the AI agent sometimes mispronounces or returns incorrect transcripts.
Example:
Spoken/Spelled: P R A D E E P
Transcript returned: PRADIP / PRADEP
5) Language Handling and Translation Requirement
When the Spanish language is selected, the call should be conducted entirely in Spanish. However, the transcript and summary should be automatically translated into English for review and reporting purposes.
Please review and share your suggestions or fixes for the above issues.These are currently blocking our progress (source stoppers).We have attached the Assistant JSON format that we are passing to VAPI. Kindly go through it and advise if any modifications or additions are needed to improve the accuracy and performance.
1) Issue Passing Key Terms in VAPI for Multilingual Models
We are using the assistance feature in VAPI with multiple languages (English, Spanish, Hindi, etc.). However, we’re facing an issue while passing key terms:
The key terms work correctly only with model Nova 3.
Unfortunately, Spanish and Hindi are not supported in Nova 3.
When we switch to Nova 2 (to support these languages) and try to pass the key terms, we receive a 400 Bad Request error.
2) Transcript Accuracy Issue
The transcripts are being returned inconsistently. When the AI agent speaks, the generated transcript does not accurately match the spoken words.
Example:
Actual spoken: “I am working as an Associate Developer in this company.”
Transcript generated: “Working associate in company.”
3) Agent Not Responding to Single-Word Inputs
When short responses such as “Yes,” “No,” or “Okay” are spoken, the AI agent often fails to recognize or respond to them on the first attempt. We have to repeat the word louder or multiple times before the agent responds.
4) Agent Mispronounces and Misspells Spelled-Out Words
When we spell out words (for example, P R A D E E P), the AI agent sometimes mispronounces or returns incorrect transcripts.
Example:
Spoken/Spelled: P R A D E E P
Transcript returned: PRADIP / PRADEP
5) Language Handling and Translation Requirement
When the Spanish language is selected, the call should be conducted entirely in Spanish. However, the transcript and summary should be automatically translated into English for review and reporting purposes.
Please review and share your suggestions or fixes for the above issues.These are currently blocking our progress (source stoppers).We have attached the Assistant JSON format that we are passing to VAPI. Kindly go through it and advise if any modifications or additions are needed to improve the accuracy and performance.
Assitan_JSON.txt2.37KB
