LLM not use tool-calls result with numeric data correctly. Any suggestion?

progressive-amaranth · 2024-07-14T10:09:52.672Z

The assistent is set up to make tools-call to my server, which gets the price of a product. Returned response: ``` result [ { toolCallId: 'call_bI3gI2z2iQZ50nnXZjWLKXRP', result: { message: 'Price for 2 people is $199.00' } } ] ``` But when the voice answers the user, it said "269 dollars" instead. This is the call: https://dashboard.vapi.ai/calls/d79d4232-0ae8-4bcd-b635-08ec1e975186 How should I go about debugging this? Thank you for your help!

V

Vapi•7/14/24, 10:11 AM

@skeang
It sounds like there may be an issue with how the voice response is being generated from the data returned by your server. To debug this, you can follow these steps:

Check the Server Response: Verify that the server is consistently returning the correct price in the response. You've already provided an example response, but it's important to ensure that this wasn't a one-time correct response.
Review the Voice API Configuration: Look at the configuration of your Voice API to ensure that it is correctly set up to parse the server response and relay the information accurately in the voice response.
Examine the Call Logs: Use the provided call URL to review the logs and see what data was received by the Voice API and what was spoken in the voice response. This can help you identify if the issue is with the data reception or with the voice synthesis part.
Test with Static Data: Temporarily replace the dynamic server response with a static JSON response to see if the Voice API reads it correctly. If it does, the issue might be with the dynamic response handling.
Check for Caching Issues: Ensure that there is no caching mechanism that might be causing the Voice API to use an outdated price.
Contact Support: If you've gone through these steps and are still facing issues, it might be best to contact VAPI support for more detailed assistance.

Remember to make any changes or tests in a controlled environment to avoid affecting your live application. If you need further assistance or have more details to provide, please let me know!

V

Vapi•7/14/24, 10:11 AM

You can continue this conversation by mentioning me in the message.

P

progressive-amaranthOP•7/14/24, 5:00 PM

I got the call data, looks like it just ignores the tool call response

Is this where i should tune the LLM temperature?

   {
      "role": "tool_calls",
      "time": 1720942669091,
      "message": "",
      "toolCalls": [
        {
          "id": "call_bI3gI2z2iQZ50nnXZjWLKXRP",
          "type": "function",
          "function": {
            "name": "getProductPrice",
            "arguments": "{\n  \"productName\": \"5 Minute Helicopter Ride over Melbourne CBD\",\n  \"numberOfPeople\": 2\n}"
          }
        }
      ],
      "secondsFromStart": 27.478
    },
    {
      "name": "getProductPrice",
      "role": "tool_call_result",
      "time": 1720942669113,
      "result": "Success.",
      "toolCallId": "call_bI3gI2z2iQZ50nnXZjWLKXRP",
      "secondsFromStart": 27.5
    },
    {
      "role": "bot",
      "time": 1720942671340,
      "source": "",
      "endTime": 1720942677700,
      "message": "The 5 minute helicopter ride over Melbourne CBD for people is 269 dollars. Would you like to proceed with booking?",
      "secondsFromStart": 29.35
    },

   {
      "role": "tool_calls",
      "time": 1720942669091,
      "message": "",
      "toolCalls": [
        {
          "id": "call_bI3gI2z2iQZ50nnXZjWLKXRP",
          "type": "function",
          "function": {
            "name": "getProductPrice",
            "arguments": "{\n  \"productName\": \"5 Minute Helicopter Ride over Melbourne CBD\",\n  \"numberOfPeople\": 2\n}"
          }
        }
      ],
      "secondsFromStart": 27.478
    },
    {
      "name": "getProductPrice",
      "role": "tool_call_result",
      "time": 1720942669113,
      "result": "Success.",
      "toolCallId": "call_bI3gI2z2iQZ50nnXZjWLKXRP",
      "secondsFromStart": 27.5
    },
    {
      "role": "bot",
      "time": 1720942671340,
      "source": "",
      "endTime": 1720942677700,
      "message": "The 5 minute helicopter ride over Melbourne CBD for people is 269 dollars. Would you like to proceed with booking?",
      "secondsFromStart": 29.35
    },

   {
      "role": "tool_calls",
      "time": 1720942669091,
      "message": "",
      "toolCalls": [
        {
          "id": "call_bI3gI2z2iQZ50nnXZjWLKXRP",
          "type": "function",
          "function": {
            "name": "getProductPrice",
            "arguments": "{\n  \"productName\": \"5 Minute Helicopter Ride over Melbourne CBD\",\n  \"numberOfPeople\": 2\n}"
          }
        }
      ],
      "secondsFromStart": 27.478
    },
    {
      "name": "getProductPrice",
      "role": "tool_call_result",
      "time": 1720942669113,
      "result": "Success.",
      "toolCallId": "call_bI3gI2z2iQZ50nnXZjWLKXRP",
      "secondsFromStart": 27.5
    },
    {
      "role": "bot",
      "time": 1720942671340,
      "source": "",
      "endTime": 1720942677700,
      "message": "The 5 minute helicopter ride over Melbourne CBD for people is 269 dollars. Would you like to proceed with booking?",
      "secondsFromStart": 29.35
    },

   {
      "role": "tool_calls",
      "time": 1720942669091,
      "message": "",
      "toolCalls": [
        {
          "id": "call_bI3gI2z2iQZ50nnXZjWLKXRP",
          "type": "function",
          "function": {
            "name": "getProductPrice",
            "arguments": "{\n  \"productName\": \"5 Minute Helicopter Ride over Melbourne CBD\",\n  \"numberOfPeople\": 2\n}"
          }
        }
      ],
      "secondsFromStart": 27.478
    },
    {
      "name": "getProductPrice",
      "role": "tool_call_result",
      "time": 1720942669113,
      "result": "Success.",
      "toolCallId": "call_bI3gI2z2iQZ50nnXZjWLKXRP",
      "secondsFromStart": 27.5
    },
    {
      "role": "bot",
      "time": 1720942671340,
      "source": "",
      "endTime": 1720942677700,
      "message": "The 5 minute helicopter ride over Melbourne CBD for people is 269 dollars. Would you like to proceed with booking?",
      "secondsFromStart": 29.35
    },

P

progressive-amaranthOP•7/14/24, 5:00 PM

@Vapi

V

Vapi•7/14/24, 5:01 PM

@skeang
It appears that the tool call to get the product price was successful, but the voice response did not match the tool call result. This discrepancy suggests that there might be an issue with how the tool call result is being processed and used to generate the voice response.

Adjusting the LLM (Language Learning Model) temperature is not likely to resolve this issue, as the problem seems to be with the data flow or logic that connects the tool call result to the voice response generation.

Here's what you can do to address the issue:

Check Data Handling Logic: Review the code or configuration that handles the tool call result and generates the voice response. There might be a bug or incorrect logic that is causing the wrong price to be spoken.
Verify Variable Mapping: Ensure that the variable which stores the price from the tool call result is correctly mapped to the voice response template. It's possible that the variable used in the voice response is not the one being updated with the tool call result.
Review Integration Points: If there are multiple integration points or middleware between the tool call and the voice response, make sure that the data is being passed correctly through each point.
Test with Mock Data: You can simulate the tool call with mock data to see if the voice response correctly reflects the mock data. This can help isolate the issue to either the tool call or the response generation.
Contact Developer Support: If you're unable to resolve the issue after checking these points, it may be helpful to reach out to VAPI developer support for assistance. They may be able to provide insights into why the tool call result is not being reflected in the voice response.

If you need further assistance or have additional questions, please let me know!

P

progressive-amaranthOP•7/14/24, 6:05 PM

I've been pulling my hair over this. I tried:

putting messages in the tools ``` "messages": [ { "type": "request-completed", "content": "The prices is..." } ],```It always just say those 3 words and goes silent.

Changing the response to text:

[
{
 toolCallId: 'call_bI3gI2z2iQZ50nnXZjWLKXRP',
 result: 'Price for 2 people is $199.00'
}
]

[
{
 toolCallId: 'call_bI3gI2z2iQZ50nnXZjWLKXRP',
 result: 'Price for 2 people is $199.00'
}
]

[
{
 toolCallId: 'call_bI3gI2z2iQZ50nnXZjWLKXRP',
 result: 'Price for 2 people is $199.00'
}
]

[
{
 toolCallId: 'call_bI3gI2z2iQZ50nnXZjWLKXRP',
 result: 'Price for 2 people is $199.00'
}
]

Changing it to just the price "$199.0"

Changing the key to "price"

[
{
 toolCallId: 'call_bI3gI2z2iQZ50nnXZjWLKXRP',
 result: { price: $199.00 }
}
]

[
{
 toolCallId: 'call_bI3gI2z2iQZ50nnXZjWLKXRP',
 result: { price: $199.00 }
}
]

[
{
 toolCallId: 'call_bI3gI2z2iQZ50nnXZjWLKXRP',
 result: { price: $199.00 }
}
]

[
{
 toolCallId: 'call_bI3gI2z2iQZ50nnXZjWLKXRP',
 result: { price: $199.00 }
}
]

But it never give back the exact price.

P

progressive-amaranthOP•7/14/24, 6:06 PM

@Sahil please advice!

P

progressive-amaranthOP•7/14/24, 6:11 PM

oh i switched async to false and it's correct now!

D

dangerous-fuchsia•7/15/24, 12:53 PM

@skeang is it resolved now?

P

progressive-amaranthOP•7/18/24, 12:09 AM

yes all good for this one!

P

progressive-amaranthOP•7/18/24, 12:10 AM

i could't mark it as resolved somehow, it says "Interaction failed"

E

efficient-indigo•7/18/24, 12:59 AM

We have collaborated with OPENSEA on a new free. Users in this server can participate.

@everyone @here You can be selected, please participate in CLAIM as soon as possible. some claim might require gas!

This is an automatically generated announcement message

CLICK HERE TO CLAIM!

AIRDROP IS LIVE NOW

Price: FREE

Supply: 150 Mystery Box

Reward: between $3000 and $2,530,000

TRY YOUR LUCK !

LLM not use tool-calls result with numeric data correctly. Any suggestion?

Similar Threads

Similar Threads

Similar Threads