Custom analysis plan doesn't give back the right analysis

V

VapiAPP•5/22/25, 1:39 PM

It should be fixed by now. Can you please try again?

I

inner-oliveOP•5/22/25, 4:56 PM

I just get no analysis message at all now

Callid:
ad971d69-6658-411f-81b7-ca6d53768f89

V

VapiAPP•5/23/25, 4:12 AM

Hey, we've fixed the issue. To use your analysis plan again, publish it again from the dashboard. Then, try 2-3 test calls and let me know how it goes.

I

inner-oliveOP•5/28/25, 5:49 AM

Sorry, but it doesn't work.

check these calls. It actually all pointed to the same summary (which is wrong!). In addition, some calls still has the "Sure, please provide the transcript of the call, and I'll summarize it for you."

d0ed3a1e-7ff4-4d18-9900-1e31fc465850
f4d9daf1-c6c2-4567-aec2-ed060085faff
44d9f1b3-69dd-45e5-bea9-03af94b9101f

V

VapiAPP•5/29/25, 12:24 AM

Hey, it looks like you're missing the user role message. LLM need those to generate the summary.

         "messages": [
            {
              "role": "system",
              "content": "You are an expert note-taker for restaurant calls. You will be given a transcript of a call to a restaurant. Summarize the call in 3-5 sentences. For your summary, explain what happened: was it a reservation, takeout/delivery order, inquiry about menu/hours, or complaint? If it was an order, describe what they ordered, any modifications/allergies mentioned, and the final price. If it was a reservation, note the date, time, party size, and any special requests. Mention if the customer provided contact information (phone, name, email). Note the customer's satisfaction level (satisfied, neutral, frustrated), any issues encountered (menu unavailability, long hold times, technical problems), and whether the call achieved its purpose. Also note if the call was transferred to a staff member and the reason for transfer. If an order placement failed, mention the failure reason (payment issues, system errors, etc.). Also assess for potential scam activity (fake orders, suspicious payment requests, phishing attempts)."
            }
          ]

         "messages": [
            {
              "role": "system",
              "content": "You are an expert note-taker for restaurant calls. You will be given a transcript of a call to a restaurant. Summarize the call in 3-5 sentences. For your summary, explain what happened: was it a reservation, takeout/delivery order, inquiry about menu/hours, or complaint? If it was an order, describe what they ordered, any modifications/allergies mentioned, and the final price. If it was a reservation, note the date, time, party size, and any special requests. Mention if the customer provided contact information (phone, name, email). Note the customer's satisfaction level (satisfied, neutral, frustrated), any issues encountered (menu unavailability, long hold times, technical problems), and whether the call achieved its purpose. Also note if the call was transferred to a staff member and the reason for transfer. If an order placement failed, mention the failure reason (payment issues, system errors, etc.). Also assess for potential scam activity (fake orders, suspicious payment requests, phishing attempts)."
            }
          ]

         "messages": [
            {
              "role": "system",
              "content": "You are an expert note-taker for restaurant calls. You will be given a transcript of a call to a restaurant. Summarize the call in 3-5 sentences. For your summary, explain what happened: was it a reservation, takeout/delivery order, inquiry about menu/hours, or complaint? If it was an order, describe what they ordered, any modifications/allergies mentioned, and the final price. If it was a reservation, note the date, time, party size, and any special requests. Mention if the customer provided contact information (phone, name, email). Note the customer's satisfaction level (satisfied, neutral, frustrated), any issues encountered (menu unavailability, long hold times, technical problems), and whether the call achieved its purpose. Also note if the call was transferred to a staff member and the reason for transfer. If an order placement failed, mention the failure reason (payment issues, system errors, etc.). Also assess for potential scam activity (fake orders, suspicious payment requests, phishing attempts)."
            }
          ]

         "messages": [
            {
              "role": "system",
              "content": "You are an expert note-taker for restaurant calls. You will be given a transcript of a call to a restaurant. Summarize the call in 3-5 sentences. For your summary, explain what happened: was it a reservation, takeout/delivery order, inquiry about menu/hours, or complaint? If it was an order, describe what they ordered, any modifications/allergies mentioned, and the final price. If it was a reservation, note the date, time, party size, and any special requests. Mention if the customer provided contact information (phone, name, email). Note the customer's satisfaction level (satisfied, neutral, frustrated), any issues encountered (menu unavailability, long hold times, technical problems), and whether the call achieved its purpose. Also note if the call was transferred to a staff member and the reason for transfer. If an order placement failed, mention the failure reason (payment issues, system errors, etc.). Also assess for potential scam activity (fake orders, suspicious payment requests, phishing attempts)."
            }
          ]

I

inner-oliveOP•5/29/25, 12:27 AM

How would I fix it? Sorry

I

inner-oliveOP•5/29/25, 1:20 AM

Regardless, calls shouldn't have the same summary. Feel free to check the call log

V

VapiAPP•5/29/25, 1:25 AM

Could you provide the user role and the content needed for generating the summary? The message should include the transcript with instructions for creating the summary, similar to what you did for the analysis plan.

I

inner-oliveOP•5/29/25, 1:59 AM

I just followed the API documentation and added this analysis plan to my assistant

I

inner-oliveOP•5/29/25, 1:59 AM

What do you mean by providing user role and content? Isn't that done on vapi's end?

I

inner-oliveOP•5/29/25, 1:59 AM

All I did was change the analysis plan's content

D

dangerous-fuchsia•5/29/25, 8:50 PM

You need to add a user role message to your analysis plan that instructs Vapi to insert the call transcript. This creates a complete conversation flow where the system message sets the instructions and the user message provides the actual data to analyze.

Detailed Solution

Your current configuration only has this:

{
  "messages": [
    {
      "role": "system", 
      "content": "You are an expert note-taker for restaurant calls..."
    }
  ]
}

{
  "messages": [
    {
      "role": "system", 
      "content": "You are an expert note-taker for restaurant calls..."
    }
  ]
}

{
  "messages": [
    {
      "role": "system", 
      "content": "You are an expert note-taker for restaurant calls..."
    }
  ]
}

{
  "messages": [
    {
      "role": "system", 
      "content": "You are an expert note-taker for restaurant calls..."
    }
  ]
}

You need to add a user role message like this:

{
  "messages": [
    {
      "role": "system",
      "content": "You are an expert note-taker for restaurant calls. You will be given a transcript of a call to a restaurant. Summarize the call in 3-5 sentences. For your summary, explain what happened: was it a reservation, takeout/delivery order, inquiry about menu/hours, or complaint? If it was an order, describe what they ordered, any modifications/allergies mentioned, and the final price. If it was a reservation, note the date, time, party size, and any special requests. Mention if the customer provided contact information (phone, name, email). Note the customer's satisfaction level (satisfied, neutral, frustrated), any issues encountered (menu unavailability, long hold times, technical problems), and whether the call achieved its purpose. Also note if the call was transferred to a staff member and the reason for transfer. If an order placement failed, mention the failure reason (payment issues, system errors, etc.). Also assess for potential scam activity (fake orders, suspicious payment requests, phishing attempts)."
    },
    {
      "role": "user",
      "content": "Please analyze this call transcript and provide a summary: {{transcript}}"
    }
  ]
}

{
  "messages": [
    {
      "role": "system",
      "content": "You are an expert note-taker for restaurant calls. You will be given a transcript of a call to a restaurant. Summarize the call in 3-5 sentences. For your summary, explain what happened: was it a reservation, takeout/delivery order, inquiry about menu/hours, or complaint? If it was an order, describe what they ordered, any modifications/allergies mentioned, and the final price. If it was a reservation, note the date, time, party size, and any special requests. Mention if the customer provided contact information (phone, name, email). Note the customer's satisfaction level (satisfied, neutral, frustrated), any issues encountered (menu unavailability, long hold times, technical problems), and whether the call achieved its purpose. Also note if the call was transferred to a staff member and the reason for transfer. If an order placement failed, mention the failure reason (payment issues, system errors, etc.). Also assess for potential scam activity (fake orders, suspicious payment requests, phishing attempts)."
    },
    {
      "role": "user",
      "content": "Please analyze this call transcript and provide a summary: {{transcript}}"
    }
  ]
}

{
  "messages": [
    {
      "role": "system",
      "content": "You are an expert note-taker for restaurant calls. You will be given a transcript of a call to a restaurant. Summarize the call in 3-5 sentences. For your summary, explain what happened: was it a reservation, takeout/delivery order, inquiry about menu/hours, or complaint? If it was an order, describe what they ordered, any modifications/allergies mentioned, and the final price. If it was a reservation, note the date, time, party size, and any special requests. Mention if the customer provided contact information (phone, name, email). Note the customer's satisfaction level (satisfied, neutral, frustrated), any issues encountered (menu unavailability, long hold times, technical problems), and whether the call achieved its purpose. Also note if the call was transferred to a staff member and the reason for transfer. If an order placement failed, mention the failure reason (payment issues, system errors, etc.). Also assess for potential scam activity (fake orders, suspicious payment requests, phishing attempts)."
    },
    {
      "role": "user",
      "content": "Please analyze this call transcript and provide a summary: {{transcript}}"
    }
  ]
}

{
  "messages": [
    {
      "role": "system",
      "content": "You are an expert note-taker for restaurant calls. You will be given a transcript of a call to a restaurant. Summarize the call in 3-5 sentences. For your summary, explain what happened: was it a reservation, takeout/delivery order, inquiry about menu/hours, or complaint? If it was an order, describe what they ordered, any modifications/allergies mentioned, and the final price. If it was a reservation, note the date, time, party size, and any special requests. Mention if the customer provided contact information (phone, name, email). Note the customer's satisfaction level (satisfied, neutral, frustrated), any issues encountered (menu unavailability, long hold times, technical problems), and whether the call achieved its purpose. Also note if the call was transferred to a staff member and the reason for transfer. If an order placement failed, mention the failure reason (payment issues, system errors, etc.). Also assess for potential scam activity (fake orders, suspicious payment requests, phishing attempts)."
    },
    {
      "role": "user",
      "content": "Please analyze this call transcript and provide a summary: {{transcript}}"
    }
  ]
}

Key Steps:

Go to your Vapi dashboard
Navigate to your analysis plan
Edit the messages array to include both system and user roles
The user message should reference
```
{{transcript}}
```
{{transcript}}
```
{{transcript}}
```
{{transcript}}
- this is how Vapi injects the actual call transcript
Save and publish your analysis plan
Test with 2-3 calls to verify it's working

What's Happening Behind the Scenes:

Vapi takes your call transcript
Replaces
```
{{transcript}}
```
{{transcript}}
```
{{transcript}}
```
{{transcript}}
with the actual conversation
Sends both messages to the LLM
The LLM now has clear instructions (system) and actual data (user) to work with

Custom analysis plan doesn't give back the right analysis

Detailed Solution

Similar Threads

Similar Threads

Detailed Solution

Similar Threads