VAPI•2y ago

Vision Capabilities

I asked vapi assistant to analyze the image in the attached knowledgebase. (1st page of a pdf). A few days ago it worked perfectly. But now it cannot see the image. Is this due to a recent update? I tried it on ui as well as on API. My client wants to have this feature in their system. You already had it a few days ago, but not now. Kindly let me know how to access models with vision support.
Thanks in advance.

Vapi•7/20/24, 10:34 AM

@Ranmal Mendis
I'm sorry to hear that you're experiencing issues with the image analysis feature. It's possible that there may have been changes or updates that affected this functionality. To better assist you, I would need to check the current capabilities of our system and any recent updates that might have impacted the vision support features.

Could you please provide me with the MIME type of the file you are trying to analyze, and if possible, the URL where the image or PDF is located? This information will help me to give you more accurate guidance on how to proceed with accessing models with vision support.

If you have any additional details such as error messages or specific features you are trying to use, please share them as well. This will help me to understand the problem better and provide you with the best possible solution.

Vapi•7/20/24, 10:34 AM

You can continue this conversation by mentioning me in the message.

Mmilitary-pink You can continue this conversation by mentioning me in the message.

primary-violetOP•7/20/24, 10:36 AM

@Vapi here's the link https://auth.vapi.ai/storage/v1/object/public/files/1720981080262-dbb5b6ba-8c54-4ed2-a592-855fdae0e03c.pdf application/pdf - 269.39 KB

Vapi•7/20/24, 10:36 AM

@Ranmal Mendis
Thank you for providing the link and the MIME type of the file. It seems that you are trying to analyze a PDF document for images using VAPI's services. If the image analysis feature was working previously but is not functioning now, it could be due to several reasons, such as changes in the API, service updates, or even changes in the way the PDF is structured.

To address this issue, I recommend the following steps:

Check for Updates: Review any recent updates in the VAPI documentation or announcements that might indicate changes to the image analysis feature.
Review API Changes: Ensure that your API calls are up-to-date with the latest API version and that you're using the correct endpoints and parameters.
Contact Support: Since this seems to be a specific issue with the service, the best course of action would be to reach out to our support team. They can provide detailed information on the current status of the image analysis feature and guide you on how to access models with vision support.

Would you like me to assist you in contacting our support team regarding this matter?

Pprimary-violet I asked vapi assistant to analyze the image in the attached knowledgebase. (1st ...

dangerous-fuchsia•7/20/24, 1:48 PM

call id

primary-violetOP•7/20/24, 5:14 PM

@Shubham Bajaj the call id is cb47b2de-8cd2-4ec3-8cf4-430b638b1150

primary-violetOP•7/20/24, 7:18 PM

There's another thing. I tried to use custom llms to achieve this. In this case, vapi is not sending knowledgebase ids to our server.

primary-violetOP•7/20/24, 8:46 PM

Hi Team, Kindly update the API docs or the APIs. The following endpoints do not return knowledgebase IDs as you mentioned in current docs. https://docs.vapi.ai/api-reference/assistants/get-assistant https://docs.vapi.ai/api-reference/calls/get-call

Vapi

Get Assistant - Vapi

Vapi

Get Call - Vapi

Pprimary-violet Hi Team, Kindly update the API docs or the APIs. The following endpoints do not ...

dangerous-fuchsia•7/20/24, 9:06 PM

sure will update it,

primary-violetOP•7/20/24, 9:06 PM

thanks

Pprimary-violet thanks

dangerous-fuchsia•7/20/24, 9:07 PM

also @Ranmal Mendis if your up let's disucss over the call and solve it.

primary-violetOP•7/20/24, 9:07 PM

sure, give me 1 min

Pprimary-violet sure, give me 1 min

dangerous-fuchsia•7/20/24, 9:08 PM

call me when you ready.

primary-violetOP•7/20/24, 9:09 PM

I am ready now

primary-violetOP•7/20/24, 9:11 PM

@Shubham Bajaj https://meet.google.com/wcm-jrrd-msd

Pprimary-violet @Shubham Bajaj https://meet.google.com/wcm-jrrd-msd

dangerous-fuchsia•7/20/24, 9:33 PM

disocrd call.

dangerous-fuchsia•7/20/24, 9:34 PM

@Ranmal Mendis ?

primary-violetOP•7/20/24, 9:39 PM

I sorted out that with custom llms. But in future, if you can make vision models directly available on vapi.ai it would be great.

dangerous-fuchsia•7/20/24, 9:41 PM

the assistant missed to pull info from KB, can you share previous working call ids

primary-violetOP•7/20/24, 9:48 PM

sure

primary-violetOP•7/20/24, 9:53 PM

f6a54d3d-f727-4fd9-9cbe-b10d04ce6ed2 this one was working.

Pprimary-violet I sorted out that with custom llms. But in future, if you can make vision models...

primary-violetOP•7/21/24, 12:33 AM

@Shubham Bajaj I found another issue. In custom llms, it's not sending requests to aws ec2 IP addresses(even after adding a self-signed SSL). But it works with ngrok temporary domains. Just fyi. I really like vapi. There's a good demand for this from my clients. I just want this to get improved.

Pprimary-violet @Shubham Bajaj I found another issue. In custom llms, it's not sending requests ...

dangerous-fuchsia•7/22/24, 3:49 PM

I guess this is something with cloudfare

Pprimary-violet There's another thing. I tried to use custom llms to achieve this. In this case,...

dangerous-fuchsia•7/22/24, 4:31 PM

hey custom llm works with KB.

dangerous-fuchsia•7/22/24, 4:49 PM

@Ranmal Mendis closing this issue for now if again occurs please create a new support ticket.

dangerous-fuchsia•7/22/24, 4:54 PM

"Fred (Image Analyzer)" doesn't had the kowledge base file attached. call id

dangerous-fuchsia•7/22/24, 4:54 PM

cb47b2de-8cd2-4ec3-8cf4-430b638b1150