deep-jade•14mo ago
Vision Capabilities
I asked vapi assistant to analyze the image in the attached knowledgebase. (1st page of a pdf). A few days ago it worked perfectly. But now it cannot see the image. Is this due to a recent update? I tried it on ui as well as on API. My client wants to have this feature in their system. You already had it a few days ago, but not now. Kindly let me know how to access models with vision support.
Thanks in advance.
18 Replies
@Ranmal Mendis
I'm sorry to hear that you're experiencing issues with the image analysis feature. It's possible that there may have been changes or updates that affected this functionality. To better assist you, I would need to check the current capabilities of our system and any recent updates that might have impacted the vision support features.
Could you please provide me with the MIME type of the file you are trying to analyze, and if possible, the URL where the image or PDF is located? This information will help me to give you more accurate guidance on how to proceed with accessing models with vision support.
If you have any additional details such as error messages or specific features you are trying to use, please share them as well. This will help me to understand the problem better and provide you with the best possible solution.
You can continue this conversation by mentioning me in the message.
deep-jadeOP•14mo ago
@Vapi here's the link https://auth.vapi.ai/storage/v1/object/public/files/1720981080262-dbb5b6ba-8c54-4ed2-a592-855fdae0e03c.pdf application/pdf - 269.39 KB
@Ranmal Mendis
Thank you for providing the link and the MIME type of the file. It seems that you are trying to analyze a PDF document for images using VAPI's services. If the image analysis feature was working previously but is not functioning now, it could be due to several reasons, such as changes in the API, service updates, or even changes in the way the PDF is structured.
To address this issue, I recommend the following steps:
1. Check for Updates: Review any recent updates in the VAPI documentation or announcements that might indicate changes to the image analysis feature.
2. Review API Changes: Ensure that your API calls are up-to-date with the latest API version and that you're using the correct endpoints and parameters.
3. Contact Support: Since this seems to be a specific issue with the service, the best course of action would be to reach out to our support team. They can provide detailed information on the current status of the image analysis feature and guide you on how to access models with vision support.
Would you like me to assist you in contacting our support team regarding this matter?
wise-white•14mo ago
call id
deep-jadeOP•14mo ago
@Shubham Bajaj the call id is cb47b2de-8cd2-4ec3-8cf4-430b638b1150
There's another thing. I tried to use custom llms to achieve this. In this case, vapi is not sending knowledgebase ids to our server.
deep-jadeOP•14mo ago
Hi Team, Kindly update the API docs or the APIs. The following endpoints do not return knowledgebase IDs as you mentioned in current docs. https://docs.vapi.ai/api-reference/assistants/get-assistant https://docs.vapi.ai/api-reference/calls/get-call
wise-white•14mo ago
sure will update it,
deep-jadeOP•14mo ago
thanks
wise-white•14mo ago
also @Ranmal Mendis if your up let's disucss over the call and solve it.
deep-jadeOP•14mo ago
sure, give me 1 min
wise-white•14mo ago
call me when you ready.
deep-jadeOP•14mo ago
I am ready now
@Shubham Bajaj https://meet.google.com/wcm-jrrd-msd
wise-white•14mo ago
disocrd call.
@Ranmal Mendis ?
deep-jadeOP•14mo ago
I sorted out that with custom llms. But in future, if you can make vision models directly available on vapi.ai it would be great.
wise-white•14mo ago
the assistant missed to pull info from KB, can you share previous working call ids
deep-jadeOP•14mo ago
sure
f6a54d3d-f727-4fd9-9cbe-b10d04ce6ed2 this one was working.
@Shubham Bajaj I found another issue. In custom llms, it's not sending requests to aws ec2 IP addresses(even after adding a self-signed SSL). But it works with ngrok temporary domains. Just fyi. I really like vapi. There's a good demand for this from my clients. I just want this to get improved.
wise-white•14mo ago
I guess this is something with cloudfare
hey custom llm works with KB.
@Ranmal Mendis closing this issue for now if again occurs please create a new support ticket.
"Fred (Image Analyzer)" doesn't had the kowledge base file attached. call id
cb47b2de-8cd2-4ec3-8cf4-430b638b1150
deep-jadeOP•14mo ago
@Shubham Bajaj no idea on the reason, I just used a subdomain and sorted out this. But thanks for your support
@Shubham Bajaj yeah, custom llms works well
@Shubham Bajaj it had, but I already solved my specific issue in a different approach