Is there a way to connect vapi to custom GPTs?
That would avoid us having to build and maintain our own RAG environment. Either custom GPTs or POE..
L
CHave trouble getting the custom-llm to receive responses using standard openAI response objects. Also, the vapi custom-llm sends non-standard openai info such as 'call', 'metadata' etc. so this doens't seem to mimic openAI's format. Passing a vapi call to directly to openai would generate an error: {'error': {'message': 'Unrecognized request arguments supplied: call, metadata', ...}}. Appreciate any help to get custom-LLM working. thanks!
T@Crypto Plato Can you tell me more context about your issue?
CSure. I set up a simple openAI compatible endpoint with dummy response to use as a custom-llm, which is successfully receiving the first message from Vapi, and responds with standard openai response object. After that Vapi goes silent and doesn't use the custom-llm response. the conversation then times out. The call log shows LLL Requested, LLM end token received then nothing.
CFor example here a python snippet I used to simply pass the openAI response back to vapi which vapi isn't able to process using the custom-llm setting even though this test function works by accessing it directly: @app.post("/api/openai/chat/completions") async def custom_model_openai(request: Request):
data = await request.json()
print("Raw input received from VAPI:", data)
# Use the messages from the VAPI request
user_messages = data.get('messages', [])
# Send request to OpenAI
completion = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=user_messages
)
# Use the json() method to get the response data in JSON serializable format
completion_data = completion.model_dump_json()
# Log the response being sent back for debugging
print("Sending OpenAI response back to VAPI:", completion_data)
# Return the OpenAI response directly
return JSONResponse(content=completion_data)
data = await request.json()
print("Raw input received from VAPI:", data)
# Use the messages from the VAPI request
user_messages = data.get('messages', [])
# Send request to OpenAI
completion = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=user_messages
)
# Use the json() method to get the response data in JSON serializable format
completion_data = completion.model_dump_json()
# Log the response being sent back for debugging
print("Sending OpenAI response back to VAPI:", completion_data)
# Return the OpenAI response directly
return JSONResponse(content=completion_data)
Iah yup, you need to SSE responses back to us
Ito test, create a script to call your openai competible endpoint with
stream=TrueIif that works, it'll work with VAPI
DI just got some numbers that might be interesting to share with the @VapiTeam. A telemarketing operator costs in Brazil R$2890 with all taxes included. If you take 0,08 cents/minute. So For 10560 minutes in a month for a regular shift, we are talking $844,8/month, multiply this by 5 reais per Dollar = R$4224/month. I thought I was going to offer something like at least a 30% cost reduction, for this to be an easy sell.. Also, I’m pretty sure that our labor costs here in Brazil are higher than India where a lot of the call centers are being outsourced to. Have you guys come across the numbers for India? Any selling points that we could use to justify the higher costs? Maybe start with some more noble applications like tutoring…
LIt’s pretty incomparable
L- Consumers hate accents from other countries, immediately discredits them and puts up the consumers guard.
- For current robo callers, you can not have a conversation with that robot and ask clarifying questions
LSide note, VoiceAI isn’t at a place where it can replace human call centers and have the same effect. Soon yes, now no. For the next two years or so it’ll be a ramp up period as more companies take a risk on voiceAI, consumers will adapt behaviors, and in turn more companies will adopt voiceAI solutions, and in turn more consumers will adapt.
LPlenty of companies are trying to become ai call centers and run into a scaling problem and then get destroyed by their burn rate because the scale of mass adoption you need to make real money is 95% impossible to reach yet.
The only money in it right now is finding low risk calls to automate at scale for SMBs. Think about your value prop as how many hours you can save (not by replacing but by assisting) CSRs. 10 CSRs $15 an hour 8 hours a day is $1,200 / day. I’ve done research and interviews with CSRs & business owners and I can guarantee you 60% of what CSRs do is repetitive low risk work. If you can save someone $720 a day (60% reduction) in hours (not to mention the revenue generated from them now being free to do more high value, revenue generating tasks), what could you charge them a month?
The only money in it right now is finding low risk calls to automate at scale for SMBs. Think about your value prop as how many hours you can save (not by replacing but by assisting) CSRs. 10 CSRs $15 an hour 8 hours a day is $1,200 / day. I’ve done research and interviews with CSRs & business owners and I can guarantee you 60% of what CSRs do is repetitive low risk work. If you can save someone $720 a day (60% reduction) in hours (not to mention the revenue generated from them now being free to do more high value, revenue generating tasks), what could you charge them a month?
