adverse-sapphire•14mo ago

Custom Model with ‘authorization’ headers in request

hey, so I currently have no idea how to implement a custom model in Vapi. Because when I usually send post requests I do have to add an 'Authorization' part with Bearer Token into my headers. But I think it's not possible to add those in the vapi system right? So is there any workaround? This would be an important feature for me. Thanks

8 Replies

Vapi•14mo ago

@OGOnkel Hello! I'm glad you reached out for assistance with implementing a custom model in VAPI. You're correct that typically, when making POST requests, you would include an 'Authorization' header with a Bearer Token. However, with VAPI, you can authenticate your OpenAI-compatible endpoint by using the /credential endpoint to POST your server’s API key and URL. Here's how you can do it: 1. Prepare your server's API key and the URL for the OpenAI-compatible endpoint. 2. Use the /credential endpoint to submit your API key and URL in the following format:

{
  "provider": "custom-llm",
  "apiKey": "YOUR_API_KEY_HERE"
}

{
  "provider": "custom-llm",
  "apiKey": "YOUR_API_KEY_HERE"
}

Once you've done this, your server will be authenticated, and you can start making requests to your custom model through VAPI without needing to manually add the 'Authorization' header each time. If you need any further assistance or have additional questions, please let me know! You can continue this conversation by mentioning me in the message.

afraid-scarlet•14mo ago

hey check the following: [1] https://docs.vapi.ai/customization/custom-llm/using-your-server [2] https://github.com/VapiAI/advanced-concepts-custom-llm

Vapi

Connecting Your Custom LLM to Vapi: A Comprehensive Guide - Vapi

GitHub

GitHub - VapiAI/advanced-concepts-custom-llm: This is a sample exam...

This is a sample example repo on how to use Vapi with your custom-llm - VapiAI/advanced-concepts-custom-llm

adverse-sapphireOP•14mo ago

@Vapi I successfully managed to integrate my custom model but streaming the response does not work fully. I do see in the logs those data chunks but VAPI doesn't generate speech content although it shows "Model sent end token" in the Call Log Details: """data: {"id":"chatcmpl-9lNDIiDF9tMPniynvEFZbkWz6bYaw","object":"chat.completion.chunk","created":1721077676,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_dd932ca5d1","choices":[{"index":0,"delta":{"content":"Hey"},"logprobs":null,"finish_reason":null}]} data: {"id":"chatcmpl-9lNDIiDF9tMPniynvEFZbkWz6bYaw","object":"chat.completion.chunk","created":1721077676,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_dd932ca5d1","choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null}]} @Vapi Team

afraid-scarlet•14mo ago

@OGOnkel please share the call id.

adverse-sapphireOP•14mo ago

@Shubham Bajaj 4b5008b2-4dd9-4018-ab8b-4288c04e7f06

afraid-scarlet•14mo ago

this is your system content, "48487478-44c0-4610-b331-2e98aa8c04d9"?

adverse-sapphireOP•14mo ago

@Shubham Bajaj yes that's correct. In my custom model it's an identifier for custom data.

afraid-scarlet•14mo ago

@OGOnkel can you please share your repo link/snippet.

Custom Model with ‘authorization’ headers in request

Did you find this page helpful?