Chat History with Custom LLMs
Who manages chat history with Custom LLMs? Vapi or Our Server?
V@YasH
The management of chat history with Custom LLMs can be handled by both Vapi and your server, depending on how you choose to implement it. Vapi SDK provides the capability to silently update the chat history, which can be useful for background tasks or discreetly logging user interactions. This means that Vapi can manage the chat history to some extent. However, for more extensive management and storage, you might choose to handle the chat history on your server. This would give you more control over the data and allow for more complex operations or integrations with your systems.
The management of chat history with Custom LLMs can be handled by both Vapi and your server, depending on how you choose to implement it. Vapi SDK provides the capability to silently update the chat history, which can be useful for background tasks or discreetly logging user interactions. This means that Vapi can manage the chat history to some extent. However, for more extensive management and storage, you might choose to handle the chat history on your server. This would give you more control over the data and allow for more complex operations or integrations with your systems.
VYou can continue this conversation by mentioning me in the message.
MCould you explain in more detail
M@Vapi
V@YasH
Certainly! The management of chat history with Custom Language Models (LLMs) can be approached in two ways:
Certainly! The management of chat history with Custom Language Models (LLMs) can be approached in two ways:
- Vapi Management: Vapi can handle chat history to a certain extent. For instance, it can keep track of the current session's interactions, which is useful for maintaining context during a conversation. This is typically limited to short-term memory and is not meant for long-term storage or complex data management.
- Server-Side Management: If you require long-term storage, detailed analytics, or more sophisticated control over the chat history, you would typically handle this on your server. By doing so, you can store the chat history in your database, apply data retention policies, and integrate with other systems or analytics tools. This approach gives you full ownership and responsibility for the data.
Mhow to make vapi manage chat history?
M@Vapi
V@YasH
To make Vapi manage chat history, you would typically use the features provided by the Vapi SDK to maintain a record of the conversation as it happens. This involves:
To make Vapi manage chat history, you would typically use the features provided by the Vapi SDK to maintain a record of the conversation as it happens. This involves:
- Session Management: Vapi can store the conversation history for the duration of a session. This allows the API to maintain context between different voice interactions, ensuring a seamless conversation flow.
- Silent Updates: Vapi SDK allows for silent updates to the chat history. This means that you can programmatically add entries to the chat history without them being spoken or visible to the end-user. This is useful for keeping an accurate record of the conversation, including any background tasks or system messages that are part of the interaction but not directly communicated to the user.
- Context Passing: When making requests to Vapi, you can pass the current context, including the chat history, as part of the request. This ensures that the Custom LLM has all the necessary information to understand the conversation's history and respond appropriately.
- Use the Vapi SDK in your application to handle voice interactions.
- Maintain a session for each user to keep track of individual conversations.
- Use the SDK's features to update the chat history silently as needed.
- Pass the relevant context with each interaction to ensure continuity.
DYou have to manage it and pass on to the vapi in the required formats.
