Pipe audio stream to another application

living-lavender · 2025-10-15T00:45:30.667Z

Hello. Is it possible to take the audio stream in a web session and send that on to another application? My hope is that I can take the audio stream from the agent and send it into another application we are building. Format could be byte stream, or anything. Any thoughts, questions, or guidance would be appreciated, thanks.

V

Vapi Support BotAPP•10/15/25, 12:47 AM

Yes, you can pipe audio from a web session to another application by capturing audio in the browser and sending it as a byte stream over a WebSocket. Here’s a simple example using JavaScript:

navigator.mediaDevices.getUserMedia({ audio: true }).then(stream => {
  const audioContext = new AudioContext();
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(1024, 1, 1);
  const socket = new WebSocket('wss://your-server-url');
  processor.onaudioprocess = (event) => {
    const pcmData = event.inputBuffer.getChannelData(0);
    const int16Data = new Int16Array(pcmData.length);
    for (let i = 0; i < pcmData.length; i++) {
      int16Data[i] = Math.max(-32768, Math.min(32767, pcmData[i] * 32768));
    }
    if (socket.readyState === WebSocket.OPEN) socket.send(int16Data.buffer);
  };
  source.connect(processor);
  processor.connect(audioContext.destination);
});

navigator.mediaDevices.getUserMedia({ audio: true }).then(stream => {
  const audioContext = new AudioContext();
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(1024, 1, 1);
  const socket = new WebSocket('wss://your-server-url');
  processor.onaudioprocess = (event) => {
    const pcmData = event.inputBuffer.getChannelData(0);
    const int16Data = new Int16Array(pcmData.length);
    for (let i = 0; i < pcmData.length; i++) {
      int16Data[i] = Math.max(-32768, Math.min(32767, pcmData[i] * 32768));
    }
    if (socket.readyState === WebSocket.OPEN) socket.send(int16Data.buffer);
  };
  source.connect(processor);
  processor.connect(audioContext.destination);
});

navigator.mediaDevices.getUserMedia({ audio: true }).then(stream => {
  const audioContext = new AudioContext();
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(1024, 1, 1);
  const socket = new WebSocket('wss://your-server-url');
  processor.onaudioprocess = (event) => {
    const pcmData = event.inputBuffer.getChannelData(0);
    const int16Data = new Int16Array(pcmData.length);
    for (let i = 0; i < pcmData.length; i++) {
      int16Data[i] = Math.max(-32768, Math.min(32767, pcmData[i] * 32768));
    }
    if (socket.readyState === WebSocket.OPEN) socket.send(int16Data.buffer);
  };
  source.connect(processor);
  processor.connect(audioContext.destination);
});

navigator.mediaDevices.getUserMedia({ audio: true }).then(stream => {
  const audioContext = new AudioContext();
  const source = audioContext.createMediaStreamSource(stream);
  const processor = audioContext.createScriptProcessor(1024, 1, 1);
  const socket = new WebSocket('wss://your-server-url');
  processor.onaudioprocess = (event) => {
    const pcmData = event.inputBuffer.getChannelData(0);
    const int16Data = new Int16Array(pcmData.length);
    for (let i = 0; i < pcmData.length; i++) {
      int16Data[i] = Math.max(-32768, Math.min(32767, pcmData[i] * 32768));
    }
    if (socket.readyState === WebSocket.OPEN) socket.send(int16Data.buffer);
  };
  source.connect(processor);
  processor.connect(audioContext.destination);
});

This streams raw PCM audio to your WebSocket server, where you can forward or process it as needed.
For best results, ensure your receiving application can handle the incoming audio buffer format.
See more details in the documentation.

Source:

WebSocket Transport Vapi Documentation

⤵️ Mention @Vapi Support Bot for follow-ups 🐥

L

living-lavenderOP•10/15/25, 12:51 AM

@Vapi Support Bot can this be used to take the audio from the agent, the example looks like it takes the users microphone and sends it to another application. it doesn't seem like it would take vapi's audio output (the agent speaking)

V

Vapi Support BotAPP•10/15/25, 12:52 AM

Thank you for clarifying your goal. To help you best: Are you looking to capture the agent's audio output in real time during a live call (e.g., via WebSocket), or do you need access to the audio files after the call is finished? This will help me suggest the most suitable approach for your use case.

Sources:

L

living-lavenderOP•10/15/25, 12:52 AM

@Vapi Support Bot I'm wanting to take the real time stream of vapi's agent audio and send it to another application.

V

Vapi Support BotAPP•10/15/25, 12:53 AM

To capture the agent's audio output in real time and send it to another application, set up a custom TTS endpoint in Vapi. When your assistant needs to speak, Vapi will POST the text to your endpoint, and you can generate or proxy the audio, then forward the raw PCM bytes wherever you need.

Example (Node.js Express):

app.post('/api/synthesize', async (req, res) => {
  // Generate or fetch audioBuffer (PCM) for req.body.message.text
  // Forward audioBuffer to your other application here
  res.setHeader('Content-Type', 'application/octet-stream');
  res.write(audioBuffer);
  res.end();
});

app.post('/api/synthesize', async (req, res) => {
  // Generate or fetch audioBuffer (PCM) for req.body.message.text
  // Forward audioBuffer to your other application here
  res.setHeader('Content-Type', 'application/octet-stream');
  res.write(audioBuffer);
  res.end();
});

app.post('/api/synthesize', async (req, res) => {
  // Generate or fetch audioBuffer (PCM) for req.body.message.text
  // Forward audioBuffer to your other application here
  res.setHeader('Content-Type', 'application/octet-stream');
  res.write(audioBuffer);
  res.end();
});

app.post('/api/synthesize', async (req, res) => {
  // Generate or fetch audioBuffer (PCM) for req.body.message.text
  // Forward audioBuffer to your other application here
  res.setHeader('Content-Type', 'application/octet-stream');
  res.write(audioBuffer);
  res.end();
});

This gives you full control over the agent's audio stream before it's sent to the user.
See: Custom TTS Integration Guide

Source:

Custom TTS Integration Guide

V

VapiAPP•11/18/25, 4:31 AM

I sincerely apologize for the delayed response to your support request.
We implemented better tracking for support tickets to ensure timely responses going forward, but due to a technical limitation, this only applies to tickets created after this past week.
I'm closing this older ticket, but if your issue still needs attention, please:
Reply with 'REOPEN' or
Email support@vapi.ai with any updates
I'll personally ensure your new request gets prompt attention.
Thank you for your patience.
Best regards,
Vapi Support Team

Pipe audio stream to another application

Similar Threads

Similar Threads

Similar Threads