Picture a software company that sells Medicare appointment scheduling technology to hospitals. The hospitals license the product and run it as their own, and the company's engineering team builds everything under the hood. That team decided to embed a voice agent in the product: patients would call, and the agent would book them straight into the hospital's schedule through the scheduling system the company already built. It was a product feature, scoped and staffed by the company's own engineers, who had shipped complex healthcare integrations before. Building the voice layer in-house looked reasonable. Six weeks later, they had a working demo. One server, one call at a time, and it sounded great in the conference room.
Rolling the feature out to their hospital customers came with a different set of requirements. 200 concurrent calls at peak hours across every hospital running the product. Low-latency responses across audio, transcription, intelligence, and voice models. Failures that route to the front desk instead of dropping a patient's call. By month three, the engineering team had spent a full quarter on infrastructure problems nobody scoped. The year's budget ran out in month six. When they looked back at what they'd built, the voice agent itself was maybe 20% of it. The other 80% was plumbing and infrastructure.
They eventually moved the feature onto a platform. The migration took six weeks, and they shipped three new voice use cases to their hospital customers the following quarter.
If you're deciding how to bring voice agents into your product or your operations, that story is the decision in miniature. Here's how to make it deliberately instead of discovering it eight months in.
The biggest decision is orchestration
Teams deciding whether to build a voice agent are actually deciding whether to build an orchestration layer. The agent is the visible part, the tip of the iceberg. Underneath it sits real-time audio infrastructure that processes continuous streams rather than discrete requests, where a 500-millisecond hiccup that's invisible in a web application creates an awkward pause in a conversation and steals credibility. Speech recognition and synthesis span multiple providers with different accuracy, latency, and pricing, and your system has to fail over between them when one degrades. Telephony is its own domain: SIP trunking, audio codecs, DTMF tones, carrier-specific behaviors. That's specialized knowledge that takes months to acquire, and years to refine. Add state management so context survives transfers between agents, and observability that can trace a failed call through every component to find out whether the transcription misheard or the model hallucinated and how to fix it.
Then come the problems nobody puts on the roadmap, the ones that can truly make a model human. Barge-in handling when callers interrupt. Turn-taking detection that can tell a pause from a finished thought. Latency work where 50-millisecond improvements matter. Keeping it all consistent across hundreds of concurrent calls, even when each call has its own unique complexities.
Voice is not chat with audio attached
If you already run a chat product or a text workflow, the instinct is to treat voice as an extension of it. It isn't, and has many nuances that must be treated very differently. Latency and interruption handling alone make it a different problem, and endpointing (knowing when the caller has actually finished speaking) adds a class of bugs that chat doesn't have the nuances to deal with. Teams that bolt voice onto an existing product tend to underestimate this, and the underestimate is exactly what turned the scheduling team's six-week demo into an eight-month project.
The build never ends
A production voice pipeline isn't a one-time build. Models, telephony carriers, and transcription providers change constantly, so an in-house pipeline needs dedicated engineers permanently, not just for launch. That's the cost that breaks build-vs-buy math done on launch estimates. Every engineer maintaining telephony is an engineer not building your actual product.
When building it yourself makes sense
It does, sometimes. Requirements that need deep modifications to the audio or real-time pipeline that no platform can accommodate, confirmed by genuine investigation rather than assumption. Significant existing voice infrastructure, where extending what you have costs less than replacing it. Regulatory constraints that prevent third parties from processing your data. Voice AI as your core competitive advantage, where owning the whole stack is the point.
Most teams that choose to build don't fall into any of these categories. They choose it because they underestimate the complexity, not because their requirements demand it.
When a platform makes sense
Eight months becomes eight weeks. Your engineers spend their time on conversation design and business logic instead of buffer management. Pre-built integrations cover the common scenarios (CRM, ticketing, scheduling, payments), multi-agent orchestration comes built in, and provider flexibility means you can swap speech recognition, synthesis, or language models as better ones ship, without rewriting your stack.
If voice is a feature, not your product
The decision is sharpest for SaaS platforms adding voice to an existing product, like the scheduling team above. You already own the workflow and the data. Voice is a new interface, not a new business. Building that interface in-house means hiring dedicated voice engineers and having a 6-to-12-month runway before you know whether customers even want the feature. Embedding a platform means shipping in weeks with the team you already have, measuring adoption, and expanding as usage grows. Your customers experience the voice agent as part of your product, under your brand, and the infrastructure stays invisible.
The platform is a team, not just code
When you build in-house, your voice expertise starts at zero and grows only as fast as your own project. When you buy, you inherit the accumulated experience of people who have built their careers on this one problem: dedicated engineers who work on latency, telephony, orchestration, and turn-taking full time, plus forward-deployed engineers and consultants who have taken voice agents to production and know where the failure modes hide. Behind them sits pattern recognition you can't shortcut, drawn from over a billion calls and a developer community large enough that someone has already hit whatever edge case you're about to. Every problem another team surfaced becomes a fix your agent inherits.
The lesson
The scheduling team could build all of it. The eight months were still a mistake, because the goal was booking patients and the time went to infrastructure. Platforms solve the infrastructure problem once, so your team can focus on the agent. Don't spend eight months on someone else's problem.
This post adapts chapter 4 of the Voice Agent Playbook, which covers the platform vs. build decision in full, including the complete evaluation checklist and the rest of the strategy for getting voice agents into production. Read the playbook here.
