On Tuesday, September 22, three Vapi product engineers walked through our three newest releases. Mhar Andal, Scott Lowe, and Adham El Fawal each explained why they built Model Intelligence, Simulations, and Campaigns, then demoed the primitives they shipped this year. Amanda, who leads developer relations at Vapi, hosted and opened the floor to live questions from builders on the platform. Attendees asked more than we could get to, so we've pulled the top questions together here, answered by the team.
Model Intelligence helps you pick the right transcriber, LLM, and voice combination
The first feature Mhar demoed was Model Intelligence. It answers the question we hear most from builders, in webinars, in the community, and in support tickets: "Which models should I use?" The Vapi Dashboard makes it easy to mix and customize model configurations, but without guidance, it's hard to know which combinations actually perform, and the best one shifts as your use case changes.
Model Presets recommend a configuration tuned to a specific goal, like cost, speed, or intelligence. When you want more control, every model now carries the data to assess it yourself: cost, latency, and a quality metric. For voices, the quality metric is our proprietary Humanness Index, shown directly on the platform.
Simulations build confidence with customizable AI testers for your voice agent
Most teams find out their voice agent has a problem in production. A customer complains, or a conversion metric slips, and someone pulls call recordings to work out what happened. By then, the issue has already cost brand trust, revenue, and hours spent tracking down a specific conversational failure.
Scott built Simulations so teams can ship voice agents they trust. Simulations put AI testers on the other end of the call, each with a distinct personality and a scenario drawn from your real customers. They hold full conversations with your agent, and every run returns a pass/fail result, a complete transcript, and a recording.
Campaigns make outbound dependable at scale
Adham rebuilt Vapi Campaigns from the ground up to make outbound dependable at scale. Give a campaign a contact list of up to 10,000 contacts, an assistant or squad, and a phone number, and Vapi dials through the whole list. Each call is tailored to its contact, dialing stays inside the window you set, and anyone who's no longer eligible — like a contact who already converted — gets skipped at dial time.
The top questions, answered
The Q&A ran long, so here are the questions that came up most, answered from what the engineers said live and expanded where the team had more to add.
Model Intelligence
How do I compare transcribers, and what metrics do you use?
Vapi measures transcription accuracy with word error rate, which tracks how often a transcript misses what was actually said. Model Performance Metrics shows it per transcriber alongside latency and cost, so you can compare options against your use case instead of guessing. If you'd rather not compare manually, the preset optimized for intelligence selects one of the most accurate transcribers for you.
How do you consider latency when selecting models for the presets?
Latency matters, but it's not the only thing we optimize for. We don't just pick the fastest models; we pick models that are fast and don't sacrifice quality. Other transcribers might shave off a couple hundred milliseconds, but the team chose a model it trusts to perform consistently over a small latency gain. Our latency figures also come from live calls, which can differ significantly from a demo environment but better represent what you'll see in production.
Simulations
Can I create, trigger, and evaluate simulations through the API instead of the dashboard?
Yes. Everything available for simulations in the dashboard is available through the API, including from tools like Codex or Claude Code. We prioritize parity between what you can configure in the dashboard and what you can configure via the API.
How are simulation test cases generated?
Test cases are built manually. You define the personality and scenario for each simulation and add it to a suite, which lets you decide exactly which scenarios to cover and which types of callers your agent needs to handle.
Where do I configure the LLM judge, and how reliable is it given that LLMs are non-deterministic?
The judge runs through Vapi's structured output primitive, so any model you can assign to structured outputs also works as the judge. Outcomes are non-deterministic; a single run isn't a strong signal. Iterations support up to 10 runs per test case, which gives you multiple data points before you trust a result, especially ahead of a production deploy.
Can I manually test my agent instead of only running pre-built simulations?
Yes. From the assistant page, you can start a web call or dial in over the phone and talk to it directly. The team frames testing as three layers: simulations check full-conversation outcomes, evals check specific in-conversation decisions, and manual testing confirms the agent actually feels right. None of the automated layers replace picking up the phone yourself, and we recommend manually testing every agent before it ships.
Campaigns
Why does Vapi recommend low concurrency for outbound campaigns?
Answer rates vary by time of day, so spacing calls out keeps a campaign from landing entirely in a bad window, like lunchtime, and leaves room to catch and cancel a misconfigured run before it finishes.
Does Vapi manage max concurrency automatically?
Not yet, but planned. If you launch campaigns above your subscription's concurrency limit, the extra calls will error out. For now, use the concurrency slider to stay within your limit; automatic management is planned for a future release.
What's the best way to handle first-turn latency on an outbound call?
Set a hardcoded first message on your assistant instead of letting it generate one live. The opening line is often the slowest for an assistant to produce, and on an outbound call it's also the moment that decides whether the person stays on. A fixed first message removes that delay, and the conversation flows more naturally from there.
Wrapping up
Each of these features started with something builders told us they couldn't do: pick a model with confidence, prove an agent works before it hits production, run outbound at scale without building the plumbing. That's how we prioritize features at Vapi, and the questions from this session are already shaping what comes next. If you asked something we didn't get to, we sent the full recording to every registrant, and you can always find us in the community.
