Research and experiments by Ashwin Alaparthi, MTS at Vapi. Jev was developed by the TypeSafe AI team.
"I've long held the belief that while LLMs are great for general tasks, once you know a specific task which is repeatable, you can often build a small language model or just a classifier to do a lot of those tasks," says Ashwin. "And Jev is one of those models."
Jev is TypeSafe AI's early-access "System One" model. Instead of generating prose, it takes context plus focused questions and returns structured decisions such as a score or a yes/no probability.
For Ashwin, the combination of speed and context length made Jev especially interesting for voice AI.
"When you talk to a person, there are a lot of small decisions happening in your mind which are tough to translate to a pipeline like ours, especially given the latency considerations. But some of them might be possible with a model like Jev."
At Vapi, those decisions sit inside a realtime system that brings speech recognition, language models, voice generation, and tools together. Ashwin wanted to test one decision in particular: "Should the LLM output be saying this?"
The experiment: a second check before speech
"The experiment I built was guardrails," Ashwin says. "It's a highly requested feature, especially from customers in sensitive fields such as medical or finance."
Prompt instructions remain the first line of defense for a voice assistant. Ashwin gives a simple example: "Do not offer any medical advice." Models are generally good at following a clear rule like that. But, he adds, "if you forget to prompt it correctly or the context grows too long, the model might ignore these rules."
The prototype added a separate check to the response a voice assistant was about to speak. A developer defined a rule, and Jev scored whether the proposed response violated it.
"It's good to have a slightly more deterministic check on top of that. Jev lets us do that because it responds faster than speech can be generated, which makes it ideal to run in parallel with our TTS."
In one early test, Ashwin says, Jev caught "at least three out of four" responses that violated the evaluated rule. He is equally direct about what that result does not mean: "It's not a 100% compliance guarantee, but it's an additional check."
What the early tests taught us
We learned that the wording of a rule matters enormously. Copying an instruction directly from a larger system prompt often stripped away the context that made the instruction precise. A condition that applied only at one stage of a call could suddenly look universal, producing false positives.
Ashwin's notes recommend starting "plain and short." For the medical-advice test, using a simple rule like "Do not provide any medical advice" was difficult to beat.
When a rule needs more detail, his recommendation is just as concrete:
"Name the acts, and repeat the verb for each one. 'Do not recommend a medicine, do not recommend a dose, do not recommend a procedure.'"
Why repeat the verb? A list of nouns can make any mention of the noun look like a violation. The repeated action keeps the check focused on recommending rather than merely mentioning.
Another lesson was that the check has a narrow job. It was most useful when all of the following were true:
- The rule concerned something the assistant said, not an action it took.
- The rule applied throughout the conversation rather than only at one stage.
- The response itself contained enough information to judge the rule.
- A single violation would matter enough to justify the risk of an interruption.
A response-only classifier cannot know whether the assistant called the correct tool unless it sees the tool state. It cannot verify a claim against a knowledge base it was not given. It may also struggle with rules whose meaning changes across the stages of a call.
Finally, false positives can be as important as missed violations. In voice AI, a wrong interruption is not an abstract classification error because the caller hears it. "Every extra rule is another chance to misfire," Ashwin writes.
The prototype improved when rules were tested against a representative evaluation set before being used as active checks. Vapi's broader guidance on testing voice agents with focused Evals and end-to-end Simulations follows the same principle: one successful call is not enough evidence that a probabilistic system is reliable.
Promising, but not a guarantee
Ashwin's conclusion is deliberately modest:
"It definitely catches clear violations and sometimes misses subtle ones, so it can't be called a deterministic guardrail. That said, it can be an extra layer on top of prompt guardrails and not instead of."
A fast decision model may help with a carefully chosen set of high-cost, response-level mistakes. It should not replace clear prompts, evaluation, monitoring, or domain-specific controls. Teams working with sensitive data still need the appropriate security and privacy configuration for their use case. A model's output is not proof of compliance.
What did Vapi learn from testing Jev?
The short answer: "It's not a 100% compliance guarantee, but it's an additional check." Fast decision models look promising as a second check for narrow, response-level rules, but they do not turn voice AI into a deterministic system. Rule design, representative testing, and the cost of false positives matter as much as raw model speed.
For teams evaluating a similar pattern, the practical sequence is straightforward: start with a clear voice AI prompt and concise guardrails, turn critical behaviors into repeatable test coverage, and measure the new decision step as part of the full conversation—not as an isolated model benchmark.
Where else this pattern might help
Guardrails are only one possible application. Ashwin is already curious about other applications:
"Some ideas that come to mind are voicemail detection, given the first few seconds of a call. Jev is probably pretty good at recognizing if that is a voicemail or not, without the additional latency of a heavier LLM. It would be interesting to see how it performs."
He also wonders whether the same pattern could help with endpointing: deciding whether a caller has finished speaking.
So far these are hypotheses and need additional testing before consideration for addition to the Vapi platform. As Ashwin puts it, "We'll still need to run some experiments to see if these assumptions are true."
That is the broader idea worth exploring: use a general purpose model for open-ended language, conventional code for exact rules and permissions, and fast decision models for bounded judgments in between. Jev may give voice AI systems a way to make more of those small decisions without sacrificing the pace of a natural conversation.
