LLMs are great at lots of things. You can point a foundation model at nearly any task today and get decent responses. The trade-off is that general intelligence is a hammer - it can do many things, but is not meaningfully specialized toward anything.
Enter Jev. Jev has ruled the discourse (at least for the last week or so), on the promise of obscene classification speed out of the box, at the bare minimum of cost. Data analytics has lots of use cases that demand fast, intelligent classifiers at much lower costs than LLMs!
We gave it a whirl and TLDR; will be shipping a bunch of new Jev functionality this week.
The test #
On a test classifying 1,000 products into the 26 categories in Omni’s Ecommerce Demo, Jev performed essentially identically to Claude Haiku on accuracy, while being dramatically faster and cheaper:
| Jev | Haiku |
Correct classifications | 736 / 1,000 | 739 / 1,000 |
Correct including near-duplicates | 773 / 1,000 | 767 / 1,000 |
Requests | 7 | 995 |
Wall time | 0.9 sec | 82 sec |
Cost at list price | $0.011 | $0.22 |
That’s roughly 90x faster and 20x cheaper, while producing effectively the same classification accuracy. And the reality is that Haiku wasn’t practical as a real-time classifier, because 82 seconds of latency isn’t an acceptable trade-off for interactive work. One second, on the other hand, opens all sorts of new opportunities!
Why Jev is interesting for Omni #
A surprising amount of “AI” work in analytics doesn’t actually require unstructured text generation. Consider questions like:
Which of these 26 categories does this product belong to?
Is this customer asking to cancel?
Is this feedback positive, neutral, or negative?
How frustrated does this customer sound?
Is this result important enough to trigger an alert?
Today, these tasks are often handed to a general-purpose LLM. That works, but it means paying for a model capable of doing far more than the task requires.
Jev is built specifically for these narrower decisions. Instead of generating text, it answers typed questions: true/false probabilities, choices among categories, or scores on a rubric, along with calibrated probabilities.
And our initial results suggest the specialization matters. On the 1,000-product test, Jev and Haiku agreed on 869 rows. Beyond those, each got roughly 45 classifications right that the other missed.
The errors were also instructive. Both models struggled when the correct category couldn't really be inferred from the product name. For example, Plus representing a size range or Active representing a use case. In other words, a meaningful portion of the remaining error appears to come from missing information rather than simply weaker classification. These are also areas where high-quality context and semantics may improve accuracy.
Some surprising learnings #
One particularly interesting difference is that confidence is a first-class output of Jev.
In our test, Jev returned confidence of at least 90% on 601 of the 1,000 rows. Within that group, it was 88% accurate.
That opens up a pattern that's particularly useful in analytics: we don't necessarily have to treat every AI answer equally.
We can make inexpensive decisions automatically when confidence is high, flag uncertain rows, or fall back to a larger model when the problem is ambiguous.
Going back to the product category example, instead of AI simply returning “Pants,” we can potentially know that the model thinks Pants has a 97% probability, and build product behavior around that distinction.
Where we think this fits in Omni #
The most obvious starting point is AI calculations.
Omni already supports functions like AI_CLASSIFY, AI_SENTIMENT, AI_EXTRACT, AI_COMPLETE, and AI_SUMMARIZE. The latter functions genuinely need a generative model. But classification and sentiment don't necessarily need one.
We can route those narrow tasks to a model like Jev while continuing to use larger LLMs where their capabilities matter.
More interestingly, System One models let us expose AI primitives that are much more natural inside analytics. Imagine:
AI_IS(comment, "the customer is asking to cancel")
returning 0.94, or:
AI_SCORE(comment, "Calm", "Frustrated", "Very angry")
returning a numeric score.
The same primitive potentially applies elsewhere in Omni: deciding whether a scheduled result is worth alerting someone about, making small routing and guardrail decisions inside our agent, scoring eval results, or automatically filling categorical input columns. These can be tied to alerting, flagging, and other operational workflows.
Where Jev appears differentially strong #
We're not thinking about Jev as a replacement for Haiku or other general-purpose models. It's a different tool.
Haiku is substantially more flexible: it can interpret complicated instructions, generate arbitrary text, summarize information, extract open-ended values, and reason through ambiguous tasks. Haiku is the standard for cheap, fast general AI.
Jev looks particularly compelling when the output space is known ahead of time.
If the question is “write something,” “explain something,” or “figure out what to do,” we still want a general-purpose model. If the question is “which one?”, “how much?”, or “is this true?”, a specialized model may be a much better fit.
At analytics scale, that difference adds up quickly. In this test, we went from 995 model requests to 7, 82 seconds to 0.9 seconds, and $0.22 to $0.011, without meaningfully sacrificing accuracy.
There are still things we need to validate - particularly run-to-run stability, batching behavior, confidence calibration across more datasets, and the data-handling requirements of adding another model provider.
But the initial result is promising: for narrow analytical judgments, we may not need to reach for a full LLM at all. System One models give us another point on the intelligence curve, one optimized for extremely fast, inexpensive, structured decisions, and that's a particularly natural primitive for a data product like Omni.
