I wanted to try TypeSafe because it promises to make small AI decisions much faster and cheaper.
I had two pet projects to try it: byownai.com, an AI tools BYOK and subs directory, and a personal shopper comparison tool.
Both involve a lot of classifications for known states, so it made sense to have a tool that does not generate text to check it.
For instance: Does a documentation page support the claim I’m about to add to the directory? Is that cheap laptop the configuration I’m looking for? I wanted to see whether TypeSafe's Jev could take some of that work off my plate and other cheap LLMs’ plates.
I compared Jev with GPT-5.6-Luna, GLM-5.3-Flash, Muse Spark 1.3 Contributor, MiMo-V2.5 and DeepSeek V4.1 Flash. They worked through the same 6,600 decisions from my two projects, with no external benchmark dataset.
TL;DR
Jev was the fastest and cheapest at published API rates across 6,600 decisions from my two projects: a median of 338 ms per 24-decision batch and $0.0856 for the full run. Across both tasks, Jev agreed with 300 of all 320 reference answers (93.8%); Muse led with 311 (97.2%). This is the full labelled sample, not all 6,600 decisions. Jev makes these checks a bit faster and a much cheaper, though it wasn’t the most accurate model in this sample.
Parsing setup
For BYOwn AI, I used 213 source pages and asked eight focused questions about each named tool: API keys, provider subscription sign-in, local models, MCP client, MCP server, a self-hosted backend, a CLI and a desktop app. That gave 1,704 checks, each answered yes, no or not stated.
“Not stated” is useful here. A page can name a provider without establishing which features work or how a particular connection is billed. I asked about one explicit feature at a time, using only the supplied excerpt.
For the shopping tool, I used all 612 successfully captured listings in its saved collection. Each received eight checks: item type, condition, chip family, installed RAM, installed storage, screen size, battery evidence and functional defects. That gave 4,896 decisions, with defined options for each question and a way to leave missing information unknown.
These are 6,600 different checks on 825 sources from the same saved collections. The questions are narrower and better suited to Jev’s small, independent judgments; the source pool itself has not grown.
The collection also contained accessories, old machines and unrelated search results. Those belong in this workload because the shopping tool has to reject them. They make much of the test easy, though. This is one saved collection, not a balanced examination of every possible computer configuration.
What Jev does differently
Jev takes evidence and questions with defined answer options. Its Choice primitive returns the selected options and their probabilities. It does not need to write a paragraph and then have my code interpret it.
I gave the other models the same evidence, questions and criteria, and asked for JSON containing only their choices. Independent questions travelled together in batches of 24, covering three sources per request. That produced 275 requests per model. The counts below are different decisions, not repeated runs of the same handful of examples. Jev used native typed choices; the other models were prompted for JSON without schema-enforced decoding. The comparison reflects those configurations.
Did the answers hold up?
A fixed random sample of 24 listings and 16 documentation pages was labelled by an agent before any candidate calls. That gave 320 reference decisions, with no independent human grading. Twelve ambiguous labels were flagged before the run.
On the 192 listing checks, Muse agreed with 188 reference answers, Jev with 181, GLM with 179, Luna and DeepSeek with 178, and MiMo with 170. Predicting the most common reference label separately for each question type would score 100/192.
On the 128 documentation checks, DeepSeek agreed with 124 reference answers, Muse with 123, Luna with 122, Jev with 119, MiMo with 117 and GLM with 111. The equivalent majority-label baseline was 103/128. All six beat it. This measures support in the supplied page, not whether an integration works in practice.
The small reference sample still matters. Excluding the 12 pre-flagged ambiguous labels, Jev agreed with 294 of 308 answers, Luna with 295 and Muse with 300. These are checks on a sample, not an accuracy score for all 6,600 decisions.
For example, one listing was marked “new” but described the laptop as almost unused. That conflict was flagged in advance for the separate ambiguity check.
Jev and Muse returned the expected choices in all 275 batches. Removing a single Markdown fence, using the same rule for every model, made another 127 MiMo batches, three GLM batches and one DeepSeek batch valid. Some answers still used an option outside the allowed categories or omitted a requested choice; those count as disagreements when they fall in the reference sample.
The time and cost of the same batch
Jev’s median batch took 338 ms. The other medians ranged from 1,609 ms for DeepSeek to 7,664 ms for MiMo. These are batch times, with 24 decisions in each response.
For the full 6,600-decision workload, Jev’s calculated token cost was $0.0856. The five general models ranged from $0.2038 for Muse to $0.4717 for Luna.
These are complete response times measured from the same machine, including network and provider overhead. Each model processed one request at a time, while models could run concurrently. Jev used TypeSafe’s API; the five competitors used OpenCode Go. The results compare those routes as used, rather than isolating inference speed.
I disabled reasoning where the route supported it, requested low effort for GLM and minimal effort for Muse. Output and reasoning tokens are included in the cost calculation. Every dollar figure uses published API rates applied to returned token counts, including reported cache reads and writes. TypeSafe pricing, OpenCode model rates.
Muse’s price is its Contributor rate, which permits training on submitted content. This test used public source material. Cache writes are included where returned, and DeepSeek uses the published off-peak rate applicable during the run.
Fast classification opens up new possibilities?
What interested me about Jev is how cheap and fast it makes experimentation. At $0.0856 in calculated API token costs for these 6,600 decisions, I can try different classification ideas on my pet projects for virtually nothing and focus my existing subs on other type of work. Responses in a few hundred milliseconds also make it easier to imagine those checks happening while I’m exploring a catalogue or comparing listings.
My current projects are small and not time-sensitive on the other side. I already use subscriptions that feel “subsidized” compared with paying published API rates, and I can usually find a decent free model on OpenCode for simple classifications. Jev would not reduce those subscription bills, but it gives me another way to build fast checks with typed answers and probabilities I can use in code.
The documentation results still leave me work to do on the questions and review process. The low token cost gives me room to experiment with both. I’ll keep playing with Jev, trying new classification steps and seeing where the faster responses make a difference.
At the moment I do not see the gigantic hype on those type of tasks, and I will keep using Deepseek 4.1 or similar types of models.
If subscriptions become more restrictive or free model access gets harder to find, I’ll revisit the economics too.