JuicyTalk
Nokia’s applied research team has open-sourced AnyJev, a Python library that turns an open LLM into a decision model. It needs no training. It targets a common production job: picking one answer from a fixed set instead of writing a sentence.
Is it deployable? Yes, it installs from PyPI, ships under Apache-2.0, and has transformers and vLLM backends with shared-prefix scoring.
What is AnyJev?
AnyJev borrows its interface from Jev. Jev is the System One decision model that TypeSafe AI launched in September 2026 (our coverage). You give AnyJev a typed question and get back a decision with a probability you can threshold. That probability is read from the model’s next-token distribution. Nothing is generated, parsed, or trained.
The library supports 3 question types:
- A
choicequestion picks one of K options. - A
noulquestion is yes or no. - A
scorequestion places the answer in one of several ordered bins.
The Problem With Reading Logits Directly
Many open projects already restrict the next token to the option labels and read the scores. The Nokia research team flags 2 flaws in that shortcut. First, the answer can change when the options are reordered. Second, the probabilities are not calibrated.
The levels doc names 2 causes:
- The first is prior bias: the model favors some labels, such as “Yes” over “No”, whatever the input.
- The second is position bias: the model favors certain slots in the option list.
How AnyJev Works: L0 and L1
Every decision carries a level field.
L0 (zero labels, on by default) applies 2 fixes:
- Cyclic shifts. For a question with K options, the list is shown in K rotations, so every option appears in every position once. The results are combined in log space as a geometric mean. If the position bias is additive in logit space, this removes it exactly.
- Prior correction. By default, AnyJev uses batch calibration. It keeps a running mean of the predicted distributions on real inputs and divides it out at strength 0.75. The correction starts after 8 items.
L0 costs K prefills per decision, batched over a shared prefix. That is about 0.25 s per decision at batch 32 on one H100, with K = 20.
L1 (100 to 500 labels per question) adds temperature scaling on top of L0. The fitted values are saved as a small JSON artifact. L1 reshapes confidence but does not change the ranking of answers.
Benchmark Results
On Qwen3-8B with BANKING77 (20-way, 300 test items), the numbers look like this:
| Metric | Raw logits | AnyJev L0 | AnyJev L1 |
|---|---|---|---|
| Labels required | 0 | 0 | 100 to 500 |
| Flip rate when options reversed | 0.230 | 0.073 | 0.077 |
| Accuracy | 0.747 | 0.803 | 0.807 |
| Calibration error (ECE) | 0.240 | 0.184 | 0.095 |
| Auto-decidable at 5% error | 7.7% | 46.3% | 52.0% |
A few other results from the repo:
- L0 reduced order flips on all 9 model and task rows tested.
- On a typed-decisions set, Qwen3-32B with L1 reached an ECE of 0.036, compared with 0.144 published for Jev. On accuracy, the fine-tuned Laya still leads.
- The full ablation table covers Qwen, OLMo, Granite, Phi and Mistral models.
- Wu says the team tried AnyJev on an internal Nokia routing problem and saw promising results.
How to Use AnyJev
# pip install "anyjev[hf]"
from anyjev import Decider, Question
from anyjev.backends.hf import HFBackend
d = Decider(HFBackend("Qwen/Qwen3-8B"))
route = Question.choice("Which team should handle this?",
["billing", "technical", "sales", "other"], name="route")
r = d.decide({"conversation": [...]}, [route])
r["route"].distribution # probabilities per optionFor serving, you start vLLM with prefix caching and point a VLLMBackend at it.
