Stop Parsing LLM Text for Yes/No: TypeSafe Jev on FastRouter

A common pattern in LLM-powered routers and classifiers looks like this: ask a chat model a yes/no question, demand "ONLY YES OR NO," then write a parser that tries to recover a boolean from whatever the model produced that day.

It works right up until it doesn't. And when it breaks, it usually breaks silently: a model update changes phrasing, someone bumps temperature, an edge case includes both "yes" and "no" in the explanation, and the system keeps running while quietly misrouting tickets, moderation calls, or actions.

The fix is boring and practical: stop using a text generator as a boolean API.

TypeSafe's Jev models are built for structured evaluation. A request sends state plus typed questions and gets typed answers plus probabilities back. No prose, no parsing, no "respond in valid JSON" prompting. FastRouter exposes Jev through a dedicated endpoint, using the same API key, the same billing surface, and the same Activity Log as every other model call.

TL;DR

Regex or string-matching a chat model's output to make a routing decision is a production footgun. It's worth replacing.
Use Jev when the output needs to be typed: noul (yes/no, returns P(yes)), choice (enum, returns a full distribution), score (rubric, returns a weighted score plus distribution).
Treat outputs as probabilities, not verdicts. Thresholds and an abstain path handle the messy middle better than a single cutoff.
Batch multiple questions against the same input. One call, multiple decisions, less latency and overhead than separate requests.
Start with typesafe/jev-latest, then pin to a specific version once behavior is validated (the response reports exactly which version served the request).

The production pain: free-form text is a fragile interface

The common pattern looks like this:

Prompt: "Is this message urgent? Answer ONLY YES or NO."
Parse the output into a boolean.
Add a fallback parser for "Yes.", "YES ✅", "No (not urgent)", and similar variants.
Patch it again when a provider or model update lands.
Eventually ship a bug where a valid but unexpected output routes real traffic incorrectly.

Failure modes that show up repeatedly in production systems built this way:

Format drift — "YES" becomes "Yes." or "Yes, urgent." and a strict parser fails.
Parser poisoning — user content itself contains "YES/NO" and the extractor grabs the wrong token.
Ambiguous dual answers — "No, unless you consider payouts failing urgent," which satisfies no simple parser.
JSON mode isn't a full escape hatch — even with forced JSON, teams still validate, retry, and handle partial or invalid payloads across edge cases and model changes.
Silent misroutes — nothing crashes, metrics may not even spike, traffic just routes wrong.

When the actual need is a boolean, an enum, or a score, asking for text and hoping it behaves is the wrong abstraction.

What Jev is, and what it isn't

Jev is not a chat model that happens to be good at following formatting instructions. It's an evaluation model: it takes the content to evaluate (the state) and a set of typed questions about that state, and returns typed answers.

What it does well:

Stable machine interface — booleans, enums, and scores with no output parsing.
First-class uncertainty — probabilities and distributions that support real routing policies, not just a top-1 guess.

What it doesn't do:

It doesn't generate user-facing text, explanations, summaries, or tool plans. It isn't a generator.
It doesn't make classification correct on its own. Good criteria, thresholds, monitoring, and version control still matter.
For tasks that need heavy multi-step reasoning, a reasoning-focused chat model may still be the right tool for the reasoning step, with Jev handling the final typed adjudication.

The API contract: state plus questions

FastRouter exposes Jev on a dedicated endpoint:

POST
Model: typesafe/jev-latest (an alias), or a pinned Jev version once behavior has been validated.

The payload has three parts: state (the content being evaluated, plain text or structured JSON), and questions, a map of question objects each carrying a type (noul/choice/score), instructions describing what to evaluate, and optional criteria defining boundaries, options, or a rubric.

A detail that catches teams early: question keys are identifiers for the caller's own code. The model never sees them. Naming a key "is_urgent" versus "q1" makes no difference to the model — everything the model needs to know belongs in instructions and criteria, not the key name.

Quickstart: one yes/no decision, no parsing

1curl https://api.fastrouter.ai/api/v1/systemone \
2  -H "Authorization: Bearer $FASTROUTER_API_KEY" \
3  -H "Content-Type: application/json" \
4  -d '{
5    "model": "typesafe/jev-latest",
6    "state": "Help! My payouts have been failing for 3 days.",
7    "questions": {
8      "is_urgent": {
9        "type": "noul",
10        "instructions": "Does this convey urgency?"
11      }
12    }
13  }'

The response returns a probability, not a label:

1{
2  "model": "jev-1.13.0",
3  "answers": {
4    "is_urgent": { "type": "noul", "noul": 0.95 }
5  },
6  "usage": { "input_tokens": 296, "output_tokens": 20 }
7}

Two details worth noting: the response's model field reports the exact version that served the request, since typesafe/jev-latest is an alias; and noul returns P(yes), leaving the calling code to decide what to do with 0.95.

Calling it directly over HTTP

The OpenAI SDK is a good fit for FastRouter's standard OpenAI-shaped endpoints. For /systemone, the reliable approach is a direct HTTP call rather than an SDK method that doesn't exist for this endpoint. A Python example using httpx:

1import os
2import httpx
3
4FASTROUTER_API_KEY = os.environ["FASTROUTER_API_KEY"]
5
6def jev_eval(state: str) -> dict:
7    url = "https://api.fastrouter.ai/api/v1/systemone"
8    payload = {
9        "model": "typesafe/jev-latest",
10        "state": state,
11        "questions": {
12            "is_urgent": {
13                "type": "noul",
14                "instructions": "Does this convey urgency?",
15                "criteria": {
16                    "true": "User indicates time pressure, blocked",
17                    "false": "General question or non-blocking issue"
18                }
19            }
20        }
21    }
22    resp = httpx.post(
23        url,
24        headers={"Authorization": f"Bearer {FASTROUTER_API_KEY}"},
25        json=payload,
26        timeout=30.0,
27    )
28    resp.raise_for_status()
29    return resp.json()
30
31data = jev_eval("Help! My payouts have been failing for 3 days.")
32p_yes = data["answers"]["is_urgent"]["noul"]
33
34if p_yes >= 0.9:
35    action = "auto-escalate"
36elif p_yes <= 0.1:
37    action = "normal-queue"
38else:
39    action = "manual-review"

The equivalent in Node/TypeScript, using fetch:

1async function jevEval(state) {
2  const res = await fetch("https://api.fastrouter.ai/api/v1/systemone", {
3    method: "POST",
4    headers: {
5      Authorization: `Bearer ${process.env.FASTROUTER_API_KEY}`,
6      "Content-Type": "application/json",
7    },
8    body: JSON.stringify({
9      model: "typesafe/jev-latest",
10      state,
11      questions: {
12        is_urgent: {
13          type: "noul",
14          instructions: "Does this convey urgency?",
15        },
16      },
17    }),
18  });
19  if (!res.ok) throw new Error(`FastRouter error: ${res.status}`);
20  return res.json();
21}

The three question types

Noul: yes/no, returned as P(yes)

This replaces "return YES or NO" prompts, for things like whether a ticket is urgent, whether text violates a policy, or whether a message requests a refund. The output is a float from 0 to 1.

A common mistake is treating 0.51 as "yes." That's a coin flip with extra steps. A policy that holds up in production looks more like: auto-act above 0.9, auto-reject or auto-allow below 0.1, and abstain to human review or a second pass in between.

Criteria matter here. "Urgent" alone is vague; adding criteria.true ("money blocked, payroll impact, outage, security incident, explicit time pressure") and criteria.false ("how-to question, feature request, minor bug, no time pressure") defines the decision boundary explicitly instead of leaving it to guesswork.

Choice: pick one option, get the full distribution

This is the enum classifier: routing to a team, picking a category, choosing a policy bucket. Up to 255 options can be defined, and the model returns a probability distribution across all of them, not just the top pick. A distribution like 0.93 billing / 0.04 technical / 0.03 sales supports auto-routing; 0.41 / 0.39 / 0.20 is a clear signal to abstain to a general queue instead.

A large flat list of options with weak descriptions tends to produce flat, unhelpful distributions. For a taxonomy with dozens of labels, hierarchical routing, a coarse category first, then a second choice question inside the winner, works better than one enormous question.

Score: rubric scoring with a distribution

This fits ordinal ratings rather than categories: frustration or sentiment intensity, risk level, grading an LLM output for helpfulness or completeness. The useful part is the distribution, not just a single integer, which makes it possible to route boundary cases (uncertain 2-3 splits, for example) to review rather than trusting a rounded average.

Batching questions: one call, many decisions

Most real pipelines need more than one signal at once. A support triage pipeline typically wants urgency, department, and frustration together, and all three can be asked in a single request:

1curl https://api.fastrouter.ai/api/v1/systemone \
2  -H "Authorization: Bearer $FASTROUTER_API_KEY" \
3  -H "Content-Type: application/json" \
4  -d '{
5    "model": "typesafe/jev-latest",
6    "state": "Your product charged me twice and nobody has
7      responded in 48 hours. Fix this now.",
8    "questions": {
9      "is_urgent": {
10        "type": "noul",
11        "instructions": "Does this need fast handling?"
12      },
13      "department": {
14        "type": "choice",
15        "instructions": "Which team should handle this ticket?",
16        "criteria": {
17          "billing": "Payments, invoicing, refunds, chargebacks",
18          "technical": "Bugs, outages, API errors, integrations",
19          "sales": "Pricing, upgrades, plans, new accounts"
20        }
21      },
22      "frustration": {
23        "type": "score",
24        "instructions": "How frustrated is the user?",
25        "criteria": ["Calm", "Frustrated", "Very angry"]
26      }
27    }
28  }'

One request, three typed outputs, no parsing.

Where Jev fits across an agent's loop

Structured evaluation isn't limited to intake triage. The same typed-question pattern applies at several points before, during, and after a model call inside an agent loop: screening input before it reaches the model, gating which tool calls are allowed to execute, deciding when a loop is actually done, routing each step to the right model tier, checking whether a final answer is grounded in its sources, and deciding what's worth writing to memory. The diagram below maps nine of these decision points against where they sit in that loop.

Concept adapted from Brij Kishore Pandey's (@brijpandeyji) agent-loop framework for Jev, redrawn here against FastRouter's own model catalog and endpoint.

Why not just use structured outputs or function calling on a chat model?

That approach works for many cases, and often should be the default. But a few things tend to show up in real systems that rely on it exclusively.

Output stability isn't the whole story. Structured outputs cut down on invalid JSON, but prompt sensitivity, model version changes under the same schema, and the need for downstream validation and retry logic don't disappear.

Probabilities are first-class in Jev. A chat model asked to add a confidence field will often return 0.99 while being wrong, because that confidence is just another generated token. Jev's P(yes) and distributions are structurally different: they support thresholds, abstention, margin checks between the top two options, and drift monitoring over time.

The tradeoff: Jev won't explain itself. A product that needs the model's reasoning behind a decision ("why is this a policy violation?") still needs a second, generative call for that. That's a reasonable separation of concerns rather than a gap: Jev decides in a stable, typed way, and a chat model explains for a human audience when an explanation is actually needed.

Why run it through FastRouter

One API key to manage, the same one already used for chat and completion calls.
One Activity Log view for latency, tokens, and cost across every model, Jev included.
One billing surface instead of a second vendor relationship and a second invoice.

Failure modes worth planning for

Vague instructions produce useless 0.4-0.6 outputs. "Is this bad?" invites a mushy probability. Writing criteria like edge-case tests, not just the happy-path center, fixes this.
Treating a probability like a boolean. Auto-acting above 0.5 means being wrong often. Conservative thresholds plus an abstain path hold up better.
Aliases move under a fixed threshold. typesafe/jev-latest tracks the newest release, so a tuned threshold can drift as the underlying model changes. Pinning the validated version (using the response's model field as the reference) avoids that.
Too many labels in one choice question. A large flat taxonomy produces flat, unhelpful distributions. Hierarchical classification or sharper per-option criteria fixes it.
Treating keys as prompts. Keys are never seen by the model. Meaning belongs in instructions and criteria; keys exist only for the calling code's readability.

A practical starting checklist

Find one place today where a chat model's output is being parsed for a routing or classification decision.
Replace it with one Jev call to POST using typesafe/jev-latest.
Batch the two to four questions already needed (urgency, routing, sentiment) into that one call instead of separate requests.
Write criteria from real logs and edge cases already on hand, not invented ones.
Implement thresholds with an abstain path: high confidence auto-acts, low confidence auto-rejects or auto-allows, everything else goes to review or a slower pass.
Shadow-run the old path and the Jev path side by side for a few days, log disagreements, then cut over.
Pin the served model version once behavior looks right, and treat a version bump as a change that needs re-validation.

Getting started

Full parameter reference, structured-state examples, and response schemas are documented at https://docs.fastrouter.ai/explore-features/openai-decisions-and-typesafe-systemone/typesafe-systemone-jev.

Stop Parsing LLM Text for Yes/No Answers: TypeSafe Jev on FastRouter