Output validation
Check a model's output against a rubric before it reaches a user.
138 rows are filed under this pattern, 136 of them with code. Rows are listed as the catalogue orders them: official first, then rows with code, then by stars.
Filter and search them in the catalogue →
Rows marked “not Jev itself” or “shadow mode” stay listed, but they are not examples of deciding with Jev.
-
Cookbook: Double-checking citations
Catches wrong or invented citations against the source document with one Choice, using its confidence to flag borderline cases for review.
-
Cookbook: Guardrails for LLMs
Screens every message in and out of an LLM app in one request, naming hazards and scoring how much harm complying would do.
-
latitude-llm
Open-source observability for AI agents that flags where agents fail in their traces. A shadow provider asks Jev one noul per flagger dimension, at most 11 per call; the cited file does not show whether those answers change any flag, and the README does not mention Jev.written by a model
-
reticle
Checks an agent's "done" by driving the running app and returning pass, fail or cannot tell. Its benchmark harness asks Jev, in one call on captured evidence, five nouls (did the intended consequence happen, is the capture clean, did the app error), a pass/fail/unknown choice and a severity score.written by a model
-
atomic
Coding-agent runtime in which workflows are versioned TypeScript stages with executable checks, review gates and human approvals. Its AI package lists Jev from models.dev as a System One decision model on five providers, apart from the chat models; the README does not say where a workflow calls it.written by a model
-
abide
Hooks into Claude Code, Codex, OpenCode and Pi and, on every edit or turn, asks Jev one yes-or-no question per rule in AGENTS.md or CLAUDE.md, showing only the rule and the diff; a broken rule goes back to the agent to fix. On 93 replayed sessions Jev flagged 39 edits; a reviewer confirmed 10.written by a model
-
jev-mcp
A ready-made judgement toolbox for agents: fact verification, content screening, semantic ranking, classification and extraction as separate tools.
-
vexjoy-agent
Toolkit of 44 agents, 63 skills and 79 hooks routed from plain-English requests. Its /d router asks Jev to classify a request and gates dispatch on whether the route keeps the requested outcome; its auto-compact plugin, past 60% context, asks three nouls per unclear tool call and prunes verbatim.written by a model
-
perch: semantic code linting
Tree-sitter finds and ranks methods, then user-authored YAML rules compile into nouls, with severity read as the rubric's expected value rather than the top band.
-
jev-review
A local-first MCP plugin for continuous code-quality review by coding agents.
-
supercov
Code quality and coverage judgements for coding agents, in Rust.
-
jev-guard
Security hook for coding agents: before each tool call Jev judges how much harm it could do, whether the user asked for it and whether it follows untrusted text, and code denies, asks or allows; afterwards Jev scans the result for prompt injection. For Claude Code, Codex, Cursor and more.written by a model
-
Canny
Guards against a coding agent claiming it finished: reads tool output, the diff and test results, then judges whether the completion claim holds.
-
jev-lint
Linter for what parsers miss: ast-grep picks the code, Jev is asked each rule's one-sentence question about every match, such as whether the comment above claims something untrue of the code, and matches over the cutoff are flagged. 99 rules ship, from 48 questions; judged code goes to the API.written by a model
-
Jevlint (codegirl-007)
Linter that checks code against plain-language rules: Tree-sitter extracts each code unit and Jev answers one `choice` per rule (pass, fail, optionally skip or abstain); a failure is reported only when its confidence reaches the rule's floor. Other backends: OpenAI, OpenRouter, Cloudflare.written by a model
-
fastbrowse
Browser agent: each page is indexed into candidate controls and Jev chooses the next one, so it cannot click what is not there; a Gemini Flash LLM plans and reads, and answers cite the page. Its maintainer recorded 99.2% on 46 tasks against 97.8% for hosted Browser Use, at under a tenth of the cost.written by a model
-
jev-eval-agent
An agent that routes evaluation work through typed decisions.
-
formanator
Submits Forma benefit claims from the command line or an MCP client. With --category-provider jev, Jev picks the benefit and category with a choice over the pairs Forma allows plus no_match, seeing only merchant and description; under 0.5 confidence or no match it falls back to the configured LLM.written by a model
-
oxlint-plugin-jev
An oxlint rule that is a yes/no question about a function, call, JSX element or file: each match goes to Jev, and the plugin reports an error when the yes-probability clears your cutoff.
-
jev-libero
Fine-grained robot control with Jev, physics previews, and configurable LIBERO tasks.upstream description
-
jgrep (npm: jevgrep)
grep for what code does: one Noul per code chunk, diff hunk or CSV row, printed as file:line hits with probabilities. --diff gates a PR in CI on a rule written in English (exit 0 match / 1 clean / 2 error); --tests lists the test files a diff can affect.
-
jev-code
Jev, TypeSafe's System One classifier, as a tool inside Claude Code, Codex, Pi, and OpenCode: typed classify, check, score, rank, and ask, plus one-command setup.upstream description
-
jev-recruiter
A Jev powered LinkedIn recruiting agent. Watch it browse relevant profiles, save links, and review evidence against your hiring brief.upstream description
-
vibecheck
Chrome extension: vibe-check your X posts with TypeSafe's Jev before you hit Postupstream description
-
hermes-jev
Typed System One decisions, ranking, verification, and an opt-in Hermes tool gate using TypeSafe Jev.
-
jev-reviewer
Data extraction for systematic reviews, quoted from the papers. Ask a trial report and its supplements your extraction form or a RoB 2, ROBINS-I, QUADAS-2 or TIDieR template; Jev points at the lines, every answer is a verbatim quote with its page, you check it and export the table. Files stay i
-
jev-belay
Claude Code Stop hook that blocks an unverified done: reads the transcript for evidence, asks Jev once, fails open on everything elseupstream description
-
jev-cua
Voice and text control for macOS. One floating bar, live UI action selection with Jev, and a continuous observe–act–verify loop.upstream description
-
snifftest
A prose linter that sniffs out AI writing tells. Zero dependencies, countable rules plus one judgment model.upstream description
-
yoshi
Context-pruning proxy for Claude Code and Codex: Jev judges which history is still needed, measured not claimed. POC here now, heading soon into https://github.com/compozy/compozyupstream description
-
smartmoney-cub
Read-only trading journal and review harness: Jev typed judgments, agent integration, and a reproducible finance benchmark. No orders, no advice.upstream description
-
claude-jev
Claude Code and Pi plugin that sends small judgments to Jev: whether an edit breaks a rule in CLAUDE.md (blocked with a file:line citation), which context blocks to keep verbatim at compaction, a model tier per subagent with nouls on its brief, and a routing hint per prompt. Every hook fails open.written by a model
-
invalidate
The invalidation layer for AI memory. Every fact gets a lease; new evidence ends it. Built on TypeSafe Jev.upstream description
-
jev-column-race
Jev vs Gemini 3.8 Flash: labelling 1,000 app reviews, 4.1× faster and 7× cheaperupstream description
-
cmd-mod-jev-nudge
Command Code mod: nudges the agent to keep going when it stops with work left, judged by Jevupstream description
-
jev-feels
Semantic decisions as ordinary Ruby — feels?, decide, score, Rails validations and pattern matching powered by Jev
-
jev-rag-benchmark
Reproducible benchmark for measuring Jev reranking quality, latency, and cost in RAGupstream description
-
jev-commit
A pre-commit hook: one call judges whether the commit message matches the diff.
-
jev-e2e
Natural-language end-to-end tests for web apps, powered by Jev and Playwright.upstream description
-
jev-lint
Fuzzy linter for coding agents: after an agent writes a file, each of the team's plain-language rules is asked as a Jev noul, and about 0.3 s later the agent hears which rules it broke. Can generate rules from the repository; Cloudflare and OpenAI judges as alternatives.written by a model
-
typed_evals
Fast, typed, calibrated evaluations for LLM and agent outputs, powered by Jev — with simple, framework-agnostic Python APIsearlier upstream description
-
patdown
Block, steer, and "fuzzy lint" with Jev to make agents follow your rules and conventions. CLI, github action, pi package, and more. Built with Effect.
-
dsh-jev-tools
DeepSeek Harness plugin asking Jev, before tool output enters the context, which segments of a long read or fetch matter (kept by rank, not threshold), whether a fetched page holds instructions aimed at an AI (warned at 0.75, left unchanged), which skill to suggest, and if a done claim has evidence.written by a model
-
jevlint
Configurable semantic linting powered by Jev, with file-level NOUL judgments and a magic-strings plugin.upstream description
-
pi-heed
Runtime constraints for the pi coding agent: checks every side-effecting tool call against what you said, before it runs. Powered by TypeSafe Jev.upstream description
-
Sedum (npm: sedum-cli)
Plain-English browser end-to-end tests next to Playwright code. In goal mode a Jev Choice picks each next operation (click, type, done or blocked) and its target; authored steps use a Choice per sentence; each verify claim is two Nouls (holds, contradicted). Playwright acts; code applies thresholds.
-
jev-browser
One grounded Jev/Playwright core: typed SDK, persistent CLI, and MCP server with native browser operations and deterministic assertions.upstream description
-
lejudge-jev-jepa
Natural-language constraints for JEPA world-model planning, judged by a decision model instead of an LLM.upstream description
-
augustus
Agent skill for the decision-model class (classifiers, encoders/decoders, specialized AR heads, System One). TypeSafe Jev is the dominant exemplar. Composition algebra, question design, validation gates. MIT.
-
jev-enforce
Claude Code plugin that checks every reply and file edit against your CLAUDE.md or AGENTS.md: one Jev noul per rule in a single request, and broken rules are quoted back as a Stop or PostToolUse block so Claude fixes them in the same turn. Also a CLI for CI.written by a model
-
jev-pref
Turn your AGENTS.md preferences into a fast, Jev-powered AI linter.upstream description
-
lintus
A linter whose rules are written in plain language.upstream description
-
citation-verifier
Check whether each cited paper supports the sentence citing it. Claude proves the quote, TypeSafe's Jev scores it, a human decides.upstream description
-
hermes-jev-plugin
TypeSafe Jev (System One) decision tools for Hermes Agent: jev_check / jev_route / jev_score / jev_evaluateupstream description
-
jev-security-scan
使用 TypeSafe Jev 审查 Skill 与 MCP 可疑行为 | Review Agent Skills and MCP code with Jev, static evidence, and explicit coverage gapsupstream description
-
jev-spec
⚡ Catch spec drift on every commit: check your code against your Markdown specs with TypeSafe AI's Jev model.upstream description
-
taste-lint
Catch AI slop before you ship.upstream description
-
jev-subtitle-translator
Translate SRT subtitles with structured LLM output and check every translation with Jev.upstream description
-
JevPR
PR Risk review, automated by Jevupstream description
-
jev-auto-router
Jev Auto Router (Jev Router): experimental per-call GPT model routing for Codex via TypeSafe Jev and a local Responses proxy, with independent task verification.earlier upstream description
-
jev-block-android-ad
JevNoiseGate filters unwanted notifications and SMS on Android. Rather than matching keywords, an LLM decides what's noise — and only what it explicitly flags is blocked. Verification codes are matched on-device and never uploaded; anything uncertain passes through.upstream description
-
jevgate
Code-review gate for CI and coding agents: asks TypeSafe Jev small typed questions about functions, files, tests and docs, and reports findings with locations and probabilitiesupstream description
-
pytest-jev
Semantic assertions for pytest: test what your LLM app's output means, judged by TypeSafe's Jev.
-
zod-jev
Zod validates the shape; Jev validates the meaning. Semantic checks on a schema — personal data present? price plausible? category matches? — go to Jev in one request and come back as probabilities that become Zod issues.
-
diffjury
DiffJury — TypeSafe Jev PR risk router + code review coachupstream description
-
jev-for-engineers
Eight minimal working examples of TypeSafe's Jev (a System One model) applied to mechanical and electrical engineering: CAD/CAE/CAM routing, FEM result triage, DFM screening, BOM alignment, hallucination-proof extraction. Zero dependencies.upstream description
-
jev-guard
Probability-scored guardrails for Claude Code: deny rule-breaking edits and unasked-for deploys, route your docs into each prompt, and check the final answer against the turn's own evidence.upstream description
-
jev-lm
A word-level language model whose output layer is Jev: n-gram drafter, Noul chunk verification, bits-per-token evalupstream description
-
n8n-nodes-jev-classification
n8n community node for Jev by TypeSafe AI: classify, score and check text with calibrated probabilities. Parallel requests and multi-item batching.upstream description
-
riff
A small, fast prose linter: ruff-style rule codes for writing, backed by TypeSafe's Jev modelupstream description
-
dinostomp
A verification layer for AI evaluations. Checks the instrument, not just the score: data, scorer, runs, numbers, claims, and itself.upstream description
-
hunch
Semantic code review with Jev, plain-English rules and Agent Skills.upstream description
-
jev-code-review-benchmark
Comparing Jev, Gemini Flash, and Claude Fable on Python code review rules: cost, speed, accuracy, and consistency. Includes results, charts, and reproducible experiments.upstream description
-
limpet
A Stop hook that stops your coding agent from stopping too early. Plain-language rules, judged by jev.upstream description
-
jev-align
Calibrated alignment verifier for LLM responses and agent plans — powered by Jevupstream description
-
jev-codes
Audit your git diff against YAML coding-standards packs using TypeSafe's Jev model, from a CLI or your AI agent's command/skill.upstream description
-
jev-oas-sentinel
Catch breaking API behavior hidden in OpenAPI prose with deterministic checks and TypeSafe JEV System One semantic review.upstream description
-
jevarena
Open-source BYOK arena for Jev and other AI judges. Find failures, compare quality, cost, and latency.upstream description
-
jevkit
Fast Rust CLI for TypeSafe Jev: typed decisions, offline linting before you payupstream description
-
jevtest
Semantic test matchers for Vitest and Jest, powered by TypeSafe's Jev model. Write expectations in plain English, get calibrated probabilities back.upstream description
-
semantic-assert
Testing lib for asserting the real requirement.upstream description
-
solvi
Python runtime with a System One API adapter: Jev, an LLM or a local checkpoint proposes typed answers; hard checks, rules and calibrated thresholds decide, abstain or escalate to a person, and every decision is a hash-chained replayable trace.
-
tenbin
MCP server and agent skill for the TypeSafe AI System One API (Jev): decompose a judgment into Choice / Score / Noul questions, lint them, measure on labelled data, and put calibrated thresholds in codeupstream description
-
Jev by Example
Ten runnable JavaScript agent decisions, one file each: reconciling a new memory against a stored one, gating whether an HTTP 200 really satisfied the task, retry vs. reconcile after an uncertain write, scoring context against a budget, checking a handoff for dropped prohibitions.
-
jev-behavior-study
Independent Jev 1.13.0 behavior study: report, controlled prompt experiments, raw results, and offline verification.upstream description
-
jev-decisions
Jev Decisions Plugin for Hermes (and other AI Agents): tool risk reviews, human approval recommendations, evidence checks, and a local decision journal.
-
jev-exploration
Jev (TypeSafe) exploratory thread: claim audit, live demos, and runnable codeupstream description
-
jev-preflight
A bounded Jev risk check for Claude Code: eight risk axes, one request, one optional reinspection.upstream description
-
jev-review-action
Configurable GitHub submission review and PR classification with TypeSafe Jev. No text-generation model.upstream description
-
jev-rust-review
Rust-aware code review for Claude Code and coding agents, powered by TypeSafe Jevupstream description
-
jevsume
ATS-friendly resume review powered by Jev (TypeSafe System One). The frontend extracts resume text the way a parser would, then a Cloudflare Worker runs typed JEV questions and composes a JevScore.upstream description
-
jod
Semantic schemas over TypeSafe's Jev — validate the state locally, then project typed answers.upstream description
-
open-jev-approvals
Binary approval gate for Codex and Claude Code — every intercepted tool call is reviewed by TypeSafe JEV and composed through a versioned local policy, with scoped authorization.upstream description
-
tripwire
Judge every LLM response before the user sees it. AI SDK middleware and OpenAI-compatible proxy.upstream description
-
typesafe-jev-calibrate-for-code-review
About calibrating Jev for code reviewsupstream description
-
clear-head
Claude Code Stop hook that checks an AI assistant's claims against what it actually read this session, using TypeSafe's Jev as the judgeupstream description
-
Footwork
A verified browser agent: a cheap Jev guard (evidence-checked completions, a destructive gate) in front of any LLM browser driver, with Jev taking the mechanical steps in dual mode. Built on browser-use; every number pre-registered and measured.upstream description
-
jackalope
A desktop workspace for coding agents, parallel Git worktrees, and code review.upstream description
-
jev-browser-pilot
A bounded decision layer for browser and desktop automation: a decision-only model picks one next step; the code owns perception, content, actuation and verification.upstream description
-
jev-labs
Never confidently wrong: a TLA+-verified consensus kernel around TypeSafe's Jev, run through 1,680 chaos-tested pharmacy decisions with zero wrong verdicts. Film, code, and every captured call.upstream description
-
jev-pr-judge
Typed verdicts on pull requests with TypeSafe System One (Jev): one parallel call, policy in code, usable as a GitHub Actionupstream description
-
jev-suite
Four decision-quality tools on Jev (TypeSafe System One): Jev answers structured questions, deterministic code keeps the final say.upstream description
-
jevibe-check
A live tone labeler for Bluesky posts and drafts, using TypeSafe's Jev API.upstream description
-
omp-typesafe
TypeSafe AI (Jev) adversarial reviewer and typesafe_ask tool for the omp coding agentupstream description
-
typesafe-ai-firewall
Shadow-mode validation harness for a pre-execution firewall on AI agent tool calls (TypeSafe/Jev). Real run, findings in report.md.upstream description
-
typesafe-migration-guard
Automated database migration safety reviewer powered by TypeSafe AI (Jev System One model)upstream description
-
dsh-jev-verify
Jev (TypeSafe System One) decision tools + live verification benchmark for DeepSeek Harness: jev_decision (choice/score/noul) and jev_verify, honest by design.earlier upstream description
-
human-compiler
A compiler for human language. Paste text, get diagnostics. Measured by TypeSafe Jev.upstream description
-
jev-bench
Does the cited source actually say it? A 42-claim benchmark: Jev (TypeSafe System One) against GPT-5.4, Claude Sonnet 5 and Gemini 3.1 Pro.upstream description
-
jev-debtgate
Jev-powered technical debt gate for coding agents and CI. Bring your own TypeSafe API key.upstream description
-
jev-resume-analyzer
CV diagnostics and job alignment with TypeSafe Jev, React and FastAPIupstream description
-
jev-review
Autonomous Jev pull-request review with typed decisions, calibrated approval gates, and trusted-owner escalationupstream description
-
jev-the-janitor
A janitor for markdown vaults powered by TypeSafe Jev: Jev votes on each note, your code files it, you review the low-confidence pile.
-
jevguard
Claude Code + Codex CLI plugin that verifies the agent follows project rules through a System One (Jev) modelupstream description
-
lossless-rewrite
AI text rewriting and document summarization with Jev checks for missing ideas and automatic repair. Local editor and CLI.earlier upstream description
-
plotveil
A quiet spoiler blocker for YouTube comments. One typed Jev (TypeSafe System One) Noul decision per comment; covered while checked, still covered if the check fails.upstream description
-
profanity-checker
Cloudflare Worker to check for profanity using TypeSafe Jevupstream description
-
stepwarden
Every tool call your agent makes, checked before it runs. A Claude Code plugin that uses TypeSafe AI's Jev to verify each pending tool call against the session plan, then allows it, asks you, or blocks it. Proof of conceptupstream description
-
system-one-playground
Readable scripting, semantic code checks, a Go System One client, and Studio.upstream description
-
typesafe-as-a-judge
Unofficial community MCP plugin for Codex and Claude Code using TypeSafe Jev for bounded routing, ranking, extraction, verification, and escalationupstream description
-
agent-gate-loop
Reusable GitHub Action: agent fix loop gated by checks, an AI reviewer, and TypeSafe Jevupstream description
-
agent-handoff-gate
An experimental protocol for evidence-aware agent handoffs, bounded worker continuation, and TypeSafe/Jev-assisted review, with reproducible evaluation.earlier upstream description
-
assay-001
ASSAY-001: independent, pre-registered verification of TypeSafe Jev's calibration and type-safety claims. Split verdict, published in full.upstream description
-
check-risk
A CLI and GitHub Action that assesses code-change risk using deterministic rules and TypeSafe Jev, recommending checks and reviewers before merge.upstream description
-
deepseek-harness-jev (luobosibing2)
DeepSeek Harness plugin with twelve opt-in features using Jev as the judgment model: skill selection and glob-result ranking (one `noul` per candidate), completion and goal checks, drift reminders, interjection routing, tool-output and test-log filtering, and approval of single workspace operations.written by a model
-
jev-agent-skill
Free typed judgments for AI agents: offload classify/screen/score/verify to Jev (TypeSafe System One) via OpenCode Zen. Claude Code / ZCode skill. 给AI代理省token的免费决策分流技能upstream description
-
jev-enterprise-decision-fabric
Architecture for running many semantic decisions through one validated path, with a labelled 111-case benchmark comparing TypeSafe Jev against a Claude baseline, and a dashboard for inspecting any single decision. Experimental, not production.upstream description
-
jev-gates
Six calibrated gates for Claude Code, judged by TypeSafe Jev: rules, scope, intent, done, claims, and commit honesty. Each one escalates, none ever approves.upstream description
-
jev-shadcn-lint-eval
A small second eval for shadcn-ui/lint that uses TypeSafe's Jev to judge the linter's own output.upstream description
-
jev-ui-test
UI test framework, a fork of jev-ultrafast: natural-language YAML cases run by pytest, Jev picks each step's action and target, and semantic assertions ask one `noul` per atomic claim about the page, with thresholds left in test code. The author reports a 458 ms median per action decision.written by a model
-
jev-verify
Check whether a published Jev output was actually produced by Jev. Implements the Yurin confidence identity; flags published fixtures that violate it.upstream description
-
JevTest
Bounded exploratory browser testing with Jev, deterministic assertions, and replayable evidence.upstream description
-
openclaw-typesafe-ai
Optional typed TypeSafe AI Jev decisions for OpenClaw, with SecretRef credentials and strict API validation.upstream description
-
pi-jev-code
Single-agent Pi coding coprocessor with Jev semantic gates, baseline-to-current diff review, and append-only observability telemetry.upstream description
-
typesafeai-review
Using Typesafe.AI to generate diff reviews.upstream description
-
Testing TypeSafe Jev, Mistral and Gemini for local event validation
The only three-way head-to-head found, with each model's prompt tuned separately and the scope limited to one task rather than a general ranking.
-
TypeSafe's Jev: Can decision models replace LLM judges?
Collects the third-party evaluations that exist so far and frames the question of where a decision model can stand in for an LLM judge.