Reinforcement Learning for Calibrated Decisions (RLCD)

Created by Diogo Almeida and TypeSafe AI - https://typesafe.ai/

Overview

Reinforcement Learning for Calibrated Decisions (RLCD) is a post-training method in which the reward is tied to whether a model's stated probability matches how often its answer turns out to be correct, rather than to human preference ratings or automatic verification, as the Sanity glossary entry on RLCD summarizes it. TypeSafe AI's own machine learning primer states the goal plainly: the model does not generate text, it returns decisions and probabilities, and a higher probability should correspond to a greater chance that the answer is correct. The launch material frames the target as "calibrated decisions: answers with epistemically honest probabilities on System One tasks," according to the RLCD explainer guide. Put simply, when the model says 0.8, the thing should happen about 80% of the time, as one practitioner write-up puts it.

TypeSafe AI introduced RLCD alongside Jev, which its launch post, Introducing System One Models & Jev describes as the company's first System One Model, released after two years in stealth. Sources disagree on the date: the TypeSafe launch post is dated September 21, 2026, while RuntimeWire covered Jev's early access in mid-September and a LinkedIn explainer says the company came out of stealth on September 15, 2026 (source). That explainer credits founder Diogo Almeida with co-inventing RLHF and InstructGPT, and TechCrunch reports that Jev is trained exclusively on synthetic data. RLCD is one of three named parts of the stack, alongside a new model architecture and a parallel sampler, none of which is explained further, as Digital Applied notes.

flowchart TD
  A[New model architecture] --> D[Jev System One Model]
  B[Parallel sampler] --> D
  C[RLCD training] --> D
  D --> E[Typed decision]
  D --> F[Calibrated probability]
  E --> G[Software acts or escalates]
  F --> G

TypeSafe positions RLCD as an alternative to RLHF. Where RLHF trains a model to produce answers human raters approve of, RLCD trains a model to produce a decision paired with a confidence value, according to MindStudio's RLCD vs RLHF comparison. The launch post also contrasts it with reinforcement learning from verifiable rewards (RLVR), a distinction Pear Pages records, and DataCamp notes that leading chat models are trained with RLHF and verifiable rewards instead.

The output is not a paragraph. Coverage describes structured decisions such as a choice, a score or a yes/no probability (ExplainX profile). The Jev RLCD page says each answer carries a probability distribution over possible outcomes plus a confidence value describing how concentrated that distribution is, and one practitioner-facing description calls these confidence distributions and schema choices. Spreading uncertainty across alternatives lets downstream code see that a runner-up option was close, instead of receiving one unqualified answer. TypeSafe's stated intent is that software acts when confidence is high and escalates when it is not (TypeSafe home page). The motivation, per the launch post, is that models prompted to give confidence estimates are often overconfident and inconsistent.

Searches for RLCD also surface an unrelated method. A paper by Yang and colleagues in 2023 (source) introduced Reinforcement Learning from Contrastive Distillation, which generates preference pairs from contrastive prompts to simulate RLHF data without human labelers (Ginger Labs). It shares the acronym and nothing else, so its results say nothing about TypeSafe's method.

RLCD is a proprietary term, not a standardized technique, as Forbes coverage points out, and Forbes Japan makes the same point. The evidence base is thin: there is no paper, no reliability curve, no expected-calibration-error number and no ablation separating RLCD from the architecture, per Agentpedia's claim-versus-evidence guide. Critics add that calibration describes a population of predictions and does not tell you whether one particular answer is right (Anthony Maio), and a Japanese deep-research note warns about benchmarks that treat an average of generative-AI outputs as the correct answer. Until TypeSafe publishes results, treat RLCD as a stated objective with useful vocabulary and verify calibration on your own logged outcomes. Teams running that verification in Hamster can keep prediction logs, review decisions and open questions in the same shared workspace as the rest of the project.

Core Principles

Calibration is the objective, not eloquence

The Jev RLCD page states the training objective as calibration rather than fluent writing, so that when Jev says 80% it should be right about 80% (source) of the time. That changes what a good output looks like: a modest probability that is right about as often as it claims beats a confident one that overstates its hit rate. The TypeSafe primer makes the same promise in ordinal form, that higher probability should mean a greater chance of being correct. Treat the probability as a product feature you test, not decoration on the answer.

Calibration lives in the aggregate

TypeSafe's documentation narrows its own claim, saying calibration is measured across groups of predictions, as Agentpedia quotes it. Anthony Maio stresses that this describes a population and does not tell you whether one particular answer is right. A single high-confidence decision can still be wrong while the model remains perfectly calibrated. Design review and rollback paths on the assumption that some confident answers will fail.

Accuracy and calibration are different tests

A model that reports 85% confidence should be correct approximately 85% (source) of the time across many similar cases, and being right more often overall is not enough if confidence does not track outcomes, per Blockchain Council. The common mistake is to see high overall accuracy and assume the probabilities are trustworthy. They can be systematically inflated in one band and deflated in another while the headline accuracy looks fine. You only find this by comparing stated confidence with observed accuracy inside confidence groups.

Decisions are typed and bounded

RLCD-style models return decisions and probabilities instead of generated text, according to the TypeSafe primer. Coverage lists the shapes as a choice, a score or a yes/no probability (ExplainX). A bounded answer space is what makes calibration measurable, because every output maps to an outcome you can later mark right or wrong. Open-ended prose has no such mapping, which is why the method starts from a schema.

Confidence is part of the software contract

MindStudio frames the contrast with RLHF as optimizing for being right with a numerical confidence that software can act on, rather than for human conversational preference. That means the probability field belongs in the interface specification alongside the decision itself. TypeSafe describes the intended use as acting when confidence is high and escalating when it is not (TypeSafe). If your integration discards the probability, you have thrown away the part RLCD exists to produce.

Claims stay claims until evidence arrives

RLCD is a vendor-coined, proprietary term according to Forbes, and no reliability curve, calibration error figure or ablation has been published (Agentpedia). Do not borrow credibility from the unrelated 2023 contrastive distillation paper that shares the acronym (Ginger Labs). The practical stance is to adopt the objective and the measurement habits, and to hold any specific model to your own logged outcomes.

Steps

1. Define the decision and its answer space Write down exactly what the model must decide and the full set of permitted outcomes, such as a fixed list of queues or a yes/no flag. Specify field types so every output is machine-checkable and maps to an outcome you can later score. Avoid free-text answers, which force downstream code to parse and guess intent. The detailed practice is covered in Designing Schema-Constrained Decisions.

   You know this went wrong when reviewers argue about whether an output counts as correct.

2. Attach a probability to every decision Require the model to return a probability or distribution alongside each decision, not a verbal hedge. TypeSafe's launch post argues that simply prompting a model for a confidence number tends to produce overconfident, inconsistent values, which is why RLCD trains confidence against outcomes. Record the full distribution across alternatives when available, since a close runner-up is useful signal. See Estimating Decision Confidence for how the number gets produced and interpreted.

3. Log predictions against outcomes Store each decision, its stated probability and, once known, the true outcome, as the RLCD explainer guide recommends. Without the outcome column the probability cannot be validated from model output alone. Keep an identifier and task type on every row so you can slice later. The logging workflow is detailed in Calibrating Confidence to Outcomes.

   Tip: decide how long you wait for an outcome before labeling, and apply that window consistently.

4. Measure calibration by task type Group logged predictions into probability bins, compare each bin's mean stated confidence with its observed accuracy, and summarize the gap. The RLCD explainer guide advises measuring each question type separately because calibration varies by task. Pooling unlike tasks can hide a badly miscalibrated category behind a well-behaved one. Evaluating Probabilistic Calibration covers bins, reliability plots and expected calibration error.

   Tip: check that each bin holds enough decisions before trusting its accuracy figure.

5. Set the act-or-escalate policy Choose a confidence threshold per task above which software acts and below which it escalates, for example auto-applying above 0.9 and sending the rest to review. No source specifies a universal threshold, so derive yours from the measured accuracy in each band and the cost of an error. Revisit thresholds when calibration drifts or the input mix changes. Handling Abstention and Uncertainty walks through the routing design.

   You know the policy is wrong when the escalation queue is either empty or overwhelmed.

6. Train or tune against outcomes If you build your own calibrated model, reward the match between stated probability and realized outcome rather than rater preference. TypeSafe has not disclosed its reward function (Anthony Maio), but the open OpenJev reimplementation combines cross-entropy with a Brier score term (OpenJev repository). Treat such reimplementations as one reasonable design, not TypeSafe's recipe. Designing Outcome-Based Reward Signals covers scoring rules and post-hoc adjustments.

   Re-run the calibration measurement after every training change.

When to Use

• An automated pipeline must act on a label without a human reading it, such as routing or approving items, because a calibrated probability tells the code when the label is safe to apply.
• You are designing an act-or-escalate workflow and need a principled way to decide which decisions go to human review rather than a gut-feel cutoff.
• You are evaluating a vendor's claim that its model is calibrated and need a checklist of what evidence to request and what to measure yourself.
• Your current system relies on a chat model's self-reported confidence and you suspect those numbers are overconfident or inconsistent across similar inputs.
• Ground-truth outcomes arrive after the decision, such as chargebacks, resolved tickets or confirmed labels, so you can compare stated probabilities with what actually happened.

When Not to Use

• The task is open-ended writing, summarizing or conversation, because RLCD targets typed decisions and Jev is described as a specialized tool rather than a generative assistant.
• No outcome signal ever arrives to mark a decision right or wrong, because calibration cannot be checked without ground truth.
• Each decision is a rare one-off with no comparable population, because calibration is a property of many similar predictions and says little about a single case.
• You need a reproducible, peer-reviewed training recipe today, because TypeSafe has not published the RLCD algorithm, reward function or training details.