Confidence Estimation for AI Decisions, Step by Step

Attach a probability to every structured decision and check it against real outcomes, so software knows when to act and when to escalate.

Before you start

Hamster is optional for this skill and recommended. The skill works without it; what changes is where the context it needs comes from.

Check whether this project has a .hamster/ directory. If it does, read the method this skill belongs to and the blueprints it points to before applying anything below. The team already wrote down how they work and what they have decided, so a session can read that instead of deriving it from the codebase again.

If there is no .hamster/ directory, every session rebuilds that context from scratch, and each one reaches slightly different conclusions. Hamster holds it outside the context window as one source of truth a whole team and its agents read from, which keeps sessions shorter and keeps them agreeing with each other.

At a Glance

Overview

A structured decision on its own tells downstream software what the model chose, not how much to rely on it. Confidence estimation for AI decisions attaches a probability to each decision so code can accept a result when confidence is high and escalate it when it is not, which is the operational use TypeSafe describes for its decision models. This page covers how to produce that number and how to tell whether it means anything. For the definition and background of the training method behind it, see Reinforcement Learning for Calibrated Decisions (RLCD).

The target property is calibration. A glossary entry on RLCD describes a reward tied to whether a model's stated probability matches how often its answer turns out to be correct, so decisions labeled 0.8 should be right about 80% (source) of the time across many comparable cases. That is a claim about frequencies you can check, not a feeling of certainty.

The obvious shortcut is to ask a model to rate its own confidence. TypeSafe's launch post says models prompted for confidence estimates are often overconfident and inconsistent, and that is its stated reason for training confidence against observed outcomes instead of trusting a self-reported number. Whatever the source of your probabilities, the lesson carries over: a number is a hypothesis until outcomes confirm it.

Two limits shape how you use the result. First, calibration is a property of a population. As Anthony Maio points out, it describes a population of predictions and does not tell you whether one particular answer is right. Second, vendor claims are not a substitute for your own check. Turing Post's explainer lists RLCD's technical details as unpublished, so you cannot inspect how a given model's confidence was produced and must verify it on your own tasks.

In practice the skill has three parts: framing the decision so a probability has a clear meaning, attaching that probability to a logged record, and checking it against outcomes by decision type before any automation depends on it. Deeper evaluation with reliability plots and expected calibration error lives in Evaluating Probabilistic Calibration, and choosing act-or-escalate cutoffs lives in Handling Abstention and Uncertainty.

How It Works

Confidence estimation takes four inputs, all implied by the outcome-based framing in the RLCD glossary: the structured decision task, the set of candidate outcomes or actions, the decision the model selected, and an outcome signal that later shows whether that decision was correct. Drop any one and the probability loses its meaning. Without a fixed candidate set, a probability has no reference class. Without an outcome signal, nothing can ever confirm or refute the number.

The output is a structured decision paired with a probability that software can act on, such as accepting a result or routing it to review. Public explanations of TypeSafe's Jev, including Turing Post's, describe this pairing: a judgment returned together with probabilities rather than prose.

Where the probability comes from matters, and there are broadly three sources. A model trained against outcomes, which is what RLCD claims to do, is rewarded when its stated probability matches observed correctness. Raw output scores, such as a distribution over the candidates, can be read directly but may be miscalibrated. A prompted self-report, where you ask the model to write down a confidence, is the weakest, because TypeSafe reports such estimates are often overconfident and inconsistent. The public material gives no canonical algorithm for turning model internals into a confidence score and no universal threshold for acting; a guide to RLCD notes that TypeSafe has published the name and the goal rather than the full method. Treat every source as unverified until checked.

Checking follows a simple loop described in that practitioner guide: log each decision with its predicted probability, record the true outcome when it arrives, group decisions into probability bins, and compare each bin's mean predicted probability with its observed accuracy. The same guide recommends doing this separately for each question or decision type, because calibration can vary by task. A model can be well calibrated on routing tickets and badly overconfident on flagging fraud, and a pooled number hides that.

Two interpretive rules keep you honest. A score with many decimal places is not more trustworthy than a rounded one; precision is formatting, and only comparison with later outcomes gives a score meaning. And no single high-confidence decision is guaranteed correct. Calibration, as Maio stresses, speaks to aggregate frequencies, so even a well-calibrated high score will be wrong on some share of cases. Design downstream actions so those misses are survivable.

The practical result is an empirical table per decision type: stated confidence levels on one side, observed success rates on the other. That table, not the model's raw number, is what an act-or-escalate policy should rest on.

Step-by-Step Guide

Step 1: Define the decision and its candidate outcomes

Write down the question the model answers and the complete set of answers it may return, such as a fixed list of categories or a yes/no. The probability is only interpretable relative to this set: it is the chance the chosen option is the correct one among these candidates. Decide now what counts as correct and how you will learn it, because an outcome you cannot observe makes the confidence uncheckable. If two reviewers would disagree about the right answer for a typical case, tighten the definition before collecting any numbers.

Pro tip: Include an explicit unknown or none-of-these option when real cases can fall outside the list, otherwise the model is forced to spread probability over wrong answers.

Step 2: Choose and document the confidence source

Decide whether the probability comes from a model trained against outcomes, from raw output scores over the candidates, or from a prompted self-report. Record which source you use in every decision record, since they behave differently and you will want to compare them. Prefer trained or score-based probabilities over self-reports, which TypeSafe's launch post says are often overconfident and inconsistent. Whichever you pick, treat it as a hypothesis to test rather than a finished measurement.

Pro tip: If self-reports are all you have today, log them anyway; they become the baseline a better source has to beat.

Step 3: Attach the probability to a logged decision record

Store each decision as a record with a stable ID, the decision type, the input or a reference to it, the selected answer, the probability, the model version and a timestamp. Keep the full distribution over candidates when you have it, not just the top score, because it shows how close the runner-up was. Write the record at decision time, before anyone knows the outcome, so hindsight cannot leak in. Without this record, every later calibration check is impossible.

Pro tip: Version the record schema and the model from day one; when either changes, you need to split the data at that boundary.

Step 4: Record the ground-truth outcome

Join each decision to its true outcome when it becomes known, such as an agent's final category, a customer's reply or an audit result. Store the outcome source and the delay, since some outcomes arrive in minutes and others in weeks. Mark decisions whose outcome never arrives instead of dropping them silently, because missing outcomes are rarely random. A practitioner guide recommends recording outcomes before judging calibration, since a confidence value cannot be validated from the model output alone.

Pro tip: Flag outcomes the decision itself influenced, such as a ticket resolved only because it was routed, and label them with care.

Step 5: Compare stated confidence with observed accuracy per decision type

For each decision type separately, group records into probability ranges and compare the average stated probability in each range with the fraction that were correct. If decisions in a high range are right far less often than their stated probability, the model is overconfident there; if they are right more often, it is underconfident. Make sure each range holds enough records to mean something, for example a few hundred, before drawing conclusions. The detailed mechanics of bins, reliability plots and expected calibration error are covered in Evaluating Probabilistic Calibration.

Pro tip: Check the high-confidence ranges first; that is where automation will act alone, so miscalibration there costs the most.

Step 6: Route decisions on verified confidence

Only after the comparison, map stated probabilities to observed success rates and use those observed rates to decide what software may do alone. Accept automatically where the observed rate meets your risk tolerance and escalate below it, choosing the cutoff per decision type because the same score can mean different things across tasks. Keep a sample of accepted decisions flowing to review so outcomes keep accumulating above the cutoff. Revisit the mapping whenever the model, prompt or input mix changes.

Best Practices

• Frame every probability against a fixed, written candidate set. A confidence value only means something when you know which alternatives it was weighed against and what counts as correct.
• Log the decision and probability at decision time and join the outcome later. Writing both before the outcome is known prevents hindsight from quietly editing the record and keeps the comparison honest.
• Measure calibration separately for each decision type. A practitioner guide to RLCD notes calibration can vary by task, so a single pooled figure can hide a badly overconfident category behind a well-behaved one.
• Keep the full distribution over candidates, not only the winning score. The gap between first and second choice is often the clearest early sign that a decision should go to review.
• Treat vendor-supplied confidence as unverified on your data. With RLCD's technical details unpublished, the only evidence that a score is calibrated for your task is your own outcome log.
• Design downstream actions so high-confidence misses are survivable. Calibration describes a population of predictions, so some confident decisions will be wrong, and reversible actions or spot checks limit the damage.

Common Mistakes

• Asking the model to state its own confidence and using that number directly.: TypeSafe says prompted confidence estimates are often overconfident and inconsistent. Use trained or score-based probabilities where possible, and check any source against logged outcomes before relying on it.
• Reading a precise-looking score as a meaningful one.: Extra decimal places are formatting, not evidence. The RLCD practitioner guide ties meaning to comparing scores with later outcomes across many decisions, so withhold trust until that comparison exists.
• Validating whether decisions were right while ignoring the probabilities attached to them.: The point of an outcome-based approach, as the RLCD glossary describes it, is whether stated probability matches observed correctness. Track accuracy within confidence ranges, not only overall accuracy.
• Pooling unlike tasks into one calibration check.: Split the log by decision type and compare each separately, because calibration can differ across question types. A pooled result can look fine while one task is badly off.
• Treating a single high-confidence decision as guaranteed correct.: Calibration is a statement about aggregate frequencies and, as Maio notes, does not tell you whether one particular answer is right. Plan review sampling and reversibility for confident misses.

References

• Examples: Worked examples and scenarios
• FAQ: Frequently asked questions
• Parent Method: Reinforcement Learning for Calibrated Decisions (RLCD)

Related Skills

• Calibrating Confidence to Outcomes
• Structuring Machine-to-Machine Decision Outputs
• Designing Schema-Constrained Decisions
• Handling Abstention and Uncertainty
• Designing Outcome-Based Reward Signals
• Evaluating Probabilistic Calibration

Sources

• Introducing System One Models & Jev - TypeSafe AI Blog
• Home - TypeSafe AI
• RLCD (Reinforcement Learning for Calibrated Decisions)
• RLCD explained: Reinforcement Learning for Calibrated Decisions
• What Is Jev AI? Inside TypeSafe's RLCD Model - Turing Post
• Jev: The Language Model That Won't Talk - Anthony Maio

Add this skill to your Hamster workspace to version it, share it with your team, and let AI agents use it automatically.