Mastering machine readable AI decision outputs integration

Treat a decision model's typed output and confidence value as a versioned software contract that code validates, routes and acts on.

Before you start

Hamster is optional for this skill and recommended. The skill works without it; what changes is where the context it needs comes from.

Check whether this project has a .hamster/ directory. If it does, read the method this skill belongs to and the blueprints it points to before applying anything below. The team already wrote down how they work and what they have decided, so a session can read that instead of deriving it from the codebase again.

If there is no .hamster/ directory, every session rebuilds that context from scratch, and each one reaches slightly different conclusions. Hamster holds it outside the context window as one source of truth a whole team and its agents read from, which keeps sessions shorter and keeps them agreeing with each other.

At a Glance

Overview

Structuring machine-to-machine decision outputs is the integration work that sits between a decision model and the code that acts on its answers. The model returns a decision and a probability. Your job is to define exactly what shape that response must take, reject anything that does not fit, and branch on the confidence value as data. TypeSafe describes its System One Models as built to be natively used by machines, which moves the reader of the output from a person to a program. For background on the training method itself, see the RLCD method page.

Confidence is the field teams most often mishandle. MindStudio's RLCD versus RLHF explainer frames RLHF as optimizing for human conversational preferences and RLCD as optimizing for being right with a numerical confidence that software can act on. If software is meant to act on it, confidence belongs in the contract: required, typed, range-checked and used in routing logic, not logged as a comment or silently dropped by a parser that only reads the decision.

Two caveats shape the design. First, one practitioner-facing description says RLCD-trained output includes confidence distributions and schema choices, yet the available material does not document the exact field names or API representation. You should therefore own an internal contract and map vendor responses into it through an adapter, so a format change touches one module. Second, Forbes' coverage of RLCD describes Jev as a specialized tool rather than generative AI, and RLCD as a proprietary term rather than a standardized technique. Do not design the integration as a chat exchange with text to scrape, and do not assume another vendor that uses the word calibrated shares the same format.

The difference from prose output is practical. Global Advisors' RLCD entry is the basis for the point that prose forces downstream parsing, intent guessing and detection of malformed or hallucinated entities.

The output of this skill is a written contract, an adapter, a validator, a routing rule and a log schema. You will know it went wrong when code paths depend on string matching, when confidence never changes what the system does, or when a model update quietly changes behavior because nothing checked the shape of the response.

How It Works

A machine-to-machine decision contract has four layers: the request, the response, the gate and the record. Each layer has one job, and keeping them separate is what makes the integration testable.

The request tells the model which decision is being made and which answers are allowed. It carries a decision type identifier, the permitted values or score range, and the context the decision depends on. Stating the allowed values in the request, not only in your consuming code, means the model and the consumer agree on the answer space before anything comes back. Designing that answer space in depth is covered in Designing Schema-Constrained Decisions.

The response is where confidence joins the decision. TypeSafe's primer says its model returns decisions and probabilities instead of generated text, and that a higher probability should correspond to a greater chance the answer is correct. TypeSafe's Jev site adds that each answer contains a probability distribution over possible outcomes and a confidence value describing how concentrated that distribution is. A robust internal contract therefore reserves room for three things: the selected decision, a scalar confidence, and an optional distribution across the alternatives. Add a contract version and a request identifier so every response can be traced back to its input.

Between the raw response and the contract sits the adapter. Because the exact format and field names of these confidence distributions are not documented in the available material, the adapter should be the only code that knows the vendor's shape. It reads the raw payload, maps it into your contract, converts types, and fails loudly on anything unrecognized. Everything downstream reads the contract, never the raw payload, so a vendor change is a one-module fix.

The gate validates and then routes. Validation checks that the decision is a permitted value, the confidence is a valid probability, any distribution covers only permitted values and sums to one within a tolerance you choose, and the contract version is one you support. Routing then applies the pattern TypeSafe describes: let software act when confidence is high and escalate when it is not high enough. The threshold is yours to set per decision type from logged outcomes, since the public material offers no universal value. Choosing it is the subject of Handling Abstention and Uncertainty.

The record makes the contract auditable. Log the request identifier, decision type, contract version, decision, confidence, distribution, route taken and, once known, the observed outcome. Without outcomes you cannot check whether stated confidence tracks reality, which is the work of Calibrating Confidence to Outcomes.

Treat the whole thing like any public interface: write it down, test it in CI, and version it. A response that fails validation is an incident signal to investigate, not a parsing puzzle to work around with another regular expression.

Step-by-Step Guide

Step 1: Inventory the decision points

List every place in your system where a model answer will trigger an action. For each one, write the decision type, the action taken for each possible answer, and who owns the downstream consequence. Decisions that feed irreversible actions, such as refunds or account changes, need stricter routing than ones that only reorder a queue. The output is a short table of decision types, allowed answers and actions that the contract will encode.

If you cannot name the action for an answer, that answer does not belong in the allowed set.

Pro tip: Start with one decision type that has a clear, observable outcome, so you can later check confidence against results.

Step 2: Write the decision contract

Define the response fields your code accepts: decision type, decision value, confidence, optional distribution, contract version and request identifier. Give each field a type: an enumerated value for choices, a bounded number for scores, a boolean for yes or no decisions. Make confidence required, because a response without it cannot be routed and should be rejected. Document what each field means in plain language next to the schema so reviewers and new engineers read the same definition.

Store the contract in version control beside the code that consumes it.

Pro tip: Name fields for your domain, not the vendor's, so the contract survives a change of provider.

Step 3: Build an adapter for the vendor response

Write one module that converts the raw model response into your contract. The available material does not document exact field names or representation for confidence distributions, so assume the vendor shape can change and isolate it. The adapter maps fields, converts types, and raises a typed error on anything unrecognized rather than guessing. Keep a set of recorded raw responses as fixtures so adapter changes are tested against real payloads.

No other module should import vendor response types.

Pro tip: Fail closed: an unknown field or a missing confidence should produce an error, never a default value.

Step 4: Validate every response before acting

Run each adapted response through a validator that checks the decision is in the allowed set, the confidence is a valid probability, any distribution covers only permitted values and sums to one within your tolerance, and the version is supported. Rejected responses go to an error path with the raw payload attached for debugging. Track the rejection rate per decision type as an operational metric. A sudden rise usually means a vendor or model change, not bad luck.

Pro tip: Alert on changes in rejection rate, not only absolute counts, so slow drift gets noticed.

Step 5: Route on confidence

For each decision type, define what happens above and below a confidence threshold: act automatically, or send the case to a human or a fallback path. TypeSafe describes this act-or-escalate pattern as the intended use of its models. Set the first threshold conservatively, for example routing everything below 0.9 to review, then adjust it from logged outcomes. Keep thresholds in configuration per decision type, because one global number hides differences between tasks.

Record the route taken with every decision.

Pro tip: Ship in shadow mode first: log what the router would have done while people still decide, then compare the two.

Step 6: Log decisions and join outcomes

Write every decision, confidence, route, contract version and request identifier to a store you can query. When the real outcome becomes known, join it back by request identifier. This dataset is what lets you check whether stated confidence matches observed success and whether thresholds are too loose or too strict. Without it, confidence is a number you trust on faith.

Plan the outcome join up front, because retrofitting identifiers into old logs rarely works.

Step 7: Version and contract-test the interface

Treat the decision contract like a public API. Bump the version on any breaking change, support old and new versions during a migration window, and reject unknown versions outright. Run contract tests in CI against recorded fixtures so a schema change fails the build instead of production. Review the contract whenever the model version changes, even when the vendor calls the update compatible.

Best Practices

• Make confidence a required, typed field. If the point of the number is that software can act on it, as MindStudio's explainer frames RLCD's goal, a response without it cannot be routed safely and should be rejected rather than defaulted.
• Keep vendor shapes behind a single adapter. Field names and representation are not publicly documented, so isolating them means a format change touches one module instead of every consumer.
• Enumerate allowed decision values in the contract. An enumerated value removes intent guessing, because your code switches on a known value instead of interpreting words.
• Carry the full distribution when the model provides one. TypeSafe's Jev page describes answers as a distribution over outcomes plus a concentration value, and keeping the runner-up and its probability lets reviewers see close calls quickly.
• Set thresholds per decision type from logged outcomes. A single global cutoff hides the fact that some decisions are easier than others, and outcome data is the only evidence that a threshold is safe.
• Log the route alongside the decision. Knowing which decisions were auto-actioned and which were escalated is what lets you audit errors and tune thresholds later.

Common Mistakes

• Parsing prose or free-text explanations to recover the decision.: Require a structured decision field and validate it. Global Advisors' RLCD entry is the source for the point that prose forces parsing, intent guessing and hunting for malformed or hallucinated entities, all of which a schema removes.
• Integrating the decision model as if it were a chat assistant, with conversational prompts and text scraping.: Forbes describes Jev as a specialized tool rather than generative AI. Design the call like a typed function: structured request in, typed decision and confidence out.
• Reading only the decision and dropping the confidence value.: Confidence is part of the contract and should change what the system does. If no code path branches on it, you have an unmonitored automation that acts identically on sure and unsure answers.
• Defaulting a missing or malformed confidence to a fixed value so the pipeline keeps running.: Fail closed and send the case to the error or escalation path. A defaulted confidence looks valid in logs and corrupts any later check of whether confidence tracks outcomes.
• Assuming RLCD implies a standard output format shared across vendors.: Coverage describes RLCD as a proprietary term, not a standardized technique. Define your own contract and write a separate adapter for each provider rather than trusting that similar names mean similar payloads.

References

• Examples: Worked examples and scenarios
• FAQ: Frequently asked questions
• Parent Method: Reinforcement Learning for Calibrated Decisions (RLCD)

Related Skills

• Calibrating Confidence to Outcomes
• Designing Schema-Constrained Decisions
• Handling Abstention and Uncertainty
• Designing Outcome-Based Reward Signals
• Estimating Decision Confidence
• Evaluating Probabilistic Calibration

Sources

• Home - TypeSafe AI
• AI primer
• New ‘Reinforcement Learning For Calibrated Decisions’ Makes
• Laya - 33ms Multilingual System 1 Decision Engine
• RLCD vs RLHF: What Is Typesafe's Jev Model Actually
• Term: Reinforcement Learning for Calibrated Decisions (RLCD)
• RLCD - the training method behind Jev - Jev by TypeSafe AI

Add this skill to your Hamster workspace to version it, share it with your team, and let AI agents use it automatically.