A decision model does not write. You send it a situation and a set of typed questions, and it answers each one with a probability. We rebuilt a planner on one over a week, and every quality gain came from the shape of the question rather than from the model.
What does a decision model return instead of text?
TypeSafe's System One takes a state (the situation, in plain words) and a bag of named questions. Three types cover most decisions:
- noul asks how likely a statement about the state is to hold.
- choice returns one option from a set, with a probability on every option and a margin between the top two.
- score returns a level as a distribution across the levels, not as a single number.
Each option is described by criteria: one sentence saying what it is for. Questions in a request are evaluated together, so the slots, the density and the widths of a screen arrive in one round trip.
Billing is the other half of the mechanism. Input is priced per billion tokens and output is free, so our planner sends about 5,900 tokens of criteria and pays 0.0244¢ in 1.45 seconds. Routing the same decision through a general-purpose model cost 0.0419–0.0766¢ and took around 4 seconds.
Why does rewording the question change the answer that much?
Because the model answers the question you asked, with no charitable reading of what you meant. Five rewrites, measured against the same model, the same options and the same screens:
| The decision | How we asked it first | Top | Asked again as | Top now |
|---|---|---|---|---|
| Does a status badge belong here | does this screen need one | 48% | would this screen be improved by one | 80% |
| What is the main body | is it not a table, a chart or numbers | 29–67% | what is it | 81–87% |
| What opens the screen | 33 options from one family | 42% | 11 options matched to the role | 97% |
| Which chart fits | which chart | 31% | how many measures, and what the x axis is | 95–100% |
| What the reader can do here | one slot for every action | 61% | split into commit, scope and field | 56–100% |
Two patterns run through all five. A gate phrased as an exclusion makes the model reason about the exclusions instead of the subject. And a question whose answer is not in the state stays ambiguous however it is worded: the right chart is a function of the data, so "which chart" sat at 31% until we asked what the data was.
The last row is the honest one. Splitting the action question three ways took two of the three to 100% and 76%, and left the third at 56%. Which input collects a value depends on the value, and a screen description does not carry it.
The control is the part worth keeping. Slate size did nothing: 3 options against 43, and confidence held between 0.96 and 1.00. Narrowing the menu is not the lever. Naming the job is.
Check one routing question
Take the weakest gate in your own pipeline. Ask two things: is it phrased as an exclusion, and is its answer present in what you sent? Rewrite it once and record both numbers.
Where does it break?
It writes nothing. Every string on our screens is authored up front or filled by a separate language-model call, so this is the wrong tool whenever the copy is the product.
A near tie is real ambiguity, not a question worth asking twice. We built the follow-up call that re-asks a close pair on a narrowed slate, then measured it: the answer flipped in two cases of three and never beat plain renormalisation.
The third limit cost us two hours. Our screens drifted between runs, 9 of 20 held identical, and the plan was to add hysteresis to the model's thresholds.
A calibrated "I don't know" is information, and it is information about the question. When a steady model looks unsteady, suspect the arithmetic downstream of it.
We write our own agent-facing material the same way: the tokens, rules and workflows a coding agent reads are there so its decisions are answerable rather than guessed.