Same model, five ways of asking. What a System One critic can and can't do in an agent harness.

Accuracy tells you if a decider is right. Brier and ECE tell you whether its confidence means anything — the number a cascade actually routes on. Jev returns a probability natively; the LLM tiers verbalize one, which is the thing TypeSafe says not to trust. Latency and cost are measured per call from the vendor's own usage report.

Walk the tiers in order; accept the first answer at least τ away from a coin flip; the last tier always accepts. Slide τ and the whole frontier recomputes in the browser from the published per-item scores. If the cheap tier is calibrated, the curve hugs frontier accuracy at a fraction of the cost. If it isn't, you can see exactly where it lies.

Several test-split tasks have dozens to hundreds of independent attempts from the same cheap coding model. Every selector draws from the same candidate pool with the same seeded draws (rows scored by all four tiers), picks its argmax, and we count whether the pick passed — against a random pick and an oracle (any passer in the draw). This is Pillbox's dispatch --critic-policy select, simulated on runs already paid for.

The baseline asked one holistic question over a flat diff — a System Two question ("simulate the tests") put to a System One model. The ladder changes one thing per rung: structure the state; add the reconstructed final files; add the tests the agent could see; then replace the holistic question with atomic ones and fit the weights on the task-disjoint dev split. The last rung gets cute: a cheap open model (Qwen3-30B) reads the task and writes the checks itself, Jev answers them, and the loop is closed three ways. Nothing here touches the test split until scoring.

"Will this pass?" asked after the run is cheap trivia — the verifier is seconds away. The decision worth 150 ms is the one made while the agent is still running. Across the test runs replayed here, — of the agents' wall-clock went to runs that ended up failing (54% across all 1,554 runs), and the first edit lands about halfway through a run. So: cut every run at every edit, score the prefix with the composite atoms, fit a combiner on dev runs only, and sweep a kill threshold on test runs. The output is a trade: agent-minutes not spent on losers vs winners wrongly killed. Result: killing on the first edit is no better than a coin flip (winners and losers killed at the same rate). From the second edit on there is a real but modest signal — at a conservative threshold, about 8% of failing-run time saved for 4.6% of winners killed, roughly two losers per winner — and only about 41% of runs live long enough to get a second edit. Mid-run prefixes are where the composite discriminates best in this whole study (AUC 0.79), which points at where a critic belongs: not at the door, in the middle.

Pillbox's Ghost loop forks k workers, grades each with a verifier, and pulls the winner. The verifier is the reward — exact, exit-derived, unforgeable — and also the slow last step. --critic typesafe puts a predicted probability in front of it, and because the verifier still runs on every graded worker, every dispatch becomes a labeled (p, passed) pair: the harness measures its own critic for free.

Every row is a real Pillbox session: an opencode worker's final edits on an aider-polyglot / sensitivity / toolz task, and the rubric verifier's verdict from the session's own scored event. Paths and injected memory preambles stripped; grader output dropped (it leaks the label). Split is by task, so no decider can pad its score on near-duplicate attempts.