How we know it is working, and what happens when it is not.
This category answers the question a technical buyer asks second, after “what can it do?”: how do you know the system is working correctly enough to trust it with anything that matters?
A green test suite proves what the tests cover.
Passing tests demonstrate that the code did what it was written to do. They say nothing about whether the source was the right source, whether the business assumption behind a calculation holds, or whether an economic model is coherent. Reliability here means both halves: the software works, and the answer is defensible.
So the work splits. Deterministic validation checks the things that have a correct answer. Evaluation measures the interpretive steps against known-good cases. Verification and review put a second, deliberately different path against the first. And detection, escalation, logging and rollback assume that something will eventually be wrong anyway, and decide how quickly it is noticed and how cleanly it is undone.
Four questions with different answers.
- Is it right? Deterministic validation for exact answers; automated evaluations for the judgement-shaped work that has no single correct output.
- Is it still right? Regression testing, so a change that quietly degrades an existing behaviour is caught rather than discovered by a customer.
- Would a second opinion agree? Cross-model verification, independent model review and adversarial review — three deliberately different postures, not one idea repeated.
- What happens when it is wrong? Confidence thresholds, failure detection, escalation logic, audit logging and rollback.
Evaluations and deterministic validation measure the primary path. The three review capabilities test it from outside: cross-model verification is the mechanism — running a second path and comparing results; independent model review is the architecture that makes that path as uncorrelated as it can be made — heterogeneous reviewers, deliberately diverse model families, evidence and prompts, so the two paths tend to fail on different things; the independence is partial, not statistical; and adversarial review changes the objective altogether, optimizing the second path for finding defects instead of producing an answer. A strong adversarial review is allowed to return nothing found.
Confidence thresholds and escalation logic turn all of that into behaviour at runtime, and audit logging and rollback make the result recoverable. Where a human sits in that chain is decided in Agents & Automation.
Eleven capability pages sit in this category. Availability in the platform is not a claim that every one is used on every engagement.