Guides

Evaluate in shadow mode

Shadow mode evaluates the real immutable policy and stores the hypothetical decision, Assurance Profile, reason codes, and customer-reported outcome. It does not issue a proof and does not tell the customer application to execute.

Pre-register criteria

{
  "name": "Payout protection integration check",
  "action_keys": ["payout.destination.change"],
  "success_criteria": {
    "min_shadow_decisions": 50,
    "max_abstention_rate": 0.5,
    "min_outcome_coverage": 0.5
  }
}

The API validates these fields and freezes them with the evaluation record. The report computes each check as pass or fail. An open evaluation is in_progress; a closed evaluation that misses a criterion is failed.

Report one outcome

await graypass.reportOutcome(decision.id, 'executed');

An authorization accepts one immutable outcome. Repeating the same report is idempotent. A contradictory second report is rejected. Notes are reduced to a digest before storage.

In Console, open Activity, choose a decision, and record what your application actually did. Filters help find holds and decisions awaiting an outcome. Return to Evaluations to see the report update from those traces.

Customer outcomes are labels supplied by the customer. They are not automatically verified ground truth and cannot turn an experimental detector into a production accuracy claim.

Report contents

GET /api/v1/evaluations/{evaluation_id}/report

The report includes:

• shadow decision count and four-way distribution;
• insufficient-evidence reason rate;
• coverage distribution;
• challenge count;
• customer outcome coverage and distribution;
• experimental operator-change indicator count;
• latency summaries for the selected traces;
• each pre-registered criterion result;
• overall in_progress, passed, or failed state.

These are integration and decision-trace statistics. They are not false-accept or false-challenge rates, and they do not establish real-human accuracy.

Close the observation window

Close the evaluation after reviewing its decisions and outcomes. Closing uses the earlier of the planned end and the current time; repeating the request preserves that cutoff. Decisions and outcomes recorded after the cutoff are excluded. Download the report for your records: the environment's retention policy still applies to its underlying traces.

Optional start and end timestamps must include a timezone, and the end must be later than the start.

Enforcement boundary

Passing an evaluation does not unlock live enforcement. The server requires an applicable, approved real-human validation and calibration release before any live action can enter enforce mode, and no such release exists yet.

Sandbox enforcement remains available so a developer can test challenge, proof, verification, one-time consumption, application execution, outcome, and deletion without representing the result as production-ready.

Generated from the API contract in contracts/ and the source in docs/. The build fails if this page disagrees with the running API.