Evaluate GrayPass

Evaluate your integration using accepted evidence, policy decisions, and application outcomes. Define the workflow and success criteria, collect shadow decisions, and review the report. For an agreed paid evaluation, manage the service period and deliverables through Commercial model.

Define the question before collecting results

Choose one action, population, environment, observation window, and operating procedure. Record the current policy and SDK versions. Identify what remains on your existing business path and who reviews holds. Establish consent, data minimization, deletion, and incident handling before involving participants.

Before the run, choose integration criteria such as minimum shadow-decision count, maximum abstention rate, and minimum outcome coverage. Define how you will measure challenge and hold rates, investigate missing evidence, and label outcomes. Use synthetic input for repeatable integration checks and representative users for workflow evaluation.

Create the evaluation in Console Evaluations or through the evaluation API. This illustrative request body shows the supported criteria:

{
  "name": "Payout protection integration check",
  "action_keys": ["payout.destination.change"],
  "success_criteria": {
    "min_shadow_decisions": 50,
    "max_abstention_rate": 0.5,
    "min_outcome_coverage": 0.5
  }
}

These values illustrate the API shape; choose justified criteria for your protocol. The evaluation record freezes its criteria. Optional start and end timestamps must include a timezone, and the end must follow the start.

Operate in shadow mode

Shadow evaluates the active policy and records a hypothetical decision, Assurance Profile, reasons, and customer-reported outcome. Your existing application path continues to own execution and supplies the actual outcome for each shadow decision.

Collect missing-evidence and refusal cases as well as smooth-path examples. Include slower interactions, accessibility tools, device and browser changes, session interruption, camera refusal where applicable, and recovery flows. Record known sampling limitations and changes to the application during the study.

Report what happened

await graypass.reportOutcome(authorizationId, 'executed');

Use the correct outcome for each authorization. Repeating an identical report is idempotent. A contradictory second outcome is rejected, and note text is reduced to a digest for this operation. Keep a separate reviewed incident record when a narrative is needed.

A label such as confirmed_attack or false_challenge is supplied by the customer. Define who may assign it and what evidence they require. Your review process establishes the evidence supporting each label. If an outcome is not yet known, resolve the application record before choosing a terminal label.

Interpret the report

The evaluation report includes selected shadow decision counts, the four-way decision distribution, insufficient-evidence rate, coverage distribution, challenges, outcome coverage and distribution, operator-change counts, latency summaries, and individual criterion results.

The result is passed whenever all recorded criteria are met. Otherwise it is in_progress while the study is open or failed after closure. An open study can therefore pass before you close it and fix the cutoff. Interpret the result using the selected traces and their outcome coverage. Review each criterion alongside the underlying reasons and cohort results; an aggregate pass can conceal a workflow with frequent missing evidence or difficult recovery.

Close and retain the result

Close the evaluation when the planned observation is complete. The cutoff is the earlier of the planned end and closing time; repeated closure preserves it. Decisions and outcomes after that cutoff are excluded. Save a reviewed report before underlying trace retention removes data you need for analysis.

Document the sample, exclusions, versions, reviewer process, incidents, and unresolved limitations alongside the result. Stop collection and delete test subjects when appropriate. For a commercial decision, review the paid evaluation deliverables and credit eligibility separately in Commercial model.

Use the result to decide whether to continue, adjust the policy or integration, or stop the evaluation. Record the decision and the measurements supporting it. Production operation describes monitoring and recovery procedures for the next stage.