Shadow Evaluation Guide

A structured, no-cost process for payer operations teams to establish a behavioral baseline for incoming AI voice calls — without changing anything in your call flow.

Shadow mode means observe-only. Your existing call handling is unchanged. You measure how today's incoming AI voice traffic behaves against the four NHID-Clinical v1.3 controls, then decide what to do with the findings.

What This Is

No cost. No vendor changes. No production risk.
You run it in shadow mode alongside your current operations.
Goal: produce a written behavioral baseline showing how AI voice agents currently disclose, escalate, and log calls to your administrative lines.
This is not a pilot program. There is no certification, no SLA, and no commitment beyond sharing anonymized findings with the community if you choose.

What You Need

Access to call logs or recordings for a sample of incoming administrative calls (eligibility, claim status, prior authorization).
About 30 minutes of staff time for orientation.
Willingness to share anonymized findings if you choose to.

The 90-Day Structure

Month 1 — Behavioral Baseline

Log incoming calls. Identify which callers are AI voice agents. Record disclosure timing, escalation behavior, and audit trail presence.

Month 2 — Gap Analysis

Evaluate identified AI calls against the four v1.3 controls (IDG-01, DBC-01, EIT-01, ATR-01). Quantify what passes, what fails, what is ambiguous.

Month 3 — Written Assessment

Compile findings into a short written assessment. Share the anonymized results with the community to help inform the next version of the proposal.

The Four Controls You Are Observing

Control	What it requires
IDG-01	Disclose AI identity at call start, before any data exchange.
DBC-01	No deceptive audio artifacts — no fake breathing, no human-name openers.
EIT-01	Immediate, clean escalation path to a human on request.
ATR-01	Basic audit log — enough to reconstruct when disclosure occurred.

How to Start

Email contact@nhid-clinical.org with the subject line "Shadow Evaluation". Include your role and the workflows you want to observe.

← Back to For Payers Evidence Pack (what you'd produce) →

Open for feedback

Questions? Reach out directly.

Whether you have a process question, a concern about the controls, or you want to share what you're seeing in your own call traffic — feedback from payer operations teams is what shapes this work.

For Payers →