Score

stage

We don't currently track AI performance by patient demographics

We're aware of the need but only review fairness metrics on an ad hoc basis

We conduct regular audits using established fairness metrics (e.g., statistical parity, equalized odds) across key patient demographics

We run real-time fairness monitoring with dynamic adjustment across all patient demographics

No — we don't have a formal fairness threshold for deployment

We discuss fairness during development, but there's no documented go/no-go standard

Yes — we have a formal, documented standard for high-risk models, signed off by data science and clinical leadership

Yes — our standard is publicly communicated, externally validated, and reviewed with community stakeholders

GRC tool (e.g., OneTrust, Archer, MetricStream)

Model documentation repository (e.g., Model Cards/validation templates)

Vendor documentation / procurement standards

We have no live monitoring — we likely wouldn't detect it until a complaint or audit

We might catch it during a scheduled review, but there's no automated detection

Automated dashboards track fairness metrics and flag issues, but response requires manual intervention

Real-time bias detection is integrated into our pipeline with pre-approved escalation and rollback protocols

Azure ML monitoring / Responsible AI tools

Tableau / Power BI / Looker dashboard (BI only)

Vendor portal (third-party AI vendor monitoring)

PagerDuty / Opsgenie (alerting/escalation)

Email / Microsoft Teams / Slack escalation