Insurance › Exam-Ready Toolkit › Readiness check

Tracking the NAIC AI Risk Evaluation Supplement version 5.0 · exposed for comment, not adopted · Next: Working Group call, Nov 2, 2026NAIC Working Group ↗

Readiness check

Check your readiness

Five sections, 31 questions, about four minutes. Your answers stay in your browser.

Each section is one pillar of the methodology, weighted the way an exam weights them. You get a live score, the pillars you are weakest on, and the specific control that closes each gap. Orientation, not a certification. When you finish we log an anonymous result (your score, band, and per-pillar scores only, never your individual answers or anything that identifies you). Your email and summary are sent only if you opt in; opting in emails you your summary, adds you to our CRM as a lead, and subscribes you to occasional AI-governance updates you can leave at any time. We may publish aggregate, anonymized statistics from completions, never your answers or anything that identifies you. See our privacy policy.

0 of 31 answered

Section 1 of 5

1. Inventory and classification

Can you name every AI system you run, including the ones inside platforms you rent, say how risky each is, and scope it all to a materiality threshold you can defend? In v5.0 this is the gate: clearing Exhibit A cleanly is what keeps you out of Exhibits C and D.

25% of the score · Exhibit A; Exhibit C ref 1 to 10

1.1You keep a written inventory of every AI system in production, including vendor-embedded models and features inside platforms you rent.

Stand up an ai-system.yaml register in the repo, one entry per system, and add a CI check that fails on any model call with no entry. Fix: Control 1

YesPartlyNo

1.2Each system is tagged to an Exhibit A operational area (underwriting, claims, fraud, and the rest).

Add an operational_area field to every register entry, using the Supplement's thirteen areas. Fix: Control 1

YesPartlyNo

1.3Each system carries an autonomy level: support, augment, or automate.

Record autonomy per system in the register, using the Supplement's three definitions. Fix: Control 1

YesPartlyNo

1.4Each system carries a risk classification with a written rationale.

Add risk_class and a one-line risk_rationale to every entry; your definition of high risk is itself evidence. Fix: Control 1

YesPartlyNo

1.5You can produce version, implementation date, and a count of systems implemented in the past 12 months on demand.

Record version and implemented date per entry so the register answers the past-12-months count as a query. Fix: Control 1

YesPartlyNo

1.6You have a written materiality threshold for which AI systems are in scope, with a documented rationale for why it is the right one.

Write a one-page materiality memo: the threshold, its basis, why it is appropriate, the owner, and a review date. v5.0 lets the company set and defend its own. Exhibit A; checklist 3o

YesPartlyNo

1.7Each model is tagged by type (GLM, generative or agentic, other) and flagged for whether it has Direct Consumer Impact and whether it has Material Financial Impact.

Add model_type, direct_consumer_impact, and material_financial_impact fields to each model entry, matching Exhibit A Part One columns. Fix: Control 1

YesPartlyNo

Section 2 of 5

2. Governance and accountability

Is there a written, owned AIS Program that someone actually reviews, and can every policy answer be pointed to a document and page, the way v5.0's checklist now asks?

20% of the score · Exhibit B Q1, Q5; checklist 1 to 3

2.1A written AIS Program exists, dated, with a review cadence.

Adopt a dated AIS Program with an explicit review cadence; scale the ceremony to your size, keep the substance. checklist 1

YesPartlyNo

2.2A named role owns the AIS Program.

Name the owning role, not a person, and record it on each register entry's owner field. Fix: Control 1

YesPartlyNo

2.3The program addresses checklist items 3a through 3p, at least by reference.

Map the program to each checklist item 3a to 3p, including the new 3o (materiality threshold) and 3p (vendor-developed model oversight); a reference is enough, silence is not. checklist 3a

YesPartlyNo

2.4AI risk is a step in your software delivery process, not a separate document.

Fold AI risk into the release checklist and SDLC so delivery itself produces exam evidence. Fix: Control 8

YesPartlyNo

2.5The program has been reviewed at least once since adoption, and the review is documented.

Run and minute one review of the program; an unreviewed policy PDF scores nothing. checklist 1

YesPartlyNo

2.6Every policy answer can be cited to a document name and page number.

Keep an evidence index: for each checklist item, the document name, the page, and the date last reviewed. v5.0's checklist asks for the document and page on every answer. checklist 3

YesPartlyNo

Section 3 of 5

3. Validation and observability

The heaviest pillar. Are your models tested before launch and watched in production, with a last-test date you can produce on demand, and can you explain what a model did?

25% of the score · Exhibit B Q3; Exhibit C ref 11, 12

3.1Pre-deployment validation is documented per model, with the reference data source stated.

Commit a dated validation report per model from a CI evaluation suite, stating internal, external, or both. Fix: Control 6

YesPartlyNo

3.2Each high-risk model is monitored in production with drift detection and a threshold that triggers review.

Build production metrics from the decision log and set a drift threshold that opens a review ticket. Fix: Control 6

YesPartlyNo

3.3Every automate-level decision is logged with inputs, output, model version, and timestamp, so any single outcome can be reconstructed.

Write an append-only decision record on the same code path that returns the decision. Fix: Control 4

YesPartlyNo

3.4Vendor-supplied models have a validation procedure that does not rely on the vendor's attestation alone.

Run your own evaluation suite against the vendor model; treat attestation as an input, not a substitute. Fix: Control 6

YesPartlyNo

3.5You can produce a last-test date for every high-risk model without asking an engineer to go look.

Have the scheduled validation job write last_test into the register so the field is always true. Fix: Control 6

YesPartlyNo

3.6For any model that recommends or makes decisions, you can explain how it reached an output, in terms a reviewer and a regulator can follow.

Record per model what drives its outputs (features, rules, or prompt and retrieval) and keep a plain-language explanation. v5.0 narrative 3d asks for explainability and transparency. Fix: Control 6

YesPartlyNo

Section 4 of 5

4. Data provenance

Can you trace every data element feeding a model to its source and vendor, justify any sensitive inputs, and say which models rely on which data, the way Exhibit D now asks?

20% of the score · Exhibit D; checklist 3d

4.1Every data element feeding a model is mapped to an Exhibit D category.

Tag each column to an Exhibit D category in the schema documentation. Fix: Control 7

YesPartlyNo

4.2Source is recorded per data element: internal, or third-party with the vendor named.

Record source per element; vendor name is a required Exhibit D field. Fix: Control 7

YesPartlyNo

4.3Training and test data lineage is documented for internally developed models.

Document the lineage of training and test data for the models you build. Fix: Control 7

YesPartlyNo

4.4Sensitive categories are absent from model inputs, or their use has a written justification and a proxy-discrimination check.

Enumerate sensitive features; remove them, or add a written justification and a proxy check in the validation suite. Fix: Control 7

YesPartlyNo

4.5The data map is versioned and updated when a model or a vendor changes.

Version the data element map and update it on any model or vendor change. Fix: Control 7

YesPartlyNo

4.6Each data set is mapped to the AI systems and models that rely on it, and a data dictionary exists for each high-risk model.

Add a reliant-systems field to the data map and keep a per-model data dictionary; v5.0 Exhibit D adds the reliance header and a possible data-dictionary request. Fix: Control 7

YesPartlyNo

Section 5 of 5

5. Consumer outcome controls

When AI touches an adverse decision, is there a human path that does real work, a complaint trail, and the disclosures your states require?

10% of the score · checklist 3a, 3c, 3m, 3n; Exhibit C ref 14, 15

5.1A human review path exists for adverse consumer decisions, with written triggers.

Route adverse decisions (declination, non-renewal, adverse rating, claim denial) to human review with written triggers. Fix: Control 5

YesPartlyNo

5.2Complaints are tagged to the AI system involved, and the tag is queryable.

Write the decision_id onto the complaint record so complaints are queryable by system. Fix: Control 4

YesPartlyNo

5.3Consumer-facing disclosure of AI use exists where the state requires it.

Check the state tracker for disclosure duties and add disclosures where required. checklist 3n

YesPartlyNo

5.4Each consumer-impacting model has a documented compliance review against unfair trade practice and claims settlement rules.

Add a compliance and bias section to each model's validation report. Fix: Control 6

YesPartlyNo

5.5A regulatory-action log exists per model, even if empty.

Add a regulatory-action field to the register; empty is a valid and expected value. Fix: Control 1

YesPartlyNo

5.6For adverse decisions, there is a record of what the human reviewer actually checks, not only that a human signed off.

Capture the reviewer's checklist and findings per adverse decision; v5.0 narrative Q2b asks whether the human in the loop does real work, not whether one exists. Fix: Control 5

YesPartlyNo

·/100

Your readiness

Answer the sections to see your score.

Clear all 31 answers?

By pillar

Inventory and classificationScore · · Weight 25%→

Governance and accountabilityScore · · Weight 20%→

Validation and observabilityScore · · Weight 25%→

Data provenanceScore · · Weight 20%→

Consumer outcome controlsScore · · Weight 10%→

By exam funnel stage

Answer the sections to see how far down the funnel you would survive. In v5.0 every company is asked whether it uses AI, then Exhibit A, then often B, and only a risk-selected set of models reaches C and D.

Exhibit AIs AI used, and how muchScore ·

Exhibit BThe AIS ProgramScore ·

Exhibit CHigh-risk modelsScore ·

Exhibit DModel dataScore ·

What to fix

Book a 30-day readiness review Read the full methodology

Keep your results

A short, tailored memo from your answers: where you stand, the two or three gaps to close first, and what "exam-ready" looks like for your book. Built from your answers only, orientation not legal advice.

Email me this summary and occasional Nolte updates on AI governance in insurance. Unsubscribe any time. How we use this.

Self-assessment and orientation only. Nothing here is legal advice, a certification, or a compliance determination. Nolte is not affiliated with or endorsed by the NAIC. Exhibit references are paraphrased from Supplement version 5.0; the wording that counts is the NAIC's. Corrections: j@nolte.io.