Nolte AI Readiness Methodology

Status note: Version 5.0 of the Supplement was exposed for comment on Aug 31, 2026, superseding draft 4.0, and it restructures the exhibits (see the changelog for the 4.0 to 5.0 changes). This guide is anchored to 5.0: the exhibit references and field numbers below reflect the 5.0 structure, including the renamed AIS Program (Exhibit B), the split of AI Systems from AI Models in Exhibit A, and Exhibit C’s 15 renumbered fields. The readiness methodology has been rebuilt against 5.0: 31 criteria across the five pillars, each tagged to an exam-funnel stage (A through D), with inventory weighted to 25% and consumer controls to 10% because Exhibit A now gates the exam. Version 5.0 is exposed, not adopted. The comment window closed at the end of September 2026, with version 7.0 as the adoption target at the Fall National Meeting in November 2026. When the guide moves, the changelog records what changed.

What this is

A readiness rubric for anyone building or operating AI in insurance: MGAs and program administrators running on a carrier’s paper, vendors selling into them, and the carriers who will be asked to describe both.

The NAIC Supplement is addressed to insurers. It is not addressed to you. That is the point. Exhibit B, question 4 asks the carrier to describe its oversight of AI systems used by professional service providers, and names MGAs explicitly. When a state examiner sends that question to your fronting carrier, the carrier answers it with whatever evidence you can produce. If you can’t produce it, the carrier will produce a version for you, on its terms, or it will stop asking and start restricting.

This rubric turns the Supplement’s four exhibits into five things any operator or vendor can be scored on. It is a self-assessment instrument. Nolte does not publish scores for named companies.

Who this matters for, and why

The thread through all five: the examiner asks the carrier, the carrier asks everyone below it, and evidence that does not exist cannot flow.

What this is not

• Not a certification and not a compliance determination. The NAIC has said the Supplement itself is not one either. Nolte is not affiliated with or endorsed by the NAIC.
• Not a legal opinion. Bulletin adoption and AI-specific rules vary by state. Check the state tracker and your own counsel.
• Not a NIST or ISO crosswalk. Those exist elsewhere and they are useful. This rubric starts from the exam question, not the framework.

The five pillars

Weights reflect what an examiner reaches for first (Exhibit A), what a carrier will demand of the operators and vendors on its book (Exhibit B Q3, Q4), and where the detailed evidence burden lands (Exhibits C and D).

Each criterion scores 0, 25, 50, 75, or 100 on a full review. Pillar score is the mean of its criteria. Composite is the weighted sum. The interactive readiness check on this site uses a faster three-point version of the same rubric (Yes 100, Partly 50, No 0).

Pillar 1: Inventory and classification (25%)

Exhibit A asks, per operational area, for the number of AI Systems currently in use, the number implemented in the past X months (the regulator sets X), and then AI Models broken out by type (GLM, generative/agentic, other) under two impact headings: Direct Consumer Impact and Material Financial Impact, plus the use case for each. Part Two then requests a Model Inventory with seven fields per model (name, how it is used, the broader use case, program areas, inherent risk level, consumer impact, financial impact). Exhibit C asks, per high-risk model, for name and version (C1), use cases and purpose (C2), model type (C3), implementation date (C4), development and vendor name (C5), whether it was purchased or can be modified (C6), risk classification (C7), risks (C8) and limitations (C9), and autonomy level (C10: automate, augment, support).

An operator that cannot answer Exhibit A in a day cannot answer anything downstream.

Criteria

1. 1.1 A written inventory exists of every AI system in production, including vendor-embedded models and features inside platforms you rent.
2. 1.2 Each entry is tagged to an Exhibit A operational area. Version 5.0 lists thirteen: marketing, premium quotes and discounts, underwriting and eligibility, ratemaking and rate classification, claims and adjudication, customer service, utilization management, fraud and waste and abuse, investment and capital management, reserves and valuations, catastrophe triage, reinsurance, and other insurance practices. (Version 5.0 dropped the legal/compliance and producer services rows.)
3. 1.3 Each entry carries an autonomy level using the Supplement's definitions: support, augment, automate.
4. 1.4 Each entry carries a risk classification with a written rationale. The Supplement leaves "high risk" to the company to define, so the definition itself is evidence.
5. 1.5 Implementation date and version are recorded, and the inventory can produce "implemented in the past 12 months" on demand.
6. 1.6 A written materiality threshold defines which AI systems are in scope, with a documented rationale for why it is appropriate. Version 5.0 lets the company set and defend its own threshold (checklist 3o), or the regulator specifies one.
7. 1.7 Each model is tagged by type (GLM, generative/agentic, other) and flagged for Direct Consumer Impact and Material Financial Impact, matching Exhibit A Part One's columns.

Common gap: the rented platform. If your PAS or rating vendor ships a model inside a feature, it is in your inventory whether or not you built it. Vendor name is a required field in Exhibit C and Exhibit D.

Pillar 2: Governance and accountability (20%)

Exhibit B, the AIS Program (renamed in version 5.0 from “Governance Framework”), narrative version, asks who maintains the program, how it reaches the board, how it is integrated and remediated, how effectiveness is assessed, and how autonomy, reversibility, and reporting impact are assessed. The checklist version asks whether a written AIS Program exists, when it was adopted, how often it is reviewed, and whether the board or management were involved. Every checklist answer now cites a document name and page number.

For a founder-led operator, “board reporting” is usually the carrier relationship plus your own leadership. Scale the ceremony, keep the substance.

Criteria

1. 2.1 A written AIS Program exists, dated, with a review cadence.
2. 2.2 A named role owns it.
3. 2.3 The program addresses the checklist items 3a through 3p at least by reference: inherent risks including unfair trade practices, legal compliance, adverse consumer outcomes, data privacy, suitability, ERM, ORSA, SDLC integration, financial reporting impact, training, risk quantification, vendor procurement standards, complaint tracking, consumer disclosure, and the two version 5.0 additions, the materiality threshold used to assess AI risk (3o) and oversight of vendor-developed models (3p).
4. 2.4 AI risk is integrated into your SDLC (checklist 3h). If you build, this is where your delivery process becomes exam evidence.
5. 2.5 The program has been reviewed at least once since adoption and the review is documented.
6. 2.6 Every policy answer can be cited to a document name and page number. Version 5.0's checklist asks for the document and page on every response, so an evidence index (item, document, page, last reviewed) is the artifact.

Common gap: a policy PDF written for the carrier’s onboarding questionnaire that nobody has reopened.

Pillar 3: Validation and observability (25%)

The heaviest pillar because it is the heaviest question. Exhibit B Q3 asks for validation and testing procedures on internally developed systems, on vendor-supplied systems, the frequency, scope, and methodology of verification, and new Q3d, how the company ensures explainability and transparency into models that recommend or make decisions. Exhibit C 11 asks how model outputs are tested for drift, accuracy, unfair trade practices, unfair discrimination, and performance degradation, how the model was validated before deployment, and how it is monitored on an ongoing basis. Exhibit C 12 asks for the last test date.

“Ongoing basis” is the operative phrase. A validation report from launch is a snapshot. An examiner asking for the last test date wants a system that produces one.

Criteria

1. 3.1 Pre-deployment validation is documented per model, with the reference data source stated (internal, external, or both, per the Supplement's "Validation Method" definition).
2. 3.2 Production monitoring exists per high-risk model: at minimum accuracy or outcome-rate tracking, drift detection, and a defined threshold that triggers review.
3. 3.3 Every decision produced by an automate-level system is logged with inputs, output, model version, and timestamp, so any single outcome can be reconstructed.
4. 3.4 Vendor-supplied models have a written validation procedure that does not depend solely on the vendor's own attestation.
5. 3.5 A "last date of model testing" can be produced for every high-risk model without asking an engineer to go look.
6. 3.6 For any model that recommends or makes decisions, you can explain how it reached an output in terms a reviewer and a regulator can follow (narrative Q3d, explainability and transparency).

Common gap: telemetry exists in the application but was never framed as governance evidence. This is usually a reporting problem, not a build problem.

Pillar 4: Data provenance (20%)

Exhibit D lists 25 data element types, from aerial imagery through telematics to non-traditional data, and asks for each: how the AI model uses the data, whether it is internally sourced, and if third-party, the source and vendor name. Version 5.0 adds a header field naming the AI Systems or AI Models that rely on the data, and the regulator may request a data dictionary depending on the responses. Checklist 3d covers data privacy and protection.

Criteria

1. 4.1 Every data element feeding an AI model is mapped to an Exhibit D category.
2. 4.2 Source is recorded per element: internal, or third-party with vendor named.
3. 4.3 Training and test data lineage is documented for internally developed models.
4. 4.4 Sensitive categories (age, gender, ethnicity, medical, criminal, income, geo-demographics) are either absent, or their use has a written justification and a proxy-discrimination check.
5. 4.5 The data map is versioned and updated when a model or a vendor changes.
6. 4.6 Each data set is mapped to the AI systems and models that rely on it (Exhibit D's version 5.0 header field), and a data dictionary exists for each high-risk model.

Common gap: geocoding and geo-demographics. Almost every rating engine uses them. Almost no one has written down that they do.

Pillar 5: Consumer outcome controls (10%)

Checklist 3a, 3c, 3m, and 3n: inherent risk of unfair trade practices, evaluation of adverse consumer outcomes, complaint identification and tracking, and consumer disclosure. Exhibit C 14 asks how each model is reviewed for compliance with state and federal law, including unfair trade practices and unfair claims settlement. Exhibit C 15 asks whether any action has been taken against the company for use of the model.

Criteria

1. 5.1 For each consumer-impacting model, a human review path exists for adverse decisions (declination, non-renewal, adverse rating, claim denial), and its trigger conditions are written.
2. 5.2 Complaints are tagged to the AI system involved, if any, and the tag is queryable.
3. 5.3 Consumer-facing disclosure of AI use exists where the state requires it, and the state tracker is the source of truth for where.
4. 5.4 Each model has a documented compliance review against applicable unfair trade practices and claims settlement rules.
5. 5.5 A regulatory-action log exists per model, even if empty.
6. 5.6 For adverse decisions, there is a record of what the human reviewer actually checks, not only that a human signed off (narrative Q2b: whether the human in the loop does real work).

Scoring bands

Readiness check

The front door is the interactive readiness check: one question per criterion across the five pillars, scored on a three-point fast-track (Yes, Partly, No), with a live score and the control that closes each gap. It is the rubric above made answerable in a few minutes, not a separate scheme.

For a thirty-second gut check before you open it, ask:

1. Can you list every AI system in production, including ones inside platforms you rent, today?
2. Does each one have a written risk classification and a reason for it?
3. Does a named person own your AI program, and has it been reviewed since it was written?
4. Is AI risk a step in your software delivery process, not a separate document?
5. For your highest-risk model, can you state the last date it was tested?
6. Is that model monitored in production for drift and outcome rates, with a threshold that triggers review?
7. Can you reconstruct any single automated decision from logs: inputs, output, model version, time?
8. Is every data element feeding your models mapped to a source, with vendors named?
9. Is there a human review path for adverse consumer decisions, with written triggers?
10. If your fronting carrier forwarded Exhibit B question 4 to you tomorrow, could you answer it in a week?

If more than a couple are “no”, the 30-day readiness review is built for exactly this.

Versioning

• v0.1 (2026-08-24): initial draft against Supplement draft 4.0.
• v0.2 (2026-09-01): distinguished the full-review scoring scale from the interactive check’s three-point fast-track, and pointed the readiness-check section at the interactive tool. Kaio Pedreira review.
• v0.3 (2026-10-12): reconciled to Supplement version 5.0. Updated the exhibit references and field numbers (Exhibit B renamed the AIS Program; Exhibit A split AI Systems from AI Models by type and dropped two program-area rows; Exhibit C renumbered to 15 fields; Exhibit D added a reliant-systems header field), applied the Exhibit C crosswalk to every field reference, and refreshed the checklist item range to 3a through 3p. Pillar and criteria structure unchanged, pending the later readiness rebuild.
• v0.4 (2026-10-13): rebuilt the readiness criteria against version 5.0. Added six criteria (a materiality threshold and model-type and impact flags in inventory, page-addressable evidence in governance, explainability in validation, model-to-data reliance in data provenance, and reviewer-checks evidence in consumer controls) for 31 total, tagged every criterion to an exam-funnel stage (A through D), and reweighted inventory to 25% and consumer controls to 10% because Exhibit A now gates the exam.
• Next revision: on movement toward version 7.0 (the Fall National Meeting adoption target, November 2026).