Underwriting support teams face a narrow line: a conversation that helps a policyholder understand their coverage is a service interaction, but the same conversation can slide into advice or solicitation if an agent recommends a policy change or implies a coverage outcome. Quality assurance for these conversations has to catch that line being crossed, at scale, with a reasoning trail regulators and compliance teams can actually audit. RevelirQA, the AI AutoQA platform built by Revelir AI, performs quality assurance to the standard of a human quality analyst on every conversation, calibrated per customer until its evaluations exceed 95% accuracy as validated by that customer's own human QA analysts reviewing a sample of its scores. This human-level judgment is what differentiates effective auto QA from coverage alone. Most AutoQA tools already claim to evaluate 100% of conversations, so coverage is table stakes; what matters is whether the AI's reasoning on a nuanced advice-versus-service call matches what your compliance team would conclude.

TL;DR

• RevelirQA performs quality assurance to the standard of a human quality analyst, calibrated per customer to over 95% accuracy as validated by that customer's own human QA analysts reviewing a sample of its evaluations, which is the differentiator in auto QA scoring.
• The legal boundary is administrative/clerical activity versus sale, solicitation, or negotiation of insurance, as defined by the NAIC Producer Licensing Model Act and state insurance department rules.
• Manual QA sampling reviews only 1-5% of tickets, leaving most advice-adjacent language in support chat and email unchecked until it becomes a complaint or an exam finding.
• A usable QA scorecard for this problem needs binary, auditable criteria (did the agent recommend, quote, or imply a coverage decision) not vague "tone" or "helpfulness" scores.
• Regulators have shown they will enforce this boundary with real money. The New York State Department of Financial Services fined an AIG subsidiary $12 million in 2021 for unlicensed insurance business.

About the Author: This article is written by the team at Revelir AI, whose AutoQA platform is used in production by enterprise clients including Xendit and Tiket.com to score thousands of support conversations a week against each company's own policies, including SOP boundaries that carry compliance risk if crossed.

What Counts as "Sales-Adjacent" Support in Insurance Underwriting?

Sales-adjacent support is any conversation where a policyholder or prospect asks a question that touches coverage, pricing, or policy terms, but where the agent's job is to inform rather than to sell or advise. Examples include a policyholder asking why a premium changed at renewal, a prospect asking whether a quote covers a specific risk, or a claimant asking what their deductible means for a pending claim. None of these require a producer license to answer factually. All of them can tip into licensed activity the moment an agent says something like "you should upgrade to the higher tier" or "I'd recommend dropping that rider." Digital-first carriers run a large share of this volume through chat and email rather than phone, which changes how QA has to work: a call can be coached live, but a chat transcript either gets reviewed after the fact or it doesn't get reviewed at all [verequest.com].

Where Exactly Is the Legal Line Between Service and Advice?

The line is defined by the NAIC Producer Licensing Model Act and the state insurance department rules that implement it. These regulations distinguish administrative or clerical acts, which any support agent can perform without a license, from the sale, solicitation, or negotiation of insurance, which legally requires a producer license. Administrative acts include things like confirming a policy is active, explaining a billing cycle, processing an address change, or reading back coverage terms that are already in writing. Solicitation or negotiation includes recommending a specific product, comparing policy options in a way meant to influence a purchase decision, or describing terms not yet in force in a way designed to close a sale. The NAIC State Licensing Handbook is the published reference most carriers' compliance teams use to classify specific phrases and scenarios, and it includes worked examples precisely because the boundary is not always obvious from a single sentence taken out of context.

That's the mechanism worth sitting with: the violation usually isn't a single bad phrase, it's a phrase plus the implied authority behind it. "Your policy includes a $500 deductible" is administrative. "You'd be better off bumping that deductible down" is solicitation, even though both sentences are short, factual-sounding, and delivered in the same tone. A QA process that only flags obviously pushy sales language will miss the second example every time, because it reads as helpful rather than aggressive.

Why Does Manual QA Sampling Struggle With This Specific Risk?

Manual QA sampling reviews 1-5% of tickets, which means a carrier's QA team is choosing, by definition, to leave 95-99% of conversations unchecked. That sampling gap is tolerable for catching broad service quality trends, but it's a poor fit for a compliance risk that can appear in a single sentence, in a single ticket, from an agent who otherwise performs well. A reviewer sampling tickets at random has a low statistical chance of pulling the exact ticket where an agent, trying to be helpful, crossed into advice. Digital channels make this worse, not better, because chat and email volume is typically higher than call volume and growing faster, while QA staffing for those channels often lags behind phone QA [verequest.com]. The result is a widening gap between where the compliance risk actually lives and where human reviewers are looking.

How Should a QA Scorecard Be Built to Catch This Without Over-Flagging?

A QA scorecard built for this problem needs criteria specific enough that two reviewers, or an AI and a reviewer, reach the same conclusion on the same ticket. Vague criteria like "stayed professional" or "was helpful" don't do that; they're subjective enough that inconsistent scoring is the expected outcome, not the exception. What works better is a short set of binary or scored criteria tied to observable agent behavior:

Scorecard Criterion	What It Checks	Scoring Type
Recommendation language	Did the agent suggest, imply, or compare specific coverage options rather than stating facts already on file	Binary (yes/no)
Quote or pricing discussion	Did the agent quote a new premium, rate, or term not already confirmed in the policy record	Binary (yes/no)
Disclaimer and handoff	Did the agent redirect advice-seeking questions to a licensed producer where required	Binary (yes/no)
Factual accuracy	Were stated policy facts (deductible, limit, effective date) correct against the policy record	Scored (1-5)
Escalation trigger missed	Did the conversation contain language that should have triggered a licensed-agent handoff but didn't	Binary (yes/no)

Each row needs a short written definition and two or three real examples of what passes and what fails, the same way the NAIC handbook uses worked examples rather than abstract language. That's what lets a scorecard be applied consistently, whether the reviewer is a person or an AI model.

Can AutoQA Actually Score This Reliably, or Does It Need a Human?

AutoQA can score this reliably if, and only if, it's calibrated to reason the way a trained human QA analyst would on the exact same scorecard, with the customer's own policies as the reference material rather than generic industry benchmarks. This is the distinction worth being precise about. Most AutoQA tools on the market already claim to evaluate 100% of conversations, so coverage alone isn't a differentiator anymore; it's table stakes. The harder question is whether the AI's judgment on a specific ticket, "did this count as a recommendation or just an explanation," matches what the carrier's own compliance-trained QA analyst would conclude. RevelirQA is built around that specific gap. Revelir works with each customer to ingest their actual SOPs and licensing guidance into a retrieval layer, so the AI retrieves the customer's own policy definitions, not a generic insurance benchmark, before scoring each conversation. Revelir then calibrates the AI against that customer's human QA analysts until more than 95 out of 100 of its evaluations are independently validated as correct on a reviewed sample. That calibration step, not the 100% coverage number, is what tells a compliance team whether they can trust the AI's judgment on a nuanced advice-versus-service call.

Coverage still matters once that judgment bar is cleared. Manual sampling at 1-5% means a recurring pattern, say, three agents on a specific team routinely over-explaining deductible options in a way that edges toward recommendation, might never surface, because no single reviewed ticket looks alarming on its own. Scoring every conversation against the same scorecard means that pattern shows up as a trend across dozens of tickets instead of staying invisible inside the 95-99% that never gets reviewed.

What Should Carriers Do When AI Scoring Flags a Possible Licensing Issue?

Every flagged conversation needs a traceable reason an underwriting compliance officer can review without re-reading the whole transcript cold. A score with no explanation is nearly as unhelpful as no score at all, because a compliance team still has to manually reconstruct why the AI flagged it. This is why RevelirQA attaches a full reasoning trace to every evaluation: the prompt used, the specific policy document retrieved, and the reasoning that connects the two to the final score. For a flagged "recommendation language" result, that trace shows the exact sentence the AI identified, the SOP language it compared that sentence against, and why it concluded the line was crossed. That auditability is what turns an AI score from a black-box alert into something a compliance team can actually act on, document, and if needed, show to a regulator as evidence of an active monitoring program rather than a reactive one.

Frequently Asked Questions

Does QA for sales-adjacent conversations need to cover every channel, or is phone enough?
It needs to cover chat and email alongside phone. Digital-first carriers often route a large share of policyholder questions through text-based channels, and licensing risk doesn't care which channel a sentence was typed or spoken in [verequest.com].

Is a generic AI QA benchmark good enough for insurance compliance scoring?
No. Scoring needs to reference the carrier's own SOPs and licensing guidance, not generic industry benchmarks, because the service-versus-advice line depends on the carrier's specific products and state licensing rules.

What's the real penalty risk if this boundary gets crossed repeatedly?
Regulators have issued significant financial penalties for unlicensed solicitation and negotiation of insurance. The New York State Department of Financial Services fined an AIG subsidiary $12 million in 2021 for this exact issue.

Can an AI distinguish "explaining a policy" from "recommending a policy change"?
It can, if it's calibrated against a specific scorecard with worked examples and validated by the carrier's own QA analysts, rather than relying on generic sentiment or tone scoring.

How much of this risk does manual QA sampling actually catch?
Very little, structurally. Reviewing 1-5% of tickets means the specific conversation containing a compliance risk has a low chance of being the one a reviewer happens to pull.

Should human agents and AI chatbots be scored on the same scorecard?
Yes. As carriers deploy chatbots alongside human reps for routine policy questions, both need to be held to the same licensing-boundary criteria, since the regulatory risk doesn't depend on who or what generated the response.

What's the first step for a carrier that doesn't have this scorecard built yet?
Start by working with compliance to write explicit pass/fail examples for the two or three boundary scenarios that come up most often in support tickets, rather than trying to cover every possible scenario on day one.

About Revelir AI

Revelir AI builds RevelirQA, an AI AutoQA platform that scores 100% of customer service conversations against a company's own policies and SOPs, retrieved through a dedicated knowledge layer rather than generic benchmarks. Founded in 2025 and headquartered in Singapore, Revelir works closely with each customer to calibrate its scoring engine to over 95% accuracy, validated by that customer's human QA analysts, and is already running in production at enterprise clients including Xendit and Tiket.com across thousands of tickets per week. Every score carries a full reasoning trace covering the model, the documents retrieved, and the reasoning behind the result, which matters directly for regulated industries like insurance where QA decisions need to be auditable after the fact. The platform scores human agents and AI agents on the same scorecard, giving underwriting support teams one consistent view of where service ends and advice begins.

If your underwriting support team needs QA that can tell the difference between a helpful explanation and an unlicensed recommendation, and show its work, get in touch with Revelir AI to see how RevelirQA calibrates to your own policies and compliance standards.