AI can help an insurer read documents, identify missing evidence, estimate damage and prioritise suspicious claims. It can also deny cover unfairly, amplify historical bias or turn an ordinary data error into a decision that is difficult for a customer to challenge.
The difference is not whether a model is “accurate” in a laboratory. It is whether the insurer can show what the system was allowed to decide, which evidence it used, how customers were affected and who can correct the outcome.
AI is already established across UK financial services. The Bank of England and FCA’s latest sector survey found AI in underwriting, pricing and claims handling, while also finding growing dependence on third-party models and incomplete understanding of some systems. The survey should not be treated as a league table, but it shows why governance must develop alongside adoption. See the Bank of England and FCA survey on AI in UK financial services.
Choose the Decision Before Choosing the Model
“Use AI in claims” is not a workable scope. A safe programme separates clerical assistance, decision support and decision authority.
| Insurance workflow | Useful AI role | Recommended decision boundary | Main failure to test |
|---|---|---|---|
| Proposal and document intake | Extract fields, identify missing evidence, classify documents | AI prepares the record; staff or established rules confirm material facts | Incorrect extraction silently changes the risk |
| Underwriting support | Rank cases, estimate risk or suggest questions | Human underwriter owns declines, exclusions and unusual premiums | A proxy variable disadvantages a customer group |
| Claims triage | Route urgency, complexity or likely fraud for review | AI prioritises; it does not label a claimant dishonest | False positives delay genuine claims |
| Damage assessment | Estimate repair type or damage extent from images and records | Straightforward cases may proceed inside a tested limit; disputes escalate | Poor images or unusual property produce false confidence |
| Customer communication | Draft updates and explain next steps | Approved facts and policy wording only; customer can reach a person | Plausible but incorrect coverage explanation |
| Fraud investigation | Connect anomalies and supporting evidence | Specialist investigator decides what the evidence means | Correlation is treated as proof |
Begin with a decision inventory. For every model output, record whether it is advice, a queue priority, a price input, a coverage decision or an action. Then identify the customer consequence if it is wrong.
A useful companion is AI Risk Management: Better Insurance Decisions Without Black Boxes, which covers the wider control model.
The UK Rules Apply to the Outcome, Not the Label
The FCA stated in June 2026 that it does not plan a separate AI rulebook for financial services. Firms are expected to apply existing frameworks, including the Consumer Duty, senior-management accountability and established governance and control expectations. That makes the operational question simpler: can the firm demonstrate good customer outcomes when AI is involved? See the FCA’s current position on AI in financial services.
For retail business, the Consumer Duty requires firms to act to deliver good outcomes, avoid foreseeable harm and support consumer understanding. The FCA’s insurance work has also emphasised that claims must be handled promptly and fairly, not merely cheaply. Its insurance claims-handling findings are a useful test for automation: a faster triage score is not a success if it creates avoidable delay, repeated evidence requests or inaccessible appeals.
Outcome monitoring is an ongoing obligation. The FCA’s July 2026 guidance says firms need to understand actual customer experience, detect emerging harm and act on it rather than simply producing dashboards. See FCA guidance on outcomes monitoring.
Data protection law also changed. The ICO confirmed that all data-protection provisions in the Data (Use and Access) Act 2025 were in force by 19 June 2026. The Act opens more lawful bases for significant automated decisions, potentially including legitimate interests in appropriate cases, but special-category data remains more restricted and safeguards still apply. The ICO explains the position in its DUAA guidance for organisations.
That change is permission to design responsibly, not permission to remove review. The ICO’s final updated automated-decision guidance is still expected in winter 2026, so teams should track it rather than freezing a March checklist into policy.
Test Fairness at Both Portfolio and Individual Level
Insurance legitimately differentiates by risk. The control question is whether the differentiation is relevant, supportable and fair.
A model can produce an acceptable average loss ratio while performing badly for a smaller group. It can also learn proxies for protected or vulnerable characteristics from postcode, occupation, device, purchasing behaviour or combinations of seemingly neutral variables.
Before release, test at least:
- quotation, acceptance and referral rates across relevant customer groups;
- premium and excess distributions, not only average model error;
- false fraud referrals and time spent in investigation;
- claim acceptance, partial settlement and rejection outcomes;
- requests for additional evidence;
- complaint, appeal and human-reversal rates;
- performance where data is incomplete, corrected or disputed;
- whether vulnerable customers experience additional friction.
The ICO says fairness covers both how personal data is processed and the effect of the resulting decision. It also warns that statistical accuracy alone does not establish fairness. Its AI fairness guidance should inform the data-protection assessment.
Protected-characteristic testing itself needs a lawful design. Involve legal, compliance and the data-protection officer before collecting or deriving sensitive attributes for bias analysis. Document why each attribute is needed, who can access it and when it will be deleted.
Make Human Review Capable of Changing the Result
A human approval click is not meaningful oversight if the reviewer cannot see the evidence, lacks authority or is expected to follow the model.
For every adverse or contested outcome, give the reviewer:
- the original customer and policy evidence;
- the model output and confidence or uncertainty information;
- the principal factors that materially influenced the result;
- known data-quality warnings and missing fields;
- the applicable underwriting or claims rule;
- a route to correct data and rerun the decision;
- authority to vary or overturn the recommendation;
- a reason field that records what the reviewer decided.
Customers also need a plain route to challenge significant decisions. The ICO describes suitable safeguards as including human intervention, the opportunity to express a view, contestability and an explanation. Its guidance makes clear that the reviewer needs the competence and authority to reassess the relevant evidence. See individual rights in AI systems.
For implementation detail, see Automated Decisions After the DUAA.
An Illustrative Claims Workflow
Consider a home escape-of-water claim. This is an illustrative design, not a claimed case study.
The system first validates policy status and extracts dates, invoices and descriptions. A vision model may estimate the affected area from photographs, while anomaly detection compares the submission with known patterns. None of those outputs establishes coverage or fraud.
A low-complexity claim can move to a settlement queue only when required evidence is present, the model is operating inside its validated range and no escalation condition applies. The case stops for review when:
- policy wording or causation is disputed;
- images are poor or inconsistent;
- the customer corrects a material fact;
- vulnerability or urgent accommodation needs are identified;
- fraud indicators are present;
- repair value crosses the approved authority;
- the model versions or source systems are unavailable.
The customer should receive the evidence still required, the next action, expected ownership and a human-contact route. The audit record should preserve the input version, model version, rules triggered, decision, reviewer and any subsequent correction.
Validate the Whole Service, Not Just Model Accuracy
| Control question | Useful measure | Release gate |
|---|---|---|
| Does triage find the intended cases? | Precision and recall by claim type | Agreed threshold on a representative holdout set |
| Does it create unequal friction? | Referral and false-positive rates by monitored cohort | Differences explained, justified and remediated |
| Can staff safely use it? | Override quality, review time and reviewer agreement | Reviewers demonstrate understanding and authority |
| Are customers better served? | End-to-end resolution, repeat contacts, complaints and appeals | No material deterioration against the pre-AI baseline |
| Can the decision be reconstructed? | Evidence and version completeness | Every sampled decision reproducible |
| Does it remain reliable? | Drift, missing-data and out-of-range rates | Alerts and a tested fallback before production |
Shadow the model against live work before allowing it to affect customers. Then introduce authority in stages: recommendation, supervised action and finally narrow automatic action where the evidence supports it.
Include difficult cases in the evaluation set: policy changes, rare losses, duplicate identities, poor-quality photographs, disputed facts, accessibility needs and customers who provide new evidence after the first decision.
Control Suppliers and Concentration Risk
Third-party insurance platforms can shorten delivery, but the regulated firm still owns the outcome. The Bank/FCA survey found that a third of reported AI use cases involved third-party implementations and that providers were concentrated in cloud, model and data services.
A supplier contract should therefore cover:
- permitted data use and retention;
- training on the insurer’s data;
- model and dependency changes;
- performance by relevant customer segment;
- security incidents and vulnerability disclosure;
- audit evidence and testing access;
- subcontractors and hosting regions;
- service continuity, export and exit;
- the insurer’s ability to suspend or reverse decisions.
Maintain a fallback that staff have actually practised. A theoretical manual process is not resilience if volumes, skills or source records make it unusable.
An AI assurance evidence pack can keep these artefacts reviewable.
A Twelve-Week First Deployment
Weeks 1–3: define the decision. Select one narrow queue with reliable historical outcomes. Map policy, data, customer harm and the current operational baseline.
Weeks 4–6: build and challenge. Create a time-separated evaluation set, test cohort outcomes and run deliberately difficult cases. Complete the data-protection and conduct assessments.
Weeks 7–9: shadow live work. Show recommendations to trained reviewers without changing the official decision. Investigate disagreement rather than averaging it away.
Weeks 10–12: release a bounded role. Allow the model to prioritise or prepare cases, keep consequential decisions with authorised staff and review customer outcomes weekly.
Expansion should depend on evidence: better resolution, fewer avoidable contacts, stable fairness results, correct escalation and complete audit trails.
The Standard for Useful Insurance AI
The strongest insurance system is not the one that automates the most decisions. It is the one that removes avoidable delay while making important decisions easier to understand, challenge and repair.
Underwriting and claims models should operate inside explicit authority, preserve source evidence, expose uncertainty and stop when the facts no longer fit. If the insurer can demonstrate those properties at customer level and portfolio level, AI becomes a disciplined operating capability rather than an invisible source of conduct risk.
