AI, Unstructured Data and Emerging Insurance Technologies

Introduction

Insurance has traditionally depended on structured information: age, location, vehicle type, declared occupation, loss history and policy limits. Contemporary insurers increasingly receive information in other forms—images, documents, conversations, sensor streams, satellite observations, social-media text and machine-generated content. These sources can improve risk selection, claims settlement, fraud detection and customer service, but they also introduce uncertainty about data quality, privacy, explainability, bias and accountability.

This chapter examines how artificial intelligence (AI) and emerging insurtech capabilities can be used across the insurance value chain. The emphasis is not on technological novelty alone, but on the conditions under which a capability creates reliable business value and remains consistent with insurance principles, regulation and professional judgment.

Learning Objectives

After studying this chapter, the reader should be able to:

1. Explain the role of natural language processing, computer vision and speech analytics in insurance.
2. Evaluate telematics, IoT and satellite data for underwriting, claims and prevention.
3. Assess generative AI and foundation models in relation to accuracy, governance and operational risk.
4. Design human-in-the-loop workflows and appropriate control points.
5. Explain the uses and limitations of synthetic data.
6. Develop a responsible-adoption framework for emerging insurance technologies.

1. From Structured Records to Multimodal Insurance Data

Structured data fits predefined fields such as date of birth or premium amount. Unstructured data does not follow a fixed relational schema. Examples include claim narratives, medical reports, repair invoices, photographs, call recordings, emails, scanned forms and satellite images. Semi-structured data, such as JSON documents or telematics event logs, has some organization but may vary between sources.

A useful distinction is between the data object and the decision purpose. The same photograph may support property underwriting, post-loss verification or catastrophe response. Its value depends on whether it is authentic, current, sufficiently detailed and connected to a decision that the insurer is legally and operationally permitted to make.

A general predictive relationship may be represented as:

Here, ŷ is a predicted outcome, X is the observed input data, D represents data-quality and provenance information, and M represents the model and its configuration. This is a conceptual representation rather than a guarantee of causality. Predictions are useful only when the data-generating process is sufficiently similar to the environment in which the model is deployed.

Important dimensions of unstructured-data quality include:

2. Natural Language Processing

Natural language processing (NLP) enables systems to analyse human language. In insurance, common tasks include document classification, named-entity recognition, information extraction, summarisation, semantic search, translation and question answering.

2.1 Underwriting applications

NLP can extract construction details from survey reports, identify exclusions in contracts, classify commercial activities and compare an insured’s description with external risk documentation. It can also assist underwriters by locating relevant clauses in lengthy policy wordings.

The appropriate use is usually decision support, not unreviewed automatic acceptance or rejection. Language is context-dependent. A phrase such as “occasional storage” may have different significance depending on commodity, volume, fire protection and contractual conditions.

2.2 Claims applications

Claims handlers can use NLP to classify first-notice-of-loss descriptions, identify missing information, summarise correspondence and detect contradictions requiring investigation. Automated extraction from invoices may reduce rekeying and processing time. However, a model may mistake a claimant’s uncertainty, translation error or colloquial expression for inconsistency.

2.3 Evaluation

NLP performance should be evaluated at both model and business levels. Precision measures the proportion of flagged items that are correct; recall measures the proportion of relevant items that are identified. For a binary classification task:

TP denotes true positives, FP false positives and FN false negatives. These metrics are meaningful only when the target definition, sampling method and operating threshold are clear. A claims-fraud model with high precision but low recall may miss many suspicious claims; one with high recall may overwhelm investigators with false alarms.

3. Computer Vision

Computer vision interprets images and video. Typical insurance tasks include object detection, image classification, segmentation, damage estimation and document-image analysis.

In motor insurance, photographs may support vehicle identification, damage triage and repair estimation. In property insurance, aerial or roof images may identify vegetation proximity, roof condition or post-catastrophe damage. Medical imaging may assist specialist workflows, although clinical use requires particularly strong validation and professional oversight.

A vision model should not be treated as an objective camera-based truth machine. Image quality, angle, lighting, weather, repairs made before inspection and deliberate manipulation can all affect results. Models may also perform differently across vehicle makes, building materials, skin tones or geographic regions.

A robust workflow records:

1. the image source and capture time;
2. whether the image is original or transformed;
3. confidence and uncertainty indicators;
4. the evidence used for the recommendation;
5. the human override and its reason; and
6. subsequent outcome data for monitoring.

Image-based automation is particularly valuable for triage. Low-complexity, high-confidence claims can move rapidly through a controlled pathway, while ambiguous or high-severity cases are routed to specialists.

4. Speech Analytics and Conversational Systems

Speech analytics converts conversations into searchable or measurable information. Automatic speech recognition produces a transcript; language models can then identify intent, sentiment, compliance phrases, complaints or unresolved questions. Acoustic analysis may identify interruptions or stress indicators, but such inferences are often uncertain and should not be treated as reliable evidence of honesty or fraud.

Applications include call summarisation, quality assurance, agent coaching, multilingual service and identification of vulnerable-customer needs. A conversational AI system should disclose its nature where required, provide escalation to a human and avoid making promises outside its authority.

The principal control problem is that a transcript is an interpretation, not a perfect record. Accents, background noise, code-switching and specialist terminology can create material errors. Audio retention also raises privacy and cybersecurity concerns.

5. Telematics and Internet of Things

Telematics uses connected devices to observe movement or operating conditions. In motor insurance, variables may include distance, time of travel, acceleration, braking, cornering and location. In commercial insurance, sensors may measure temperature, vibration, pressure, machine status or driver behaviour. IoT systems extend this idea to buildings, factories, farms and supply chains.

A simple exposure rate can be expressed as:

where N is the number of observed events and E is the exposure, such as vehicle-kilometres, machine-hours or insured-days. The units of r must match the exposure definition. Comparisons are valid only when event definitions, observation periods and exposure measurement are consistent.

Telematics may support usage-based insurance, preventive alerts, dynamic fleet management and more accurate claims reconstruction. Yet continuous monitoring can create surveillance concerns. Location data may reveal religious attendance, health visits or employment patterns. Consent must be meaningful, collection proportionate and retention limited.

Sensor systems also fail. Devices may be disconnected, calibrated incorrectly or exposed to signal loss. Missingness is not automatically evidence of high risk. Treating non-reporting as misconduct can create unfair outcomes unless the contractual and technical basis is clear.

6. Satellite and Geospatial Data

Satellite, aerial and geospatial data can inform flood, wildfire, crop, coastal and property risk. Historical imagery supports exposure mapping; near-real-time imagery may assist catastrophe response; vegetation, elevation and land-use data can improve accumulation analysis.

The information is most effective when combined with policy records, engineering surveys and local knowledge. Spatial resolution, revisit frequency, cloud cover, geolocation error and classification uncertainty must be documented. A satellite-derived roof condition score should be treated as an indicator rather than a definitive inspection.

Geospatial models also raise distributional issues. Rural and low-income areas may have less frequent or lower-quality coverage. If a model systematically assigns poorer risk estimates to such areas, the insurer should investigate whether the result reflects genuine hazard, measurement inequality or historical bias.

7. Generative AI and Foundation Models

A foundation model is trained on broad data and adapted to many downstream tasks. Generative AI can produce text, code, images, summaries and structured outputs. In insurance it may draft customer communications, create claim summaries, search policy documents, generate underwriting questions and assist employees with internal knowledge retrieval.

The major risks are hallucination, unsupported certainty, prompt injection, confidential-data leakage, copyright concerns, model drift and excessive dependence on opaque suppliers. A fluent answer is not evidence of correctness. Generative systems should therefore be connected to approved sources where factual answers are required, and outputs should be checked against authoritative records.

A controlled architecture commonly includes:

• identity and access management;
• an approved knowledge base;
• retrieval and source citation;
• input and output filtering;
• logging and version control;
• human approval for consequential actions; and
• testing for bias, security and unacceptable content.

Generative AI should not independently determine coverage, liability, claim repudiation or premium changes unless the use has been specifically validated, governed and permitted. A useful principle is proportionality: the greater the financial, legal or personal consequence, the stronger the evidence, explanation and human review required.

8. Human-in-the-Loop Automation

Human-in-the-loop (HITL) design places people at defined points in an automated process. It is not satisfied merely by having an employee somewhere in the organization. The reviewer must have adequate information, authority, time and competence to challenge the system.

Three patterns are common:

Effective escalation rules may be triggered by low confidence, conflicting evidence, vulnerable customers, unusual policy conditions, high claim severity or suspected fraud. The reviewer’s decision and reason should be recorded. Otherwise, automation can become “rubber-stamping,” where staff approve outputs they do not understand.

9. Synthetic Data

Synthetic data is artificially generated data designed to reproduce selected characteristics of real data without directly reproducing every original record. It can support software testing, model development, rare-event research and privacy-preserving collaboration.

Synthetic data is not automatically anonymous. If generated records closely resemble training cases, membership or attribute disclosure may remain possible. Its usefulness should be tested against several criteria:

1. Statistical fidelity: does it preserve relevant distributions and relationships?
2. Utility: does a model trained on it perform on real, permitted test data?
3. Privacy: does it reduce re-identification and disclosure risk?
4. Coverage: does it represent rare but important outcomes?
5. Governance: are generation, access and permitted uses documented?

Synthetic data cannot correct a conceptual defect in the original data. If historical claims reflect discriminatory access or inconsistent reporting, generating more records from the same process may reproduce that problem at scale.

10. A Responsible-Adoption Framework

A technology should be adopted because it improves a defined insurance outcome, not because it is fashionable. A practical assessment has seven stages.

Stage 1: Define the decision. Specify the business purpose, affected parties, legal basis and consequence of error.

Stage 2: Map the data. Record sources, consent, provenance, sensitivity, retention, quality and cross-border transfers.

Stage 3: Establish a baseline. Compare the proposed system with current human performance, cost, speed and error rates.

Stage 4: Validate. Test out-of-sample performance, subgroup outcomes, robustness, calibration, cybersecurity and failure cases.

Stage 5: Design controls. Define thresholds, escalation, overrides, audit logs, access rights and incident response.

Stage 6: Pilot safely. Begin with a limited population and shadow operation where the system advises without changing decisions.

Stage 7: Monitor continuously. Track drift, complaints, overrides, adverse outcomes, data changes and supplier performance.

Model governance should distinguish performance risk from decision risk. A model can have strong average accuracy yet create unacceptable harm in a subgroup or in a high-consequence decision. Monitoring should therefore include outcome distributions and process measures, not only a single accuracy score.

Applied Case: Illustrative Motor Claims Triage System

Background

An insurer receives 18,000 private-motor claims each month. Claims are submitted through text, photographs, telephone calls and repair invoices. Manual allocation causes delays, while fraud investigators receive too many low-value alerts.

Problem

The insurer proposes an AI system combining NLP for claim descriptions, computer vision for photographs and rules based on policy coverage. The stated objectives are faster settlement, better routing and more consistent investigation.

Analysis

The insurer identifies three decision classes: straightforward eligible claims, claims requiring adjuster review and claims requiring specialist investigation. It finds that photographs are less reliable at night and that some customers submit images through intermediaries. Historical fraud labels are incomplete because investigated claims are not a random sample of all claims.

Approach

The system is first deployed in shadow mode. It recommends a class but does not alter settlement. Claims with low image quality, conflicting policy information, injury allegations or high estimated severity are automatically escalated. A human adjuster sees the extracted evidence, source images, confidence indicators and policy clauses used. The system cannot reject a claim or communicate a final liability decision without approval.

Performance is assessed separately by claim type, language, geography and customer channel. The insurer measures processing time, false escalation, missed escalation, complaint rates and payment accuracy. It also tests whether the system’s apparent fraud performance simply reflects historical investigation patterns.

Outcome

After the pilot, the insurer retains automation for document extraction and low-risk triage but limits automated settlement to tightly defined claims. It introduces quarterly reviews, a customer challenge route and a rule that missing or poor-quality images cannot alone justify adverse treatment.

Learning Points

The value came from workflow redesign, not from the model alone. Data provenance, escalation and customer recourse were as important as predictive performance. The case also illustrates why historical labels should be treated as observations of institutional decisions, not unquestionable ground truth.

Ethical, Legal and Managerial Issues

Insurance decisions affect access to protection and the distribution of financial burdens. Relevant principles include fairness, transparency, privacy, purpose limitation, accountability, security and non-discrimination. Requirements differ across jurisdictions, but high-impact automated decisions commonly attract stronger duties of explanation, documentation, human oversight and contestability.

Senior management should assign ownership for each AI system, including a business owner, data owner, model-risk owner, information-security lead and compliance representative. Third-party contracts should address data use, audit rights, service continuity, incident notification, subcontracting, model changes and deletion.

Explainability must be matched to the decision. A technical feature-importance chart may not explain a claim outcome to a customer. Customer-facing explanations should identify the material factors, relevant evidence, review route and correction process without exposing security-sensitive information.

Contemporary Developments and Best Practices

Emerging capabilities include multimodal models that combine text, image and audio; edge computing that analyses sensor data locally; digital twins for industrial risk; privacy-enhancing computation; federated learning; autonomous workflow agents; and climate-risk intelligence based on increasingly detailed geospatial data.

Best practice includes:

• begin with a measurable problem and a non-AI baseline;
• use the least intrusive data that can achieve the purpose;
• separate experimentation from production data;
• validate on realistic and disadvantaged subgroups;
• preserve human authority for consequential decisions;
• maintain model, prompt, data and supplier version histories;
• provide accessible correction and appeal routes;
• test failure modes before launch;
• monitor both technical and customer outcomes; and
• retire systems that no longer meet their purpose.

Chapter Summary

AI enables insurers to use language, images, speech, sensor readings, geospatial observations and generated content in underwriting, claims and service. NLP supports extraction and search; computer vision assists inspection and damage triage; speech analytics improves service oversight; telematics and IoT enable usage-based pricing and prevention; satellite data supports catastrophe and accumulation analysis; and generative AI assists knowledge work. These capabilities are probabilistic and fallible. Their adoption therefore requires provenance, validation, privacy protection, cybersecurity, subgroup testing, human oversight and customer recourse. Human-in-the-loop design must provide genuine authority rather than symbolic review. Synthetic data can improve testing and privacy but may reproduce the defects of its source. Responsible insurtech is ultimately a governance and decision-design challenge as much as a technical one.

Key Terms

Unstructured data: Information without a fixed tabular schema, such as text, images or audio.

Natural language processing: Computational analysis and generation of human language.

Computer vision: AI methods for interpreting images and video.

Speech analytics: Analysis of spoken conversations and their transcripts.

Telematics: Connected measurement of movement, location or operating behaviour.

Internet of Things: Networked physical devices that collect and exchange data.

Satellite data: Earth-observation information used for spatial and temporal risk analysis.

Foundation model: A broadly trained model adapted to multiple tasks.

Generative AI: AI that creates new text, images, audio, code or other content.

Hallucination: A confident but unsupported or incorrect model output.

Human-in-the-loop: Workflow in which a person reviews or controls material automated outputs.

Model drift: Deterioration caused by changes in data, behaviour or operating conditions.

Synthetic data: Artificially generated data designed for specified analytical purposes.

Data provenance: Evidence about the origin, transformation and custody of data.

Triage: Allocation of cases to different processing or review pathways.

Precision: The proportion of positive model alerts that are correct.

Recall: The proportion of relevant positive cases identified.

Explainability: The ability to communicate how a system contributed to an outcome.

Purpose limitation: Restricting data use to specified, justified purposes.

Contestability: The opportunity to question, correct or appeal a decision.

Review Questions

Short Answer Questions

1. Distinguish structured, semi-structured and unstructured data with insurance examples.
2. What are precision and recall, and why can they lead to different operational choices?
3. Identify two risks associated with computer-vision claims assessment.
4. Explain why synthetic data is not automatically anonymous.
5. What makes human-in-the-loop review meaningful?

Long Answer Questions

1. Evaluate NLP applications across underwriting, claims and customer service.
2. Discuss the benefits and limitations of telematics and IoT for usage-based insurance.
3. Explain how satellite data can improve catastrophe-risk management while creating fairness concerns.
4. Analyse the governance requirements for generative AI in insurance.
5. Develop a responsible-adoption framework for a proposed AI claims system.

Analytical / Application Questions

1. An insurer’s image model performs well overall but poorly for rural properties and older roofs. Design a validation and remediation plan.
2. A generative AI assistant produces a fluent but incorrect coverage explanation. Analyse the control failures and propose a safer architecture.
3. Design escalation rules for a motor-claims triage system using text, photographs and telematics while preserving customer contestability.

Multiple Choice Questions with Answers

Question 1

Which task is most directly associated with natural language processing in insurance?

A. Extracting policy clauses from claim correspondence

B. Measuring vehicle acceleration with an onboard sensor

C. Classifying roof materials from aerial pixels

D. Calibrating a temperature sensor

Correct Answer: A. Extracting policy clauses from claim correspondence

Question 2

Recall is calculated as:

A. TP ÷ (TP + FP)

B. TP ÷ (TP + FN)

C. TN ÷ (TN + FP)

D. (TP + TN) ÷ total observations

Correct Answer: B. TP ÷ (TP + FN)

Question 3

Which is an appropriate use of computer vision in claims?

A. Automatically inferring a claimant’s honesty from facial expression

B. Assisting vehicle-damage triage from photographs

C. Replacing all policy wording with image labels

D. Treating every low-quality image as evidence of fraud

Correct Answer: B. Assisting vehicle-damage triage from photographs

Question 4

Why should missing telematics data not automatically be treated as high risk?

A. Sensors never fail

B. Missingness may result from disconnection or signal loss

C. Telematics has no relationship to exposure

D. All missing data is legally irrelevant

Correct Answer: B. Missingness may result from disconnection or signal loss

Question 5

Which characteristic is a major risk of generative AI in insurance?

A. It always cites authoritative sources

B. It cannot produce text

C. It may generate fluent but unsupported answers

D. It eliminates the need for governance

Correct Answer: C. It may generate fluent but unsupported answers

Question 6

A human-in-the-loop system requires that the reviewer:

A. Approve every output without seeing evidence

B. Have authority and information to challenge material outputs

C. Be outside the insurer’s governance structure

D. Review only after the customer has been rejected

Correct Answer: B. Have authority and information to challenge material outputs

Question 7

Which statement about synthetic data is correct?

A. It is automatically free of privacy risk

B. It can reproduce defects in the source data

C. It cannot be used for software testing

D. It always has higher utility than real data

Correct Answer: B. It can reproduce defects in the source data

Question 8

Which satellite-data limitation should an insurer consider?

A. Cloud cover and revisit frequency

B. Guaranteed perfect geolocation

C. Complete absence of spatial uncertainty

D. Inability to support catastrophe analysis

Correct Answer: A. Cloud cover and revisit frequency

Question 9

What is the first stage in the responsible-adoption framework described in the chapter?

A. Define the decision and its purpose

B. Purchase the largest available model

C. Remove all human review

D. Publish results before validation

Correct Answer: A. Define the decision and its purpose

Question 10

A high-consequence automated claim decision should generally receive:

A. Less evidence because it is important

B. Stronger validation, explanation and human oversight

C. No audit logging

D. Only synthetic test data

Correct Answer: B. Stronger validation, explanation and human oversight