Exploratory Analysis and Insurance Performance Metrics

Introduction

Exploratory data analysis (EDA) is the disciplined process of examining data before formal modelling. In insurance, EDA is not merely a graphical exercise. It connects operational records with underwriting, pricing, claims, distribution, retention and capital decisions. A sound exploratory analysis identifies the structure of a portfolio, detects data-quality problems, reveals heterogeneity in risk and performance, and tests whether apparent relationships are credible.

Insurance performance must be assessed from several perspectives. A portfolio may show attractive premium growth but deteriorating loss ratios; high conversion but poor retention; or low current losses but substantial claims-development risk. Consequently, no single metric is sufficient. Managers must interpret frequency, severity, pure premium, loss ratio, combined ratio, retention, conversion, lifetime value and claims-development measures together.

This chapter develops a framework for segmenting and visualising insurance data, calculating major performance measures, and distinguishing correlation, association and causation. The emphasis is on analytical judgement: a metric is useful only when its denominator, time basis, exposure definition, data quality and business purpose are understood.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Design meaningful insurance segments and select suitable visualisations.
2. Calculate and interpret frequency, severity, pure premium, loss ratio and combined ratio.
3. Evaluate retention, conversion and customer lifetime value.
4. Explain incurred, paid and reported claims-development measures.
5. Distinguish correlation, association and causation in insurance analysis.
6. Diagnose bias, leakage, confounding, censoring and other limitations in exploratory work.
7. Translate exploratory findings into underwriting, claims, marketing and managerial action.

1. The Analytical Object: Exposure, Time and Unit of Analysis

Insurance data commonly combine policies, policyholders, vehicles, locations, transactions, claims and payments. These records do not represent the same analytical unit. A policy-year may be appropriate for pricing, a claim may be appropriate for severity analysis, and a customer may be appropriate for retention or lifetime-value analysis.

Three questions should be answered before calculating any metric:

• What is the unit? Policy, insured, vehicle, claim, transaction or customer?
• What is the observation period? Calendar year, accident year, underwriting year, policy term or development age?
• What is the denominator? Exposure, earned premium, written premium, eligible customers or quoted risks?

Exposure measures the amount of risk insured. For annual motor insurance, one vehicle insured for six months contributes approximately 0.5 vehicle-years of exposure, subject to the organisation’s exposure convention. If exposure is measured in policy-years, frequency and pure premium must not be compared with measures based on vehicle-years without adjustment.

Earned premium is the portion of premium relating to coverage already provided. Written premium records business written during a period and may include coverage that will be earned later. Loss ratios based on incurred losses should generally be compared with earned premium, not unadjusted written premium.

A useful analytical data model separates:

Data should be reconciled across these entities. For example, claim counts should be traceable to policy exposure, and premium totals should reconcile to financial records after cancellations, endorsements and refunds.

2. Segmentation for Exploratory Analysis

Segmentation partitions a portfolio into groups expected to differ in behaviour, risk or economics. Common insurance dimensions include product, geography, age band, vehicle type, occupation, distribution channel, policy tenure, coverage level, deductible, claim history and payment method.

A useful segment has four characteristics:

1. Business relevance: it supports a decision, such as pricing or retention intervention.
2. Statistical credibility: it contains enough exposure and claims for stable estimates.
3. Operational availability: its definition can be applied consistently at decision time.
4. Governance acceptability: it does not create unlawful or unjustified discrimination.

Segmentation can be descriptive, predictive or strategic. Descriptive segmentation reports observed differences. Predictive segmentation estimates future outcomes. Strategic segmentation combines risk, profitability, growth and service considerations. These should not be confused: a group with high historical loss frequency is not automatically a group that should be declined, because differences may reflect exposure mix, claims access, reporting behaviour or historical pricing.

Cross-tabulation and aggregation

A basic segment table should contain exposure, claim count, incurred loss, earned premium, expenses and customer counts. Rates should be calculated from aggregated numerators and denominators, rather than by averaging individual rates. For example, the portfolio frequency is total claims divided by total exposure, not the arithmetic mean of policy-level frequencies.

Small cells create unstable estimates. A segment with two claims in 20 exposure units has an observed frequency of 0.10, but uncertainty is much greater than in a segment with 1,000 claims in 10,000 exposure units. Analysts may combine sparse categories, use credibility methods, or present confidence intervals.

Visualisation principles

Appropriate visualisations include:

• histograms or density plots for claim severity;
• box plots for severity by segment, with extreme values shown separately;
• bar charts for frequency, conversion or retention;
• line charts for monthly loss ratios and claim counts;
• heat maps for geography-by-product comparisons;
• scatter plots for premium and loss relationships;
• cohort curves for retention and customer lifetime value;
• run charts and development triangles for claims emergence.

A chart must identify the measure, unit, period, denominator and population. A bar chart of claim counts can be misleading when segments have different exposure. Rates are often more informative, but rates can also mislead when the denominator is small. Logarithmic scales may help display highly skewed severity, but they should be labelled clearly and not used to conceal material differences.

3. Core Risk and Performance Metrics

3.1 Claim frequency

Claim frequency measures the number of claims relative to exposure:

where N is the number of claims and E is exposure. If exposure is in vehicle-years, frequency is claims per vehicle-year. It is often reported per 100 or 1,000 exposure units:

The measure assumes claims and exposure are defined on compatible bases. Frequency may refer to reported claims, closed claims or ultimate claims; the choice must be stated. Reported frequency is affected by reporting delays and claims notification practices.

3.2 Claim severity

Severity is the average cost per claim:

where L is the relevant loss amount and N is the claim count. Loss may be paid, incurred or ultimate. Incurred severity includes paid amounts plus case reserves; ultimate severity additionally reflects an estimate of future development.

Severity distributions are usually right-skewed. A few large claims can dominate the mean. Analysts should therefore examine the median, selected percentiles, large-loss frequency and the treatment of catastrophe claims. Removing large claims without disclosure produces an incomplete view of risk.

3.3 Pure premium or loss cost

Pure premium combines frequency and severity:

Thus, under consistent definitions, pure premium equals frequency multiplied by severity. It represents expected loss cost per exposure unit before expenses, profit, capital cost and other premium components. The identity is meaningful only when the same claim count, loss basis and exposure period are used in all terms.

Worked example: frequency, severity and pure premium

An insurer observes 240 reported claims over 12,000 vehicle-years. Incurred losses are $1,080,000.

Frequency = 240 ÷ 12,000 = 0.020 claims per vehicle-year, or 20 claims per 1,000 vehicle-years.

Severity = $1,080,000 ÷ 240 = $4,500 per claim.

Pure premium = $1,080,000 ÷ 12,000 = $90 per vehicle-year.

The cross-check is 0.020 × $4,500 = $90. This result is an incurred pure premium and should not be described as an ultimate expected loss cost unless development has been incorporated.

3.4 Loss ratio

The loss ratio relates losses to earned premium:

If incurred losses are $1.08 million and earned premium is $1.80 million, the loss ratio is 60%. The ratio may be calculated using paid, incurred or ultimate losses, but the terminology must be explicit. Loss adjustment expenses may be included or excluded depending on the organisation’s convention.

Loss ratio is affected by both risk and price. A high ratio may result from high claims, inadequate premium, adverse mix, under-reserving or a denominator problem. A low ratio may reflect favourable risk, high prices, delayed claims, reserve releases or insufficient earned exposure.

3.5 Expense ratio and combined ratio

The expense ratio is commonly expressed as:

The combined ratio is:

For example, a 60% loss ratio and a 32% expense ratio produce a 92% combined ratio. A combined ratio below 100% indicates an underwriting surplus before items excluded from the calculation; it does not necessarily imply total corporate profitability, because investment income, taxes, reinsurance, capital costs and other items may remain.

Expense definitions differ. Acquisition costs, administrative expenses, premium taxes and claims-handling expenses may be treated differently across firms and jurisdictions. Comparisons require consistent accounting bases.

4. Customer and Distribution Metrics

4.1 Conversion

Conversion measures movement from an eligible opportunity to a sale:

A quote-to-bind conversion rate should use quotes meeting the defined eligibility criteria. If marketing leads, quotes and applications are mixed, the denominator changes and the result becomes ambiguous. Conversion should be analysed by channel, product, price position, customer profile and time to bind.

High conversion is not automatically beneficial. Aggressive discounts may increase conversion while worsening loss ratio or reducing lifetime value. Conversion should therefore be paired with contribution margin, persistency and claims performance.

4.2 Retention

A simple policy retention rate is:

The eligibility definition must exclude policies that cannot renew for reasons outside the customer’s choice, such as product withdrawal or death. Retention can be measured at policy, customer or exposure level. A customer-level measure is preferable where customers hold multiple policies.

Retention is time-dependent. A cohort retention curve reports the proportion remaining after one month, one year or successive renewal opportunities. Analysts should distinguish voluntary lapse, insurer non-renewal, cancellation and payment failure.

4.3 Customer lifetime value

Customer lifetime value (CLV) estimates the expected discounted future contribution from a customer. A simplified discrete model is:

where C₀ is acquisition cost at time zero, p_t is the probability the customer remains active in period t, m_t is expected contribution margin conditional on activity, s_t is expected service or retention cost, r is the discount rate per period and T is the horizon.

The formula is an expected-value model. It assumes that the estimated probabilities, margins, costs and discount rate are appropriate and that cash flows are measured consistently. If margins already include acquisition costs, C₀ must not be subtracted again. CLV should not be based solely on premium; claims, expenses, commissions, capital usage and expected cancellations matter.

A perpetual constant-margin special case is sometimes written as:

where p is a constant period-to-period survival probability and m is constant contribution margin. This special case is unsuitable when retention, premium or claims vary materially by tenure.

5. Claims Development and Reserving-Oriented Measures

Claims mature over time. A claim reported today may be paid in stages, revised through case reserves and supplemented by incurred-but-not-reported (IBNR) claims. Exploratory analysis must therefore distinguish accident date, report date, payment date and valuation date.

Paid, incurred and ultimate loss

• Paid loss: cash paid by the valuation date.
• Case-incurred loss: paid loss plus case outstanding reserve.
• Incurred loss: commonly case-incurred loss, sometimes including specific adjustments according to company convention.
• Ultimate loss: estimated final cost after all future development.

A development triangle organises claims by accident or underwriting period and development age. For example, a paid triangle may have accident years in rows and 12-, 24- and 36-month development in columns. Missing lower-right cells represent immature observations.

A cumulative development factor from age j to age k can be expressed as:

If historical development patterns are applicable and ultimate loss is estimated from a loss observed at age j, then:

These relationships are not universal laws. They require reasonably stable claims processes, adequate data and an assumption that selected historical development factors are relevant to the period being estimated. Changes in settlement speed, case reserving, legal environment, inflation, mix or catastrophe exposure can invalidate simple extrapolation.

Development diagnostics

Useful exploratory measures include paid-to-incurred ratio, reported-to-ultimate ratio, average claim age, closure rate, reopened-claim rate and reserve strengthening or release. A low paid-to-incurred ratio may indicate immature claims, slow settlement or conservative case reserves; interpretation requires operational knowledge.

6. Correlation, Association and Causation

These terms are related but not interchangeable.

• Association means that two variables show a statistical relationship, such as higher claim frequency among urban policyholders.
• Correlation is a numerical measure of linear co-movement. Pearson’s correlation is:

where Cov(X,Y) is covariance and σ_X, σ_Y are standard deviations. Pearson correlation is sensitive to outliers and measures linear association, not general dependence.

• Causation means that changing one factor would alter the outcome under a defined causal interpretation, holding relevant conditions appropriately controlled.

Correlation does not establish causation. A relationship between telematics participation and lower losses may reflect safer drivers choosing telematics, rather than telematics causing safer driving. This is self-selection confounding. A relationship between premium and claims may reflect underwriting risk: high-risk customers may receive higher premiums, so premium is not necessarily causing losses.

Potential explanations for an observed relationship include confounding, reverse causality, selection bias, measurement error, seasonality, aggregation effects and chance. Causal analysis may use randomised interventions, natural experiments, difference-in-differences, instrumental variables or carefully specified longitudinal models. Exploratory analysis should generate hypotheses and identify adjustment variables; it should not overstate causal conclusions.

Simpson’s paradox is especially relevant. A relationship observed in aggregated data can reverse after stratifying by a third variable. For example, a channel may appear to have worse loss performance overall because it sells more high-risk products, while performing better within every product category.

7. Integrated Worked Application

Consider an illustrative commercial motor portfolio for one underwriting year:

Frequency = 300 ÷ 10,000 = 0.03 claims per vehicle-year, or 30 per 1,000 vehicle-years.

Severity = $1,500,000 ÷ 300 = $5,000 per claim.

Pure premium = $1,500,000 ÷ 10,000 = $150 per vehicle-year.

Loss ratio = $1,500,000 ÷ $2,500,000 = 60%.

Expense ratio = $750,000 ÷ $2,500,000 = 30%.

Combined ratio = 60% + 30% = 90%.

Retention rate = 6,000 ÷ 8,000 = 75%.

The portfolio appears technically profitable on the stated underwriting basis, but the conclusion is conditional. The losses are incurred rather than ultimate; expenses may be incomplete; and retention does not reveal whether lapsing customers are profitable or unprofitable. Segment analysis might show that the 90% combined ratio is produced by a profitable low-risk segment offsetting an underpriced high-risk segment. Management should therefore examine segment-level ratios, claim development, renewal price changes and customer contribution rather than acting on the aggregate alone.

8. Illustrative Case: Meridian Mutual’s Renewal Portfolio

Background

Meridian Mutual, an illustrative insurer, observes that its digital channel has a 48% quote-to-bind conversion rate, compared with 31% for its intermediary channel. Senior management proposes moving marketing expenditure to digital and reducing intermediary support.

Problem

The digital channel’s conversion is higher, but its first-year retention is 61%, compared with 78% for intermediated business. Its incurred loss ratio is 74%, compared with 59% for the intermediary channel. The channels also serve different age, geography and vehicle segments.

Analysis

The analytics team constructs a policy-level dataset containing quote date, channel, risk characteristics, premium, exposure, claims, earned premium, renewal status and cancellation reason. It creates segment-adjusted tables by product, territory and tenure. It also plots cohort retention and examines severity distributions rather than relying only on mean claims.

The team finds that digital customers are more concentrated in high-mileage urban risks and are more likely to purchase lower deductibles. A raw channel comparison therefore contains mix differences. Within comparable product-territory bands, digital conversion remains higher, but the loss-ratio gap narrows substantially. A logistic retention model is used for prediction, while a separate causal design is considered before claiming that channel causes retention differences.

Approach and solution

Meridian retains digital investment but changes the objective from conversion alone to profitable conversion. It introduces a dashboard containing conversion, first-renewal retention, pure premium, loss ratio, expense ratio, combined ratio and estimated CLV. It tests clearer renewal communications and deductible options through a controlled pilot, with regulatory review of fairness and disclosure.

Outcome

After two renewal cycles, the pilot increases retention among profitable digital customers and reduces acquisition spending on opportunities with negative expected contribution. The company does not treat the observed pilot result as universally causal until exposure, seasonality and implementation differences are assessed.

Learning points

1. A funnel metric must be linked to downstream profitability.
2. Channel comparisons require mix adjustment.
3. Cohort retention is more informative than a single aggregate rate.
4. Prediction and causal evaluation answer different questions.
5. Governance is required when segmentation influences customer treatment.

9. Limitations, Governance and Professional Practice

Exploratory metrics are vulnerable to incomplete exposure, duplicate claims, inconsistent reserve practices, cancelled policies, missing values and changing definitions. Claims may be reported late, creating apparent improvements that disappear after development. Inflation and social inflation can increase severity even when claim counts are stable. Catastrophe events can dominate annual results and should be analysed separately as well as included in total portfolio performance.

Analysts should maintain a metric dictionary specifying numerator, denominator, date basis, inclusion rules, currency, inflation treatment and data owner. Version-controlled code, reconciliations to finance and reproducible extracts are essential. Every dashboard should display the valuation date and materiality thresholds.

Fairness and regulation require special care. Protected characteristics may be prohibited from direct use, yet proxy variables such as postcode, occupation or device information can reproduce their effects. Analysts should test disparate impact where legally and ethically appropriate, document legitimate business rationale, provide explanations suitable for the decision, and establish human review and appeal processes.

Artificial intelligence can support anomaly detection, automated segmentation and text-based claims exploration, but it does not remove the need for actuarial judgement. Models must be monitored for drift, leakage and stability. A predictive variable that becomes known only after a claim or renewal decision is made cannot be used legitimately for that decision. Privacy, data minimisation, cybersecurity and retention obligations also apply.

Best-Practice Workflow

A defensible exploratory study generally follows this sequence:

1. Define the management question and decision.
2. Specify unit, period, exposure and outcome definitions.
3. Reconcile source data and document exclusions.
4. Examine distributions, missingness, outliers and duplicates.
5. Produce aggregate and segment-level metrics.
6. Visualise both counts and denominator-adjusted rates.
7. Test sensitivity to claims basis, development and large losses.
8. Investigate confounding and avoid causal overclaiming.
9. Validate findings with underwriting, claims, finance and compliance experts.
10. Translate evidence into an action, monitoring plan and documented limitation.

Chapter Summary

Exploratory analysis provides the foundation for credible predictive analytics in insurance. Its central discipline is alignment: claims must be aligned with exposure, losses with the correct premium basis, and observations with the relevant time and unit. Frequency measures claims per exposure; severity measures cost per claim; and pure premium combines the two. Loss ratio and combined ratio assess underwriting performance but depend on accounting and reserving conventions. Conversion, retention and CLV connect customer behaviour to economic value. Claims-development measures recognise that observed losses mature over time and that ultimate outcomes require assumptions.

Segmentation and visualisation reveal heterogeneity, but sparse data, mix differences and changing operational processes can mislead. Correlation and association are useful descriptive tools, whereas causation requires stronger design and assumptions. Professional practice therefore combines quantitative calculation, data governance, fairness review, operational knowledge and transparent communication.

Key Terms

Exposure: The amount of risk insured, such as vehicle-years or policy-years.

Earned premium: Premium relating to coverage provided during a period.

Written premium: Premium recorded when coverage is written, before it is necessarily earned.

Claim frequency: Claims divided by compatible exposure.

Claim severity: Loss divided by claim count.

Pure premium: Loss per exposure unit, also called loss cost.

Loss ratio: Incurred or specified losses divided by earned premium.

Expense ratio: Underwriting expenses divided by earned premium.

Combined ratio: Loss ratio plus expense ratio.

Conversion rate: Policies sold divided by qualified opportunities.

Retention rate: Renewed eligible policies divided by policies eligible for renewal.

Customer lifetime value: Discounted expected future customer contribution less acquisition and relevant costs.

Development triangle: Arrangement of claims data by origin period and development age.

Case reserve: Estimate of unpaid cost for a reported claim.

IBNR: Incurred-but-not-reported claims or associated reserve.

Ultimate loss: Estimated final cost after claims have fully developed.

Association: A statistical relationship between variables.

Correlation: A numerical measure of co-movement, commonly linear co-movement.

Causation: A relationship in which changing one factor affects an outcome under a defined causal interpretation.

Data leakage: Use of information unavailable at the time a decision or prediction is made.

Review Questions

Short Answer Questions

1. Why must the unit of analysis and exposure basis be specified before calculating frequency?
2. Define pure premium and explain its relationship to frequency and severity.
3. What is the difference between earned premium and written premium?
4. Why can a high conversion rate be commercially undesirable?
5. Distinguish association from causation.

Long Answer Questions

1. Explain how segmentation and visualisation can be combined to diagnose portfolio performance. Discuss the dangers of sparse cells and inconsistent denominators.
2. Derive and interpret the relationship among frequency, severity and pure premium, including assumptions required for the identity to hold.
3. Discuss the construction and limitations of loss ratio and combined ratio comparisons across insurers or products.
4. Explain how claims-development triangles support interpretation of paid, incurred and ultimate losses.
5. Develop a governance framework for using predictive segmentation in renewal and marketing decisions.

Analytical / Application Questions

1. A portfolio has 400 claims, 20,000 exposure units, $2,400,000 incurred losses and $4,000,000 earned premium. Calculate frequency, severity, pure premium and loss ratio. Interpret the results and identify one limitation of each measure.
2. A channel has 50% conversion, 62% first-year retention and a 72% loss ratio. Another has 35% conversion, 80% retention and a 58% loss ratio. Design an analysis to determine which channel should receive additional investment without relying on conversion alone.
3. An insurer observes lower claims among customers using a telematics device. Explain at least three non-causal explanations and propose a study design that would better assess whether telematics changes driving risk.

Multiple Choice Questions with Answers

Question 1

An insurer records 150 claims over 10,000 vehicle-years. What is the claim frequency per 1,000 vehicle-years?

A. 1.5

B. 15

C. 150

D. 666.7

Correct Answer: B. 15

Question 2

Incurred losses are $900,000 and the claim count is 180. What is incurred average severity?

A. $2,000

B. $5,000

C. $720,000

D. $162,000

Correct Answer: B. $5,000

Question 3

Which metric equals losses divided by exposure?

A. Expense ratio

B. Retention rate

C. Pure premium

D. Conversion rate

Correct Answer: C. Pure premium

Question 4

Earned premium is $2 million, incurred losses are $1.1 million and underwriting expenses are $500,000. What is the combined ratio?

A. 55%

B. 75%

C. 80%

D. 105%

Correct Answer: D. 105%

Question 5

Which denominator is most appropriate for a standard policy renewal retention rate?

A. All marketing leads

B. Policies eligible for renewal

C. Total claims

D. Earned premium

Correct Answer: B. Policies eligible for renewal

Question 6

Why should a claim-development analysis distinguish paid, incurred and ultimate losses?

A. They always have identical values

B. Claims mature and financial estimates change over time

C. Only paid losses are recorded in insurance

D. Ultimate losses exclude future payments

Correct Answer: B. Claims mature and financial estimates change over time

Question 7

A high correlation between telematics participation and low claims does not prove that telematics caused lower claims because of possible:

A. Confounding and self-selection

B. Unit consistency

C. Premium earning

D. Claims closure

Correct Answer: A. Confounding and self-selection

Question 8

Which statement about a combined ratio below 100% is correct?

A. It guarantees total corporate profit

B. It indicates an underwriting surplus on the included basis

C. It means investment income is negative

D. It proves reserves are adequate

Correct Answer: B. It indicates an underwriting surplus on the included basis

Question 9

In a lifetime-value model, the discount rate is used primarily to:

A. Convert future contributions to present value

B. Increase claim frequency

C. Remove all acquisition costs

D. Calculate claim severity

Correct Answer: A. Convert future contributions to present value

Question 10

Which practice is an example of data leakage?

A. Using policy inception date in a pricing model

B. Using exposure to calculate frequency

C. Using a future claim settlement outcome to make an earlier underwriting decision

D. Stratifying loss ratio by product

Correct Answer: C. Using a future claim settlement outcome to make an earlier underwriting decision