Customer and Shopper Analytics

Introduction

Retail competition increasingly depends on understanding not only what customers buy, but also why, when, where and under what conditions they buy. Customer analytics examines individuals or households across their relationship with a retailer. Shopper analytics focuses on the buying mission, visit, browsing episode or transaction occasion. The distinction matters: a loyal customer may behave differently across a weekly grocery mission, a seasonal purchase and an online search session.

Customer and shopper analytics converts behavioural data into decisions about acquisition, assortment, pricing, communication, loyalty and service. Its value lies not in producing dashboards alone, but in improving economically and ethically defensible decisions. This chapter develops an integrated framework covering RFM analysis, segmentation, personas, customer lifetime value, acquisition, retention, churn, loyalty, attribution, propensity modelling and market-basket analysis.

Learning Objectives

By the end of this chapter, students should be able to:

1. Distinguish customer, shopper, transaction and household perspectives.
2. Construct and interpret RFM scores and customer segments.
3. Design useful personas without confusing them with evidence-based segments.
4. Estimate customer lifetime value and evaluate acquisition and retention decisions.
5. Analyse churn, loyalty and campaign attribution critically.
6. Explain propensity modelling and market-basket analysis, including their limitations.
7. Translate analytical outputs into personalised retail strategies while protecting privacy and fairness.

1. The Analytical Object: Customer, Shopper and Household

A customer is an identifiable relationship with a retailer, often represented by an account, loyalty ID or email address. A shopper is a person or group involved in a particular shopping occasion. A buyer is the individual who completes a transaction, while a user or consumer may be the person who ultimately uses the product. These identities can differ. A parent may buy children’s products, a procurement employee may buy for an organisation, and several family members may share one loyalty account.

Retail data commonly includes point-of-sale transactions, loyalty records, website events, mobile-app behaviour, customer-service contacts, promotion exposure, returns and location data. The analytical unit must therefore be selected deliberately. A transaction-level analysis answers questions about baskets; a customer-level analysis answers questions about relationships; an occasion-level analysis answers questions about missions and context.

A useful customer data model links four levels:

The distinction between observed and inferred information is essential. A purchase is observed; “price sensitive” is an inference. A customer’s age may be declared, estimated or unavailable. Analytical systems should preserve the source, date and confidence of each important attribute.

2. From Data to Customer Intelligence

A reliable analytical process contains five stages:

1. Define the decision. For example, whether to send a win-back offer.
2. Define the population and unit. Include active customers, lapsed customers or all visitors, and specify the observation window.
3. Prepare the data. Remove duplicates, reconcile refunds, identify test orders and establish a consistent customer ID.
4. Model and validate. Use descriptive, predictive or causal methods appropriate to the decision.
5. Activate and learn. Deploy an intervention, measure incremental outcomes and revise the model.

This sequence prevents a common failure: beginning with available data rather than with a commercial decision. A model may achieve high predictive accuracy yet be useless if the retailer cannot act on it, if the intervention costs more than the expected benefit, or if the model identifies correlation rather than causal impact.

3. RFM Analysis

RFM is a practical descriptive framework based on three customer-level variables:

• Recency (R): how recently the customer purchased, usually measured as days since the last purchase.
• Frequency (F): number of qualifying purchases during a defined period.
• Monetary value (M): value generated during that period, commonly net sales or contribution margin.

For customer i, a simple formulation is:

where T₀ is the analysis date and t_i is the date of the customer’s most recent qualifying purchase. Recency is measured in days if both dates are expressed in days. Lower recency is generally better.

where n_i is the number of qualifying orders in the observation window. If line items are counted instead of orders, the interpretation changes and should be labelled accordingly.

where v_ij is the net value of order j for customer i. Net value should state whether it includes tax, delivery charges, discounts, refunds and returns. For managerial decisions, contribution margin is often more useful than revenue:

RFM values are often converted into quintile scores from 1 to 5. Recency is scored in reverse order: the most recent group receives 5. Frequency and monetary value are normally scored so that larger values receive higher scores. A customer with scores R=5, F=4 and M=5 is highly recent, frequent and valuable.

Worked Example: RFM Scoring

Assume a retailer ranks 1,000 customers into five equal groups. Customer A purchased 12 days ago, made 9 orders and generated £720 net sales. The customer falls in the best recency quintile, fourth frequency quintile and fifth monetary quintile. The RFM code is therefore 545. Customer B purchased 180 days ago, made 2 orders and generated £90; the code is 122 if those values fall in the respective groups.

The code is not a probability, a causal measure or a profitability calculation. It is a compact ranking device. Its usefulness depends on an appropriate observation window and on handling customers with different buying cycles. A 90-day recency threshold may indicate serious lapse in fashion retail but normal behaviour in furniture retail.

RFM Strengths and Limitations

RFM is transparent, inexpensive and easy to operationalise. It supports initial targeting, prioritisation and customer-value discussions. However, it ignores product category, margin, returns, channel, household composition and reasons for purchase. Equal-sized quantiles can also disguise meaningful differences: the top fifth may contain customers ranging from barely above average to extremely valuable.

Best practice is to supplement RFM with category-specific purchase cycles, margin, service costs, returns and customer-level experimentation. RFM should be treated as a starting segmentation tool, not as a complete customer strategy.

4. Segmentation and Personas

Segmentation divides a heterogeneous market into groups that are sufficiently similar for a distinct managerial action. Useful segmentation may be:

• Demographic: age, income or household structure.
• Geographic: region, catchment or climate.
• Psychographic: values, attitudes or lifestyle.
• Behavioural: frequency, category mix, channel, promotion response or service use.
• Needs-based: shopping mission, convenience need, quality expectation or risk concern.
• Value-based: current margin, potential value or cost to serve.

Effective segments should be measurable, substantial, accessible, differentiable and actionable. A segment named “people who like good products” is not operationally useful unless it can be identified and served differently.

Clustering methods such as k-means can group customers according to selected variables. In k-means, each observation is assigned to the nearest centroid and centroids are updated iteratively to minimise within-cluster squared distance. The method assumes that the chosen numeric representation is meaningful and that distance reflects similarity. It is sensitive to scaling, outliers and the chosen number of clusters. It should not be presented as discovering natural, permanent customer types.

Personas are narrative representations of important segments. A sound persona combines evidence, a goal, barriers, context, preferred channels and implications for action. For example, “Time-Pressed Planner” might be supported by high online research, repeated scheduled purchases, low tolerance for delivery failure and strong response to replenishment reminders. A persona is a communication and design device; it does not replace the underlying data or justify stereotyping.

5. Customer Lifetime Value

Customer lifetime value (CLV) estimates the future economic value of a customer relationship. It is a decision metric, not a directly observed fact. A simple discounted formulation is:

Here, p_t is the probability that the customer remains active or purchases in period t, m_t is expected contribution margin conditional on activity, c_t is period-specific servicing or retention cost, d is the discount rate per period, T is the forecast horizon, and CAC is customer acquisition cost where the decision requires net value after acquisition. The formula assumes that estimates are defined on the same time basis and that contribution margin excludes costs already represented elsewhere.

A simplified perpetuity model is sometimes used when contribution per period and retention probability are constant:

where r is the probability of remaining active from one period to the next, m is expected contribution margin per period and d is the periodic discount rate. This model is highly restrictive: it assumes constant margin, retention and cost, and an indefinite horizon. It should not be used uncritically for seasonal or rapidly changing businesses.

Worked CLV Example

Suppose a customer generates expected contribution margin of £40 per quarter, has a quarterly retention probability of 0.80, and the quarterly discount rate is 0.02. Ignoring acquisition cost and using the simplified model:

If acquisition cost is £35, net CLV is approximately £110.45. The result is sensitive to retention and margin assumptions. A small overestimate of retention can materially inflate value, particularly when retention is high.

CLV supports decisions about acquisition bids, service levels and retention investment. It should not be used to deny basic service to low-value customers or to justify discriminatory treatment. Predicted value is also not the same as a customer’s human worth.

6. Acquisition, Retention and Churn

Customer acquisition cost (CAC) is commonly calculated as:

The numerator should define whether it includes media, creative, agency, incentives, technology and allocated labour. The denominator should exclude duplicate accounts and customers who were already active. CAC should be compared with expected incremental CLV, not merely with first-order revenue.

Retention is the proportion of customers active at the end of a period who were also active at the beginning, subject to a clear definition of active. A basic cohort retention rate is:

Churn can be defined as the complement of retention for a specified period, but retail churn is often difficult because customers do not have fixed contract end dates. A customer who has not purchased for 60 days may be dormant, seasonal or permanently lost. Churn definitions should therefore reflect expected purchase intervals and category seasonality.

Survival analysis is useful because it models time until an event such as lapse. A survival function can be written as:

where T is the time to churn and S(t) is the probability of remaining active beyond time t. A hazard function describes the instantaneous risk of churn conditional on surviving to that time. Such models can incorporate censoring, where a customer has not yet churned by the end of observation. The assumptions and interpretation must be explained to decision-makers; a model is not automatically valid because it is statistically sophisticated.

Retention interventions should be tested against a control group. A win-back email may coincide with natural reactivation, seasonal demand or competitor disruption. Incremental retention is the difference between the treatment group’s outcome and the comparable control group’s outcome, not the total number of treated customers who returned.

7. Loyalty Analytics

Loyalty programmes generate identification, behavioural history and opportunities for recognition. Their economics depend on incremental purchasing, margin, reward cost, breakage, programme administration and privacy obligations. A programme is not successful merely because membership is large.

Useful measures include active-member rate, purchase frequency, share of wallet where measurable, reward redemption, incremental margin and retention relative to comparable non-members. Self-selection creates a major problem: customers who join may already be more loyal. Randomised enrolment offers are rare, so analysts may use matched comparisons, pre-post designs or quasi-experimental methods, while acknowledging residual bias.

Personalisation should be evaluated across customer value and customer experience. Excessive targeting can create message fatigue, price discrimination perceptions and a “filter bubble” in assortment exposure. Customers should receive meaningful control over communication preferences and data use.

8. Attribution and Incrementality

Attribution assigns credit for a conversion to marketing touchpoints. Common rules include first-touch, last-touch, linear, position-based and time-decay attribution. These are conventions, not proof of causation. Last-touch attribution tends to over-credit the final click and under-credit awareness activity; first-touch has the reverse problem.

The strongest practical evidence usually comes from experiments. If treatment conversion is 8% and control conversion is 6%, the absolute incremental lift is 2 percentage points and the relative lift is approximately 33.3%. If the treatment group has 10,000 eligible customers, estimated incremental conversions are 10,000 × 0.02 = 200, assuming random assignment and comparable exposure.

Attribution must also consider holdout groups, cross-device identity, offline sales, delayed conversions, organic demand and cannibalisation. A predictive model can estimate who is likely to convert; it does not establish that advertising caused the conversion. This distinction is central to marketing investment decisions.

9. Propensity Modelling

A propensity model estimates the likelihood of a defined event, such as purchase, response, churn or redemption, given observed features. Logistic regression is a common interpretable model:

Here, Y=1 denotes the event, x_k are predictor variables, β_k are estimated coefficients and β₀ is the intercept. The output is a probability under the model and data conditions; it is not a guarantee.

Potential predictors include recency, category breadth, discount dependence, service contacts, browsing patterns and fulfilment failures. Features must be available before the decision point. Including a variable created after purchase causes leakage and produces misleading performance.

Model evaluation should include discrimination, calibration, lift, stability, cost-benefit value and fairness across relevant groups. Accuracy alone is inadequate, particularly when the event is rare. A churn model may identify customers likely to leave, but a retention action should target customers for whom intervention changes behaviour. This is the difference between propensity and uplift modelling. Uplift modelling estimates differential treatment effect and can prevent wasting incentives on customers who would purchase anyway.

10. Market-Basket Analysis

Market-basket analysis identifies products that occur together in transactions. For itemsets A and B, support is:

Confidence is:

Lift is:

Support measures prevalence, confidence measures the conditional proportion of A transactions containing B, and lift compares that proportion with B’s overall prevalence. Lift above 1 indicates positive association, but not causation. A high lift with extremely low support may have little commercial value.

Worked Basket Example

Assume 1,000 baskets: 200 contain coffee, 300 contain milk and 120 contain both. Support for coffee and milk is 120 ÷ 1,000 = 12%. Confidence for coffee → milk is 120 ÷ 200 = 60%. Lift is 0.60 ÷ 0.30 = 2.0. Thus, milk appears twice as prevalent among coffee-buying baskets as in all baskets. The result may support a recommendation or adjacency test, but it does not prove that placing the products together caused sales.

Rules should be filtered by support, lift, margin, inventory availability and business relevance. Association can be distorted by promotions, seasonality, store layout and household stock-up behaviour. Basket analysis is therefore most useful when combined with controlled merchandising tests.

11. Integrated Personalisation Framework

A mature customer strategy connects the methods rather than using them in isolation. RFM may identify high-value recent customers; category and channel data can refine the segment; a propensity model can estimate response; CLV can determine economic priority; an experiment can estimate incremental impact; and basket analysis can select relevant products.

A practical decision sequence is:

Figure: An iterative customer analytics decision process.

The framework highlights that learning is continuous. Customer data should be refreshed, models monitored for drift, interventions evaluated and policies revised when outcomes or social expectations change.

12. Illustrative Case: Northstar Home Retail

Background. Northstar Home Retail sells furniture and home accessories through stores and online channels. Its loyalty database contains 600,000 identified customers, but repeat purchase has declined and promotional cost has increased.

Problem. Management wants to improve repeat sales without sending discounts to customers who would purchase anyway. It also wants to understand whether its loyalty programme creates incremental value.

Analysis. The analytics team defines a 12-month customer window, excludes cancelled orders and calculates net contribution rather than gross revenue. RFM identifies recent high-frequency customers, while category analysis distinguishes furniture cycles from frequent accessory purchases. A churn model is developed using only information available before the campaign. The team also estimates CLV using category-specific margin and retention assumptions.

A randomised design divides eligible dormant customers into a personalised reminder group, an incentive group and a no-contact control group. Basket analysis suggests complementary lamps and cushions for customers who purchased sofas, but recommendations are constrained by inventory and margin. Programme members are compared with non-members using pre-programme behaviour and matched cohorts, recognising that matching cannot eliminate all selection bias.

Approach and solution. High-propensity, high-uplift customers receive relevant reminders rather than blanket discounts. Customers with high predicted value but low service satisfaction are routed to service recovery. Recommendations are tested in a holdout design. Dashboards report incremental margin, not only clicks and revenue.

Outcome. In this illustrative case, the incentive group produces more orders but lower incremental margin than the reminder group because many customers would have returned without a discount. The reminder group produces the strongest net contribution. The loyalty programme is retained, but rewards are redesigned around useful services and category benefits rather than universal price reductions.

Learning points. Propensity is not uplift; revenue is not contribution; association is not causation; and loyalty membership is not evidence of programme incrementality. The case also shows why customer strategy requires analytical integration and operational discipline.

13. Challenges, Ethics and Governance

Customer analytics creates risks involving privacy, security, surveillance, discrimination and manipulation. Key governance principles include purpose limitation, data minimisation, accuracy, retention controls, access control and transparent communication. Applicable privacy law varies by jurisdiction, but retailers should generally establish a lawful basis for processing, respect consent and opt-out requirements where relevant, and provide appropriate rights and notices.

Identity resolution is particularly sensitive. Combining devices, households and offline purchases can create false matches or reveal behaviour that customers did not expect to share. Sensitive attributes should not be inferred casually, and proxy variables can reproduce protected-group disadvantage. Models should be tested for disparate error rates, unequal treatment and harmful outcomes.

Retailers should maintain model documentation covering purpose, data sources, features, training period, validation, limitations, decision thresholds, human oversight and monitoring. Automated decisions that materially affect access, price, service or credit-like opportunities require especially careful review. Personalisation should preserve meaningful choice and avoid exploiting vulnerability.

14. Contemporary Developments and Best Practice

Retail analytics increasingly combines first-party transaction data with real-time digital behaviour, retail media exposure, computer vision and generative AI. These technologies can improve search, recommendations and service productivity, but they also increase the risk of opaque decisions, over-collection and synthetic or incorrect outputs. Generative systems should not receive unnecessary personal data, and human review remains important for consequential communications.

Best practice includes:

• Begin with a measurable decision and economic objective.
• Use contribution margin and incremental outcomes where possible.
• Define customer, order, basket and active-period rules explicitly.
• Separate descriptive, predictive and causal claims.
• Validate identity resolution and prevent data leakage.
• Use control groups for campaigns and loyalty evaluations.
• Monitor calibration, drift, subgroup performance and business impact.
• Offer relevant personalisation without excessive surveillance or pressure.
• Document assumptions, uncertainty and limitations.
• Treat analytics as a managed learning system, not a one-time project.

Chapter Summary

Customer and shopper analytics links behavioural data with decisions about acquisition, retention, loyalty, personalisation and merchandising. RFM provides a transparent first view of recency, frequency and monetary value, while segmentation and personas translate heterogeneous behaviour into actionable groups. CLV estimates future contribution under explicit assumptions and should be compared with acquisition and intervention costs.

Retention and churn require category-appropriate definitions, and survival methods can account for time and censoring. Propensity models estimate likelihood, whereas uplift methods focus on incremental treatment effect. Attribution rules organise touchpoints but do not establish causation; experiments and holdouts provide stronger evidence. Market-basket metrics such as support, confidence and lift reveal associations, but they require commercial and causal validation.

The central managerial principle is integration: identify the decision, prepare trustworthy data, model responsibly, act economically, measure incrementality and govern the use of personal information. Ethical customer analytics protects autonomy, fairness, security and transparency while creating value for both retailer and customer.

Key Terms

Customer analytics: Analysis of customer data to support relationship and commercial decisions.

Shopper analytics: Analysis of behaviour within shopping visits, missions or occasions.

RFM: Recency, frequency and monetary-value analysis.

Recency: Time elapsed since a customer’s last qualifying purchase.

Frequency: Number of qualifying purchases in a defined period.

Monetary value: Revenue or contribution attributed to a customer.

Segmentation: Division of a market or customer base into actionable groups.

Persona: Evidence-informed narrative representation of a target segment.

CLV: Estimated discounted future contribution from a customer relationship.

CAC: Cost of acquiring a new customer.

Retention: Continuation of customer activity over a defined period.

Churn: Customer lapse or loss under a specified operational definition.

Propensity: Estimated probability of a customer event.

Uplift: Incremental change caused by an intervention relative to a control.

Attribution: Assignment of conversion credit to marketing touchpoints.

Support: Proportion of transactions containing an itemset.

Confidence: Conditional frequency of one itemset given another.

Lift: Ratio comparing conditional occurrence with baseline occurrence.

Incrementality: Outcome caused by an intervention beyond what would otherwise occur.

Data leakage: Use of information unavailable at the time a prediction would be made.

Review Questions

Short Answer Questions

1. Distinguish a customer from a shopper and explain why the distinction matters.
2. What are the three components of RFM analysis?
3. Why is predicted CLV not the same as observed customer profitability?
4. Distinguish propensity from uplift.
5. Define support, confidence and lift in market-basket analysis.

Long Answer Questions

1. Explain how a retailer could construct, validate and activate an RFM-based segmentation strategy.
2. Discuss the assumptions and limitations of a simplified CLV model.
3. Compare rule-based attribution with randomised campaign evaluation.
4. Explain how loyalty-programme self-selection can bias performance evaluation.
5. Discuss the ethical and governance requirements for personalised retail analytics.

Analytical / Application Questions

1. A retailer has 2,000 baskets. Product A occurs in 400 baskets, product B in 500 baskets and both occur in 180 baskets. Calculate support for A and B, confidence for A → B and lift for A → B. Interpret the result.
2. Design a controlled win-back experiment for dormant customers. Specify the population, treatment groups, control group, primary outcome, economic metric and major threats to validity.
3. A customer produces £30 quarterly contribution margin, has a quarterly retention probability of 0.75 and faces a quarterly discount rate of 0.02. Using the simplified CLV formula, estimate gross CLV and explain how the decision would change if CAC were £40.

Multiple Choice Questions with Answers

Question 1

Which RFM component measures the time elapsed since a customer’s most recent purchase?

A. Recency

B. Frequency

C. Monetary value

D. Retention

Correct Answer: A. Recency

Question 2

In a standard RFM scoring system, which customer generally receives the highest recency score?

A. The customer who purchased longest ago

B. The customer who purchased most recently

C. The customer with the largest basket

D. The customer with the most returns

Correct Answer: B. The customer who purchased most recently

Question 3

What does customer lifetime value primarily estimate?

A. The number of products in the next basket

B. Future economic contribution from a customer relationship

C. The probability that an advertisement is clicked

D. The number of loyalty programme members

Correct Answer: B. Future economic contribution from a customer relationship

Question 4

A campaign treatment group converts at 8% and a control group at 6%. What is the absolute incremental lift?

A. 2 percentage points

B. 8 percentage points

C. 14 percentage points

D. 33.3 percentage points

Correct Answer: A. 2 percentage points

Question 5

Which statement best distinguishes propensity from uplift modelling?

A. Propensity estimates margin, while uplift estimates revenue

B. Propensity estimates likelihood, while uplift estimates differential treatment effect

C. Propensity is descriptive, while uplift is always causal

D. Propensity uses baskets, while uplift uses only demographics

Correct Answer: B. Propensity estimates likelihood, while uplift estimates differential treatment effect

Question 6

If 1,000 baskets exist, 200 contain coffee and 120 contain both coffee and milk, what is confidence for coffee → milk?

A. 12%

B. 30%

C. 60%

D. 80%

Correct Answer: C. 60%

Question 7

In market-basket analysis, lift greater than 1 generally indicates what?

A. The items have a positive association relative to baseline occurrence

B. The first item causes the second item to be purchased

C. The rule has maximum possible support

D. The products have identical prices

Correct Answer: A. The items have a positive association relative to baseline occurrence

Question 8

Why can last-touch attribution mislead a retailer?

A. It gives no credit to any touchpoint

B. It tends to over-credit the final touchpoint and under-credit earlier influences

C. It can only be used for offline sales

D. It automatically measures incremental impact

Correct Answer: B. It tends to over-credit the final touchpoint and under-credit earlier influences

Question 9

Which practice helps prevent data leakage in propensity modelling?

A. Using information created after the prediction decision

B. Including future purchase outcomes as predictors

C. Using only features available before the decision point

D. Removing the outcome variable from model evaluation

Correct Answer: C. Using only features available before the decision point

Question 10

In the simplified CLV model with r=0.75, m=£30 and d=0.02, what is approximate gross CLV?

A. £22.50

B. £75.00

C. £84.91

D. £225.00

Correct Answer: C. £84.91