Demand Data, Patterns and Segmentation

Introduction

Demand planning begins before a forecasting model is selected. It begins with a defensible understanding of the data: what was sold, to whom, through which channel, at what time, under which commercial conditions, and whether the recorded observation represents genuine market demand. A forecast generated from incomplete, duplicated, misclassified or promotion-distorted data may be statistically sophisticated but operationally misleading.

Demand data are also heterogeneous. Some products sell steadily; others sell intermittently. Some markets show trend or seasonality, while others are affected by economic cycles, competitor actions or structural breaks. A planner must therefore diagnose the data-generating process before choosing a forecasting method or setting inventory policies.

This chapter develops a practical framework for preparing and diagnosing internal and external demand data, identifying demand patterns and anomalies, correcting master-data problems, and segmenting products, customers and channels for differentiated planning.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Distinguish demand, sales, orders and shipments as planning variables.
2. Prepare internal and external data for forecasting while preserving business meaning.
3. Diagnose trend, seasonality, cycles, intermittency, outliers and structural breaks.
4. Calculate and interpret key data-quality and demand-variability measures.
5. Explain the role of product, customer, location and channel master data.
6. Apply ABC–XYZ, volume–variability and customer–product–channel segmentation.
7. Evaluate the managerial, technological and governance implications of segmentation.

1. Demand Data as a Business Measurement System

1.1 Demand is not always observed directly

Demand is the quantity customers would purchase under stated conditions. Sales are transactions actually recorded. Orders are requests placed by customers, and shipments are quantities dispatched. These measures may differ because of stockouts, credit holds, allocation rules, cancellations, minimum order quantities or delivery delays.

For example, if a retailer requests 1,000 units but receives only 700 because the supplier is out of stock, a shipment-based series reports 700. Treating 700 as unconstrained demand underestimates the market requirement. A useful demand record should therefore include both the observed quantity and the conditions under which it was observed.

A demand fact table commonly contains:

The planning grain must be explicit. “Monthly demand” could mean calendar-month sales by SKU, weekly orders by customer, or monthly shipments by distribution centre. Combining different grains without a controlled aggregation rule creates apparent volatility and double counting.

1.2 Internal and external data

Internal data arise from the organisation’s own systems: enterprise resource planning, point-of-sale, warehouse management, customer relationship management, pricing, promotion and inventory systems. External data include market size, competitor prices, macroeconomic indicators, weather, public holidays, demographic change, industry indexes and digital search or traffic measures.

Internal data are generally more accessible but may reflect the firm’s own supply constraints. External data may improve causal interpretation but often have different frequencies, revisions, definitions and publication lags. A planner should document the source, owner, update frequency, unit, time zone, revision policy and expected reliability of every variable.

2. Preparing Demand Data

2.1 Establish the data contract

Before extraction, define a data contract. It should specify:

• the planning purpose and forecast horizon;
• the observation grain and calendar;
• the demand measure and unit of measure;
• inclusion and exclusion rules;
• treatment of returns, cancellations, transfers and samples;
• the source system and responsible data owner;
• approved keys for products, customers, locations and channels; and
• rules for corrections, versioning and audit trails.

A data contract prevents a common failure: analysts produce technically correct calculations from data whose business definition has silently changed.

2.2 Time alignment and aggregation

Time stamps must be converted consistently. A transaction at 23:30 local time may belong to a different business day after conversion to UTC. Fiscal calendars may contain 4–4–5 weeks, leap weeks or 53-week years. External data may be monthly while transactions are daily. Aggregation should follow the decision being supported, not merely the format of the source system.

When aggregating quantities, addition is usually appropriate. Prices should generally be weighted by quantity when an average transaction price is required:

Here, p_i is transaction price, q_i is transaction quantity, and n is the number of transactions. The result has currency per unit. The formula assumes that all included transactions refer to comparable units and that quantity is an appropriate weighting basis.

2.3 Missing values and zero demand

A missing observation means “not recorded” or “unknown”; a zero means “measured demand was zero.” They are not interchangeable. A missing sales record during a system outage should not be converted automatically into zero. Conversely, a store reporting no sales for an item that was available and active may legitimately have zero demand.

For each missing value, determine whether it represents:

1. unavailable data;
2. a closed location or inactive SKU;
3. a genuine zero;
4. a stockout or supply restriction; or
5. a failed integration.

Imputation may be justified for analysis, but imputed values should be flagged and should not be confused with observed demand.

2.4 Duplicates, reversals and returns

Duplicate transaction IDs, repeated extracts and late-arriving records can inflate demand. Returns may be stored as positive quantities in a separate table or as negative quantities in the sales table. The organisation must choose a documented policy: gross demand, net demand, or separate sales and returns series. The choice depends on the decision. Production planning may require gross outbound demand, whereas financial analysis may require net revenue.

2.5 Unit-of-measure and conversion errors

A product may be ordered in cases, stored in cartons and sold in individual units. Conversion factors must be governed in master data. If one case contains 24 units, then:

This conversion is valid only when the pack configuration is stable for the relevant period. Historical pack changes require effective dates; otherwise, a genuine change in physical quantity may be mistaken for demand growth.

3. Demand Pattern Diagnosis

A time series is a sequence of observations indexed by time, commonly written as y_t, where t identifies the period. Diagnosis should combine plots, summary statistics, business knowledge and event calendars. No single statistic can reliably identify every pattern.

3.1 Level and variation

The mean of n observations is:

The sample standard deviation is:

The coefficient of variation is:

The mean and standard deviation have the same units as demand, while CV is unit-free. The sample standard deviation is appropriate when the observations are treated as a sample from a broader process. The coefficient of variation becomes unstable when the mean is close to zero and should not be used mechanically for highly intermittent series.

3.2 Trend

Trend is a persistent increase or decrease in the underlying level. It may arise from market growth, product adoption, distribution expansion, price changes or decline in product relevance. A simple descriptive linear trend is:

Here, a is the intercept, b is the average change per time period, t is time, and ε_t is the unexplained component. This model is descriptive, not proof that the future will continue linearly. It is most useful over a stable interval and can be distorted by outliers or breaks.

3.3 Seasonality

Seasonality is a systematic pattern that repeats at a known calendar frequency, such as weekly, monthly or annual demand. Holiday effects, school terms and weather seasons are typical causes. Seasonal demand can be represented conceptually as:

for an additive structure, where T_t is trend, S_t is seasonal effect and R_t is the remainder. An additive representation is appropriate when seasonal fluctuations are approximately constant in absolute units. A multiplicative representation, y_t = T_t × S_t × R_t, is more suitable when seasonal amplitude grows with the level and all components are positive.

Seasonality must be distinguished from promotions. A promotion repeated every December may appear seasonal, but its mechanism is commercial rather than purely calendar-driven. The distinction matters when planning future events.

3.4 Cycles

Cycles are recurrent but not necessarily fixed-period movements, often associated with economic conditions, construction activity, credit availability or industry investment. Unlike seasonality, cycle length and amplitude may vary. A two-year history is rarely sufficient to identify a reliable business cycle. External indicators and industry knowledge are essential.

3.5 Intermittency

Intermittent demand contains many zero periods separated by non-zero events. Spare parts, specialist components and low-volume medical products often exhibit this pattern. Two useful measures are the average interval between non-zero demands and the variability of non-zero demand sizes.

Let n be the number of periods and N_+ the number of periods with positive demand. The proportion of zero periods is:

A high P₀ indicates sparsity, but it does not alone establish the best forecasting method. A series with occasional large orders may require different inventory treatment from one with small, regular orders.

3.6 Outliers

An outlier is an observation unusually distant from the prevailing pattern. It may represent a recording error, a one-time project, a promotion, a disaster, a competitor stockout or genuine demand change. Removing it without investigation destroys information.

A common robust screening rule uses the interquartile range. If Q₁ and Q₃ are the first and third quartiles, then IQR = Q₃ - Q₁. Observations below Q₁ - 1.5IQR or above Q₃ + 1.5IQR are candidates for review, not automatic deletions. The rule assumes that a distributional screening tool is being used for triage; it does not prove that a point is erroneous.

3.7 Structural breaks

A structural break is a persistent change in the data-generating process. Examples include a new competitor, a product redesign, a merger, a regulatory change, a permanent price repositioning or a supply network redesign. A break differs from an outlier because its effect persists.

Useful diagnostic evidence includes a sudden level shift, a change in variance, a change in seasonal amplitude, and a corresponding business event. Forecast models should generally be trained using a regime relevant to the future, while older history may remain valuable for understanding the cause.

Figure: Schematic examples of trend, seasonality and an outlier. The patterns are illustrative rather than observations from one product.

4. Data Quality and Master Data

Data quality is multidimensional. Accuracy asks whether a value represents reality; completeness asks whether required values are present; consistency asks whether definitions agree across systems; timeliness asks whether data arrive when needed; uniqueness asks whether records are duplicated; and validity asks whether values conform to permitted formats and ranges.

A practical data-quality dashboard may report:

This metric is meaningful only when “required field” is clearly defined. A high completeness rate does not imply accuracy.

Master data provide stable reference identities. Important objects include SKU, product hierarchy, unit conversion, pack configuration, customer, ship-to location, supplier, warehouse, channel, currency and calendar. Common failures include duplicate SKUs, reused codes, inactive items still forecasted, missing hierarchy assignments, inconsistent customer names and ungoverned product substitutions.

A sound governance process includes a named data owner, approval workflow, effective dates, validation rules, duplicate detection, change logs and periodic reconciliation. Product substitutions deserve special attention: replacing an old SKU with a new one may create an artificial decline and artificial launch unless the relationship is represented explicitly.

5. Segmentation for Differentiated Planning

Segmentation recognises that one forecasting and inventory policy rarely suits every item. Segments should lead to differentiated decisions, such as review frequency, forecast method, safety-stock logic, collaboration intensity and data-maintenance effort. Segmentation is not an end in itself.

5.1 ABC analysis: value concentration

ABC analysis ranks items by annual consumption value:

where D_i is annual demand for item i in physical units and c_i is unit cost in currency per unit. The product is currency per year. Items are sorted in descending ACV_i, and cumulative shares are calculated. Typical boundaries might classify the first 70–80% of cumulative value as A, the next 15–25% as B, and the remainder as C. Boundaries are policy choices, not universal laws.

ABC highlights financial exposure, not necessarily forecastability. A low-value item can have critical service consequences, while a high-value item may be stable and easy to plan.

5.2 XYZ analysis: demand predictability

XYZ commonly classifies items using demand variability, often CV. Lower variability is usually labelled X, moderate variability Y and high variability Z. Thresholds must be calibrated to the business and demand grain. For intermittent items, non-zero demand variability, zero-demand proportion and demand intervals should be considered together.

Combining ABC and XYZ produces a useful matrix:

5.3 Volume–variability segmentation

Volume–variability segmentation combines demand magnitude with volatility. Volume may be measured by annual units, revenue, margin or order lines; variability may be measured by CV, forecast error, intermittency or peak-to-average ratio. This avoids confusing value with physical workload. A low-cost item with millions of units may dominate warehouse capacity and transport demand.

5.4 Customer–product–channel segmentation

Demand behaviour often changes across the intersection of customer, product and channel. A product may be stable in replenishment retail, sporadic in project sales and highly promotion-sensitive online. Customer–product–channel segmentation can therefore reveal hidden aggregation problems.

Useful dimensions include:

• customer service requirement and order frequency;
• product lifecycle and substitution risk;
• channel lead time, promotion intensity and return rate;
• margin, strategic importance and contractual commitments; and
• forecastability and decision impact.

The segment should be fine enough to capture materially different behaviour but not so fine that history becomes too sparse. A hierarchical approach is often preferable: forecast at an appropriate aggregate level, then allocate to lower levels using reliable proportions.

6. Worked Example: ABC–XYZ Diagnosis

Consider four products with annual demand, unit cost and monthly demand observations summarised below.

Total annual consumption value is 365,000. In descending value order, P1 contributes 54.8%, P2 raises the cumulative share to 87.7%, P3 to 98.6% and P4 to 100%. With a policy that classifies approximately the first 80% as A, the next group as B and the remainder as C, P1 is A, P2 is also A because the cumulative boundary is crossed by P2, P3 is B and P4 is C. If the organisation instead requires an exact number of A items, that rule must be stated separately; ABC classification is policy-dependent.

Using illustrative thresholds of X for CV ≤ 0.20, Y for 0.20 < CV ≤ 0.50, and Z for CV > 0.50, P1 is AX, P2 is AY, P3 is AX and P4 is CZ. P1 deserves strong parameter governance because it is financially significant and predictable. P2 requires attention to variability and perhaps commercial drivers. P4 is financially small but operationally intermittent; it may be suitable for a simplified or order-triggered policy rather than frequent manual forecasting.

7. Illustrative Case: Northstar Industrial Components

Background

Northstar, a fictional industrial-equipment distributor, experienced recurring stockouts on replacement bearings despite an apparently adequate forecasting system. The planning team forecast monthly shipments at SKU level and applied one inventory policy to all items.

Problem and analysis

An audit found three issues. First, 18% of shipment records lacked a valid customer-channel key. Second, two pack sizes shared an old product code, creating a false historical demand increase after the pack conversion. Third, several large project orders were treated as ordinary recurring demand. The team plotted order, shipment and inventory-availability series separately. Shipment demand showed low variability, but order demand contained stockout-censored periods and project spikes.

The team then segmented items using annual consumption value, non-zero demand variability, customer type and channel. High-value replacement items were separated from project business. Promotional and project orders were tagged using contract and sales records rather than deleted.

Approach and solution

Northstar established a master-data owner, introduced effective-dated pack conversions, and added an availability flag. It used a two-level process: stable replenishment demand was forecast statistically, while identified projects were entered as known events. Intermittent low-value items were reviewed using order-triggered rules and supplier lead-time information. High-value, unpredictable items received collaborative review with sales and key customers.

Outcome

The main improvement was not a more complex algorithm. It was a more accurate definition of demand and a differentiated process. Forecast bias declined for the affected product family, while planners also gained an explanation for residual error. The company avoided deleting unusual observations and instead classified them according to cause.

Learning points

1. Shipment history can understate demand when availability is constrained.
2. Master-data changes can create false trends and breaks.
3. Outliers require causal investigation, not automatic removal.
4. Segmentation should change decisions, ownership or review effort.
5. A forecast process must distinguish recurring demand from known one-time events.

8. Technology, AI and Governance

Modern planning platforms can automate profiling, anomaly detection, entity matching and hierarchy management. Machine-learning systems may identify nonlinear relationships with price, weather or digital traffic. However, automation does not resolve ambiguous business definitions. A model can detect that a value is unusual without knowing whether it represents a promotion, a supply failure or a new market regime.

AI-supported data preparation should therefore include human review queues, reason codes, confidence scores, model versioning and reproducible transformations. Historical corrections must be traceable. Access controls should protect commercially sensitive customer and price data, and personal information should be minimised and handled according to applicable privacy requirements.

A governance principle is “do not silently rewrite history.” Corrected records should preserve the original value, correction reason, approver and effective date. This supports auditability and prevents future analysts from mistaking a data correction for a demand event.

9. Challenges and Limitations

Demand diagnosis is subject to several limitations. Short histories cannot reliably distinguish seasonality from random variation. Aggregation can hide local patterns, while disaggregation can create sparse and noisy series. CV is unstable near a zero mean. ABC thresholds may encourage attention to financial value while neglecting criticality, margin or service obligations. External variables may be correlated with demand without causing it. Structural-break detection is difficult when the business event is undocumented.

Segmentation can also become excessive. Hundreds of micro-segments increase maintenance effort and may leave too little data for reliable analysis. A segment should be retained only when it supports a different decision, service promise, forecast method or governance action.

10. Best-Practice Workflow

A disciplined workflow is:

1. Define the planning decision, horizon and data grain.
2. Document demand, sales, order and shipment definitions.
3. Profile completeness, validity, uniqueness, consistency and timeliness.
4. Reconcile quantities, units, returns, cancellations and inventory availability.
5. Validate product, customer, location and channel master data.
6. Plot the series and inspect events, promotions and operational constraints.
7. Diagnose level, trend, seasonality, cycles, intermittency, outliers and breaks.
8. Calculate value and variability measures with stated thresholds.
9. Build a small number of decision-oriented segments.
10. Assign actions, owners, review frequencies and exception rules.
11. Monitor whether segments and data definitions remain appropriate.

Chapter Summary

Reliable demand planning depends on understanding how demand is observed and how operational constraints distort observations. Internal and external data must be aligned by time, grain, unit and business definition. Missing values, zeros, duplicates, returns, pack conversions and master-data errors require explicit treatment.

Demand series may contain level, trend, seasonality, cycles, intermittency, outliers and structural breaks. Diagnosis combines statistical measures such as mean, standard deviation, coefficient of variation and zero-demand proportion with plots and business-event evidence. Outliers should be investigated rather than automatically removed, and persistent changes should be treated differently from isolated anomalies.

ABC analysis identifies concentration of annual consumption value, while XYZ and volume–variability analysis address predictability and operational workload. Customer–product–channel segmentation exposes differences hidden by broad aggregation. Effective segmentation is decision-oriented: it changes forecast methods, review intensity, service policies or governance. Technology can accelerate profiling and anomaly detection, but human definitions, accountability, privacy and auditability remain essential.

Key Terms

Demand: Quantity customers would purchase under specified conditions.

Sales: Recorded commercial transactions.

Shipment: Quantity physically dispatched to a customer or location.

Planning grain: The product, customer, location and time level at which data are measured.

Data contract: Documented agreement defining data meaning, scope, quality and ownership.

Intermittent demand: Demand containing many zero periods separated by non-zero events.

Trend: Persistent upward or downward movement in demand level.

Seasonality: Repeating variation linked to a known calendar frequency.

Cycle: Recurrent movement whose timing or amplitude is not fixed.

Outlier: Observation unusually distant from the prevailing pattern.

Structural break: Persistent change in the process generating observations.

Coefficient of variation: Standard deviation divided by a positive mean.

Master data: Governed reference data for entities such as products, customers and locations.

Unit of measure: Physical or commercial basis in which quantity is recorded.

ABC analysis: Classification based on cumulative annual consumption value.

XYZ analysis: Classification based primarily on demand variability or predictability.

Volume–variability segmentation: Classification combining demand magnitude and volatility.

Customer–product–channel segmentation: Analysis of demand behaviour at the intersection of commercial dimensions.

Data lineage: Traceable record of data origin, transformations and corrections.

Demand censoring: Situation in which observed sales are limited by availability or another constraint.

Review Questions

Short Answer Questions

1. Why are sales and unconstrained demand not always equivalent?
2. Distinguish a missing observation from a zero-demand observation.
3. What is the difference between seasonality and a business cycle?
4. Why should an outlier not be deleted automatically?
5. What is the purpose of product master data in demand planning?

Long Answer Questions

1. Explain a complete workflow for preparing internal and external demand data for forecasting.
2. Discuss trend, seasonality, intermittency, outliers and structural breaks, including how each can affect planning decisions.
3. Explain ABC–XYZ analysis and evaluate its benefits and limitations.
4. Discuss the role of data governance in maintaining reliable demand histories.
5. Explain why customer–product–channel segmentation may be more useful than SKU-only analysis.

Analytical / Application Questions

1. A product has a high annual consumption value, a coefficient of variation of 0.65 and several stockout months. Diagnose the data and recommend an appropriate planning process.
2. A new pack size causes recorded unit demand to double while case demand remains stable. Identify the likely data problem and specify the corrective steps.
3. Design a segmentation scheme for a distributor containing fast-moving retail items, seasonal promotional products and intermittent spare parts. State the measures, thresholds and differentiated actions you would use.

Multiple Choice Questions with Answers

Question 1

Which observation most directly indicates that recorded shipments may understate unconstrained demand?

A. The product has a stable unit cost

B. Orders exceeded shipments during a stockout

C. The product is assigned to an A class

D. Demand was aggregated monthly

Correct Answer: B. Orders exceeded shipments during a stockout

Question 2

If the mean demand is 100 units and the standard deviation is 20 units, what is the coefficient of variation?

A. 0.02

B. 0.20

C. 5.00

D. 80.00

Correct Answer: B. 0.20

Question 3

Which statement correctly distinguishes a structural break from an outlier?

A. A structural break is always caused by a data-entry error

B. An outlier is necessarily a permanent change

C. A structural break represents a persistent change in the data-generating process

D. There is no practical distinction between them

Correct Answer: C. A structural break represents a persistent change in the data-generating process

Question 4

What does ABC analysis primarily measure?

A. Customer satisfaction

B. Cumulative annual consumption value

C. The number of zero-demand periods

D. Forecast model complexity

Correct Answer: B. Cumulative annual consumption value

Question 5

Why should a missing demand value not automatically be replaced with zero?

A. Zero is never a valid demand value

B. Missingness may represent an outage or unavailable record rather than genuine zero demand

C. Forecasting systems cannot process zeros

D. Zeros always create structural breaks

Correct Answer: B. Missingness may represent an outage or unavailable record rather than genuine zero demand

Question 6

A series has 80 observations, of which 20 have positive demand. What is the proportion of zero-demand periods?

A. 0.20

B. 0.25

C. 0.75

D. 4.00

Correct Answer: C. 0.75

Question 7

Which master-data problem can create an artificial demand change after a packaging conversion?

A. A documented holiday calendar

B. An effective-dated unit conversion

C. One product code being used inconsistently before and after the conversion

D. A correctly recorded customer location

Correct Answer: C. One product code being used inconsistently before and after the conversion

Question 8

In an ABC–XYZ matrix, an AZ item is generally characterised as:

A. Low value and highly predictable

B. High value and highly unpredictable

C. Medium value and highly predictable

D. High value and zero demand in every period

Correct Answer: B. High value and highly unpredictable

Question 9

Which measure is most appropriate for screening candidate outliers using the stated robust rule?

A. Interquartile range

B. Annual unit cost only

C. Customer count only

D. Pack-size conversion factor only

Correct Answer: A. Interquartile range

Question 10

Why can customer–product–channel segmentation improve demand planning?

A. It guarantees perfect forecasts

B. It removes the need for master data

C. It can reveal different demand behaviour hidden by broad aggregation

D. It makes every segment equally large

Correct Answer: C. It can reveal different demand behaviour hidden by broad aggregation