Demand Data, Patterns, and Preparation

Introduction

Demand forecasting begins before a forecasting model is selected. It begins with deciding what “demand” means, identifying how it has been recorded, and determining whether the observed data represent customers’ underlying requirements or merely the sales that the firm was able to fulfill. Poorly prepared data can cause a sophisticated model to produce systematically poor forecasts. Conversely, careful diagnosis of demand patterns often allows a relatively simple model to perform well.

In supply chains, demand data may be recorded at the level of a stock-keeping unit (SKU), customer, location, channel, order line, day, week, or month. These observations are influenced by customer preferences, prices, promotions, availability, holidays, weather, competitor actions, product life-cycle effects, and measurement practices. The analyst’s task is not simply to remove unusual values. It is to distinguish meaningful economic signals from recording errors and to preserve information that improves decisions.

This chapter develops a practical framework for recognizing demand structure and preparing data for forecasting. It addresses level, trend, seasonality, cycles, promotions, outliers, structural breaks, intermittency, and demand censoring. It also explains cleaning, aggregation, transformation, calendar adjustment, exploratory visualization, governance, and the implications of these choices for inventory and capacity decisions.

Learning objectives

After studying this chapter, you should be able to:

1. Define demand appropriately for a supply-chain forecasting problem.
2. Identify level, trend, seasonality, cycles, promotional effects, outliers, structural breaks, intermittency, and censoring.
3. Distinguish genuine business signals from data-quality problems.
4. Aggregate, clean, transform, and calendar-adjust demand data appropriately.
5. Use exploratory visualizations and descriptive statistics to diagnose demand.
6. Explain how data preparation affects model selection, forecast accuracy, and managerial decisions.
7. Recognize governance, ethical, and operational risks in demand-data management.

1. What is demand data?

A demand series is a sequence of observations indexed by time. Let y_t denote demand observed in period t, where the unit might be units sold, cases ordered, kilograms shipped, or revenue. A useful conceptual decomposition is:

Here, L_t is the underlying level, T_t is a trend, S_t is a seasonal component, C_t is a longer cycle, P_t represents promotional or intervention effects, I_t represents unusual interventions or structural effects, and ε_t is residual variation. This is a diagnostic framework rather than an assertion that every series can be uniquely separated into these components. Components may interact, and a multiplicative representation may be more appropriate when variability increases with the level:

The additive form assumes component effects are measured in the same units as demand. The multiplicative form assumes effects act proportionally and requires positive observations, unless a suitable transformation is used.

Demand, orders, shipments, and sales

These terms are not interchangeable. Customer demand is the quantity customers would purchase under stated conditions. Orders are quantities requested. Shipments are quantities dispatched. Sales may be transactions recorded at a point of sale. If inventory is unavailable, sales can understate demand. If a distributor places a forward-buy order during a promotion, orders can overstate final consumption.

The data specification should therefore record:

• the demand measure and unit of measure;
• product, location, customer, and channel identifiers;
• timestamp and time zone;
• price, discount, and promotion indicators;
• inventory availability, stockouts, and backorders;
• returns, cancellations, substitutions, and transfers;
• data source, revision history, and extraction date.

A forecasting target must also specify the horizon and decision context. A manufacturer may forecast weekly replenishment demand, while a financial planning team forecasts monthly revenue. These are related but different analytical problems.

2. Recognizing demand patterns

Level

Level is the typical magnitude around which observations fluctuate. It is not necessarily a constant mean. For a stable series, a sample mean is often informative:

where n is the number of observations and y_t is demand in period t. The mean is sensitive to outliers and is meaningful as a description of a typical period only when the time window is reasonably homogeneous.

The median may be preferable for skewed or intermittent demand. A rolling mean can reveal changing level, but the window length should reflect the operational cycle and should not conceal important short-term changes.

Trend

Trend is a persistent increase or decrease in the central tendency of demand. It may arise from market growth, declining product relevance, distribution expansion, demographic change, or substitution. A linear descriptive trend can be written as:

where a is the intercept, b is the average change per time period, and ε_t is unexplained variation. This equation is descriptive; a linear trend should not be extrapolated indefinitely without business justification. Growth often saturates, and product life cycles may be nonlinear.

Trend can be confused with a structural break. A gradual change over many periods suggests trend, whereas an abrupt and persistent change suggests a break caused by a new price, channel, policy, competitor, or measurement system.

Seasonality

Seasonality is a systematic pattern that repeats at a known and usually fixed calendar frequency. Examples include weekly shopping patterns, monthly billing cycles, summer beverages, and year-end retail demand. If a series has period m, a seasonal index for period position j can be estimated using the ratio-to-average method:

Under a multiplicative interpretation, seasonal indices are positive and their average across a complete cycle is approximately 1. Under an additive interpretation, seasonal effects are measured in demand units and sum approximately to zero over a complete cycle. Seasonal indices should be estimated using a sufficiently long, representative history and adjusted for promotions, stockouts, and changing assortment where possible.

Cycles

Cycles are recurrent but not necessarily fixed calendar patterns. Economic expansions, construction activity, commodity markets, and replacement cycles may produce fluctuations lasting several years or varying in duration. Cycles are difficult to distinguish from trend with short data histories. Analysts should use economic indicators and business knowledge rather than treating every long movement as a deterministic cycle.

Promotions and interventions

Promotions include temporary price reductions, displays, coupons, advertising, bundles, loyalty offers, and channel incentives. A promotion can create three distinct effects:

1. Lift: additional demand during the event.
2. Pull-forward: demand shifted from future periods into the promotion.
3. Post-promotion dip: temporarily reduced demand after customers have stocked up.

Promotion flags should be linked to dates, products, locations, discount depth, media exposure, and inventory availability. If promotional observations are mixed indiscriminately with regular demand, the model may forecast promotional lift when no promotion is planned or may underestimate baseline demand.

Outliers

An outlier is an observation unusually distant from the pattern expected under a specified model or business process. It may be caused by a data-entry error, duplicate transaction, one-time customer order, extreme weather event, strike, viral publicity, or a genuine change in demand.

Outlier detection is not equivalent to outlier deletion. A practical procedure is:

1. Detect unusual values using plots, robust statistics, or model residuals.
2. Investigate the operational cause.
3. Correct confirmed errors, retain meaningful events as explanatory variables, and treat unresolved values transparently.
4. Record every adjustment in an audit log.

A commonly used descriptive measure is the standardized score:

where s is the sample standard deviation. This measure is useful only under reasonably stable conditions; trend, seasonality, and skewness can make ordinary scores misleading. Robust alternatives use the median and median absolute deviation.

Structural breaks

A structural break is a persistent change in the data-generating process. Examples include a permanent price change, new distribution agreement, product reformulation, ERP implementation, regulatory change, pandemic effect, or competitor exit. A break can affect level, trend, variance, seasonality, or relationships with explanatory variables.

The analyst should mark the break date, investigate its cause, compare pre- and post-break behavior, and decide whether older observations remain relevant. Blindly averaging across a break can produce a forecast that represents neither regime.

Intermittent demand

Intermittent demand contains many zero periods and occasional positive observations. It is common for spare parts, maintenance items, slow-moving SKUs, and highly customized products. Two characteristics are useful:

• the interval between nonzero demands;
• the size of a nonzero demand.

Let p be the proportion of periods with nonzero demand and q be the mean size conditional on a nonzero event. Then the unconditional mean is:

This identity assumes that the estimates refer to the same stable population and time interval. It shows why forecasting only the positive quantities can overstate expected demand. Intermittent series often require specialized approaches, such as Croston-type methods, separated occurrence-and-size models, or aggregation to a longer planning interval. However, method choice should be evaluated against inventory costs, not accuracy metrics alone.

Demand censoring

Censoring occurs when the observed value is bounded by a process rather than equal to true demand. If inventory available is A_t and recorded sales are y_t, a simple stockout relationship is:

where d_t is latent customer demand. When d_t > A_t, observed sales equal availability and do not reveal the unmet quantity. Lost sales, backorders, substitutions, and customer abandonment may need separate estimation. Treating censored sales as ordinary demand biases forecasts downward, especially for high-demand items.

3. Preparing demand data

Establish a data contract

Before extraction, define the grain of the dataset: one row might represent one SKU-location-day or one customer-order line. Specify primary keys, units, time zone, calendars, permissible missingness, and revision rules. A data contract prevents accidental mixing of incompatible records.

Cleaning and reconciliation

Cleaning should be rule-based and reproducible. Typical checks include:

• duplicate transaction identifiers;
• negative quantities and whether they represent returns;
• impossible dates, future dates, and inconsistent time zones;
• product and location codes that changed over time;
• unit-of-measure conversions;
• cancellations, transfers, and intercompany movements;
• gaps caused by system outages;
• totals that do not reconcile with finance or warehouse systems.

A missing record is not automatically zero demand. Zero means the system observed no demand under a defined process; missing means the observation is unknown. The distinction is critical for intermittent products and stockout analysis.

Aggregation and temporal alignment

Transactions should be aggregated to the frequency at which decisions are made. For additive quantities, aggregation is summation:

where Y_w is demand in week w and the sum includes all transactions assigned to that week. Prices should generally be averaged using an economically appropriate weight, while inventory positions may require end-of-period values. Do not average quantities that should be summed.

Aggregation can reduce noise and intermittency, but it can also hide stockouts, promotions, and within-day patterns. The selected frequency should match replenishment lead time, review period, production schedule, and data reliability.

Transformations

A logarithmic transformation can stabilize variance when demand variability grows approximately in proportion to its level:

It requires positive observations. For zeros, analysts may use a carefully justified alternative such as ln(1 + y_t), but interpretation changes and back-transformation can be biased. Transformations should be validated by examining residual behavior and decision performance, not adopted automatically.

Calendar adjustment

Calendar effects arise from different numbers of selling days, weekdays, public holidays, month lengths, and movable holidays. A monthly total may be higher simply because the month contains more trading days. If Y_m is monthly demand and N_m is the number of effective selling days, a basic daily-rate measure is:

This adjustment assumes selling days are comparable and that demand is approximately proportional to effective days. It is unsuitable when weekends, holidays, or store openings have materially different demand rates; in those cases, use day-type indicators or a daily model.

Calendar adjustments must not remove genuine seasonal demand. The purpose is to separate mechanical exposure differences from customer behavior.

4. Exploratory analysis and visualization

Exploratory analysis should proceed from broad structure to detailed causes. A useful sequence is:

1. Plot the full time series at the decision frequency.
2. Plot seasonal subseries, such as all Januaries or all Mondays.
3. Examine rolling means and rolling variability.
4. Display zeros, stockouts, promotions, and known events on the same timeline.
5. Compare demand across products, locations, channels, and customer segments.
6. Inspect distributions of positive demand and interarrival intervals.
7. Use a scatterplot or correlation analysis only when a causal interpretation is not being assumed automatically.

The rolling mean over a window of length k is:

It is a smoothing device, not a forecast. At the beginning of a series, fewer than k observations may be available; analysts should state how edge values are handled.

For forecast evaluation later, an often-used scale-free measure is mean absolute percentage error:

where ŷ_t is the forecast. MAPE is undefined when y_t = 0 and unstable near zero, so it is inappropriate as the sole measure for intermittent demand. Alternatives include weighted absolute percentage error, mean absolute error, or scaled errors, selected according to the decision problem.

A time-series plot should be accompanied by business annotations. A spike on the day of a national television advertisement has a different interpretation from a spike caused by duplicate uploads. Visualizations are most valuable when operational metadata are integrated rather than examined separately.

5. An integrated preparation workflow

A robust workflow is iterative:

Define. Specify target, unit, horizon, grain, population, and decision use.

Profile. Measure coverage, missingness, duplicates, zeros, negative values, revisions, and reconciliation differences.

Diagnose. Visualize patterns and annotate promotions, holidays, stockouts, launches, and system changes.

Classify. Label observations as regular, promotional, censored, erroneous, exceptional, or structurally changed where evidence supports the label.

Prepare. Aggregate, transform, calendar-adjust, impute only where justified, and create explanatory variables.

Validate. Compare prepared totals with source systems and review a sample of adjustments with domain experts.

Document. Preserve raw data, transformed data, code, assumptions, version history, and approval records.

Monitor. Reassess the data pipeline as products, systems, channels, and customer behavior change.

This workflow prevents the common error of treating data preparation as a one-time technical task. It is a controlled analytical process connected to business governance.

Illustrative case: preparing demand data for a spare-parts network

Background

A fictional industrial-equipment manufacturer forecasts demand for a pump seal stocked at 18 regional service centers. The raw system contains daily shipments for 30 months. The planning team uses the forecast to set monthly purchase quantities and service-level targets.

Problem

The initial forecast is persistently low. Investigation shows that the series contains 61% zero days, several apparent spikes, two months affected by a warehouse-management-system migration, and repeated stockouts at three service centers. Some shipments are emergency transfers rather than external customer demand.

Analysis

The team first defines the target as external service-center issues to customers, excluding internal transfers. It creates a SKU-location-day table and distinguishes zero demand from missing system days. Stockout dates are joined to inventory records. The apparent spikes are reviewed: one is a legitimate fleet-maintenance campaign, while another is a duplicated upload and is removed from the analytical copy with an audit note.

The team then aggregates daily observations to weeks because replenishment decisions are weekly. It calculates the proportion of nonzero weeks and the mean positive demand. For one center, 18 of 52 weeks have positive demand, with an average positive quantity of 4.0 seals. The estimated mean weekly demand is therefore:

This estimate is descriptive and assumes the observed weeks are representative and not materially censored. Stockout weeks are flagged rather than treated as ordinary zero demand. The maintenance campaign is retained as an event variable because it may recur through the installed-base schedule.

Approach and outcome

The planning team evaluates an intermittent-demand method against a simple weekly mean and an occurrence-and-size approach. Evaluation uses rolling time splits and service-level consequences, not MAPE alone. The resulting process produces a higher forecast for centers with repeated stockouts, a lower baseline after excluding internal transfers, and a separate uplift during scheduled maintenance campaigns.

The outcome is not merely a more accurate numerical forecast. Inventory reviews become explainable: planners can distinguish low underlying demand from constrained availability, and procurement can see when a forecast increase is caused by a known campaign rather than permanent growth.

Learning points

1. The target must represent the decision, not merely the easiest available field.
2. Intermittency and censoring require separate diagnosis.
3. Aggregation can improve stability but must preserve event information.
4. Outlier treatment requires operational investigation.
5. Data preparation should be evaluated through supply-chain outcomes as well as statistical metrics.

Managerial, governance, and technology implications

Data preparation affects working capital, service levels, production schedules, supplier commitments, and customer promises. Managers should require traceability for overrides and should avoid rewarding teams for artificially suppressing demand signals before a performance review.

Governance controls should address access rights, personally identifiable customer information, retention, model documentation, and segregation between raw records and adjusted datasets. Promotional data may reveal commercially sensitive pricing arrangements. Ethical practice requires that forecasts not systematically deprioritize smaller customers or regions merely because their historical data are sparse; apparent low demand may reflect poor availability or unequal service.

Modern platforms can automate anomaly alerts, calendar joins, product hierarchies, and data-quality tests. Machine-learning systems can detect nonlinear patterns, but automation does not establish causality. An algorithm may label a recurring stockout as low demand unless inventory availability is supplied as a feature. Generative AI can assist with documentation and code review, but all transformations require reproducible testing, human validation, and protection of confidential data.

Contemporary practice increasingly combines transactional data with point-of-sale information, inventory availability, web traffic, weather, mobility, and event calendars. Such enrichment is valuable only when the variable is available at forecast time, measured consistently, and linked to a plausible mechanism. Data abundance is not the same as information quality.

Challenges and limitations

Demand components are not uniquely identifiable from a short historical series. Trend and cycle can be confounded; promotions may coincide with holidays; product launches may create both structural breaks and seasonality; and stockouts conceal the very demand needed for estimation. Aggregation may reduce noise but can create temporal aggregation bias and conceal lead-time risk. Transformations can improve statistical behavior while complicating interpretation. Imputation can create false certainty, especially when missingness is related to demand intensity.

Consequently, preparation should be judged by assumptions, transparency, robustness, and decision usefulness. Analysts should perform sensitivity analyses—for example, comparing forecasts with and without uncertain outliers or censored periods—and communicate uncertainty rather than presenting adjusted data as unquestionable truth.

Best-practice checklist

• Define demand and exclude transactions that do not represent the decision target.
• Preserve raw data and maintain an adjustment log.
• Treat missing, zero, negative, and censored observations differently.
• Annotate promotions, holidays, stockouts, launches, and system changes.
• Choose aggregation based on decisions, lead times, and data reliability.
• Use transformations only when their assumptions and interpretation are clear.
• Combine visual evidence with operational investigation.
• Evaluate preparation using both statistical and supply-chain performance.
• Reassess patterns after structural changes.
• Document ownership, access, validation, and model-ready data versions.

Chapter Summary

Demand forecasting depends on a clear definition of the target and disciplined preparation of its historical evidence. Demand data may represent customer requirements, orders, shipments, or sales, and these measures diverge when promotions, inventory constraints, returns, transfers, or recording practices intervene. Level describes typical magnitude; trend describes persistent movement; seasonality repeats at known calendar frequencies; cycles are recurrent but less regular; promotions create lift and possible pull-forward; outliers may be errors or meaningful events; structural breaks change the underlying process; intermittent demand contains many zeros; and censoring occurs when observed sales are limited by availability.

Effective preparation establishes a data contract, reconciles sources, distinguishes missingness from zero demand, aggregates at a decision-relevant frequency, applies justified transformations and calendar adjustments, and integrates operational annotations. Exploratory plots and descriptive statistics are essential for discovering structure, but they do not replace domain investigation. Governance, auditability, ethical treatment, and human oversight are necessary because data adjustments influence inventory, capacity, supplier, and customer decisions. The central principle is simple: prepare data to represent the business process that the forecast will support, not merely the records that are easiest to retrieve.

Key Terms

Demand: Customer requirements for a product or service under specified conditions.

Level: The typical magnitude around which demand fluctuates.

Trend: Persistent upward or downward movement in demand.

Seasonality: A repeating pattern associated with a known calendar frequency.

Cycle: Recurrent movement whose timing or duration is not fixed.

Promotion: A temporary commercial intervention intended to influence demand.

Pull-forward: Demand shifted from a future period into an earlier promotional period.

Outlier: An unusually distant observation relative to a specified pattern or model.

Structural break: A persistent change in the process generating observations.

Intermittent demand: Demand with many zero periods and occasional nonzero observations.

Censoring: Observation constrained by a process such as limited inventory.

Stockout: A condition in which requested demand cannot be supplied from available inventory.

Aggregation: Combining observations into a larger time, product, or geographic unit.

Calendar adjustment: Modification that accounts for exposure differences such as selling days or holidays.

Transformation: A mathematical re-expression of data, such as a logarithm.

Data contract: A documented specification of data structure, meaning, quality, and ownership.

Demand lift: Incremental demand associated with an intervention such as a promotion.

Exploratory visualization: Graphical analysis used to discover structure, anomalies, and relationships.

Review Questions

Short Answer Questions

1. Why are sales and customer demand not always equivalent?
2. Distinguish seasonality from a cycle.
3. Why should a confirmed promotion not automatically be deleted as an outlier?
4. What is the difference between a zero observation and a missing observation?
5. Define demand censoring and explain how a stockout can bias a forecast.

Long Answer Questions

1. Explain how level, trend, seasonality, cycles, and structural breaks can be distinguished using time-series plots and business knowledge.
2. Describe a complete workflow for cleaning and preparing SKU-location demand data.
3. Discuss the advantages and limitations of temporal aggregation in supply-chain forecasting.
4. Explain intermittent demand and compare occurrence-and-size analysis with ordinary averaging.
5. Discuss governance and ethical risks associated with demand-data adjustment and automated forecasting.

Analytical / Application Questions

1. A product has 30 nonzero weeks out of 60, and mean positive demand is 6 units. Estimate mean weekly demand under the stated intermittent-demand identity, and explain when the estimate may be biased.
2. A monthly series rises sharply during a discount campaign, falls below baseline afterward, and then returns to its prior pattern. Diagnose the likely mechanisms and recommend how the data should be represented for forecasting.
3. A retailer reports low sales for a SKU during four weeks, but inventory was zero for two of those weeks. Explain how you would prepare the data and what additional information you would seek before selecting a forecasting method.

Multiple Choice Questions with Answers

Question 1

Which observation best represents demand censoring?

A. A duplicate transaction is recorded twice.

B. Sales equal available inventory during a stockout although customers wanted more.

C. Demand increases every December.

D. A product has a long-term declining trend.

Correct Answer: B. Sales equal available inventory during a stockout although customers wanted more.

Question 2

Which pattern repeats at a known calendar frequency?

A. Seasonality

B. A structural break

C. An isolated outlier

D. Random noise

Correct Answer: A. Seasonality

Question 3

If 18 of 52 weeks have positive demand and mean positive demand is 4 units, what is the estimated mean weekly demand?

A. 0.35 units

B. 1.38 units

C. 4.00 units

D. 14.44 units

Correct Answer: B. 1.38 units

Question 4

What is the most appropriate first response to an apparent demand outlier?

A. Delete it immediately

B. Replace it with zero

C. Investigate its operational cause

D. Duplicate it to preserve variance

Correct Answer: C. Investigate its operational cause

Question 5

Why can MAPE be unsuitable for intermittent demand?

A. It cannot compare forecasts across products

B. It is undefined when actual demand is zero

C. It always produces negative values

D. It requires promotional data

Correct Answer: B. It is undefined when actual demand is zero

Question 6

Which transaction would commonly be excluded when the target is external customer demand?

A. A customer shipment

B. A recorded stockout

C. An internal warehouse transfer

D. A point-of-sale transaction

Correct Answer: C. An internal warehouse transfer

Question 7

What does a structural break indicate?

A. A one-period random fluctuation only

B. A persistent change in the data-generating process

C. A fixed weekly seasonal pattern

D. A missing record that should be imputed

Correct Answer: B. A persistent change in the data-generating process

Question 8

A basic calendar adjustment for monthly demand may divide total demand by:

A. The number of effective selling days

B. The number of product codes

C. The maximum daily demand

D. The number of promotions only

Correct Answer: A. The number of effective selling days

Question 9

Which statement about aggregation is correct?

A. All quantities should be averaged

B. Aggregation always improves forecasts

C. Additive quantities are generally summed over the chosen interval

D. Aggregation eliminates the need to record stockouts

Correct Answer: C. Additive quantities are generally summed over the chosen interval

Question 10

What is pull-forward demand?

A. Demand permanently lost after a product launch

B. Demand shifted from a future period into an earlier promotional period

C. Demand caused by a data-entry error

D. Demand hidden by a stockout

Correct Answer: B. Demand shifted from a future period into an earlier promotional period