Demand Estimation and Business Forecasting

Introduction

Managers rarely make decisions with complete information. They must decide how much to produce, which markets to enter, what price to charge, how many employees to schedule, and how much inventory or capacity to reserve. Each decision depends partly on an estimate of future demand. Demand estimation asks, “What determines the quantity customers will buy?” Business forecasting asks, “What quantity is likely to be bought under specified future conditions?”

The distinction is important. Estimation is primarily explanatory and seeks to measure relationships among demand, price, income, promotion, competition, and other variables. Forecasting is primarily predictive and converts information into a future-oriented estimate. A useful forecast is not necessarily based on the most sophisticated model; it is based on a model appropriate to the decision, the data, the time horizon, and the cost of error.

This chapter develops managerial approaches to market research, experiments, regression, causal inference, time-series forecasting, scenario analysis, forecast evaluation, and responsible use of analytics and artificial intelligence.

Learning Objectives

After studying this chapter, you should be able to:

1. Distinguish demand estimation from forecasting and explain the determinants of demand.
2. Design and evaluate surveys, experiments, and observational studies.
3. Interpret demand regressions, elasticities, causal effects, and common statistical risks.
4. Apply time-series, causal, and scenario-based forecasting methods.
5. Evaluate forecast accuracy using suitable metrics and validation procedures.
6. Recognize data-quality, privacy, ethical, governance, and model-risk issues.
7. Translate estimates and forecasts into pricing, capacity, marketing, and strategic decisions.

1. The Managerial Meaning of Demand

For a product or service, market demand is the total quantity buyers are willing and able to purchase at alternative prices during a specified period. A firm's demand is the quantity purchased from that firm. These are not identical: a firm may lose market share even when total market demand increases.

A general demand function may be written as:

Here, Q_d is quantity demanded per period; P is the product's price; Y is buyer income or purchasing power; P_s and P_c are prices of substitutes and complements; A is advertising or promotion; T represents tastes and preferences; E represents expectations; and Z includes demographic, institutional, seasonal, and environmental factors. The function is conceptual unless a particular functional form and data-generating process are specified.

The law of demand states that, other things equal, quantity demanded generally decreases as own price increases. The phrase “other things equal” is essential. A price increase accompanied by a strong income increase, product improvement, or shortage of substitutes may be associated with an increase in observed sales.

Demand, sales, and market potential

Observed sales are not always equal to unconstrained demand. If a retailer sells out, sales understate demand. If distribution is unavailable, customers may be willing to buy but unable to do so. Managers should therefore distinguish:

• Observed sales: transactions recorded.
• Effective demand: purchases possible given availability, distribution, and information.
• Latent demand: desired purchases that were not completed.
• Market potential: the maximum realistic demand under specified conditions.

A forecast must define the target variable precisely: units ordered, units shipped, revenue, active users, transactions, or contribution margin. Forecasting revenue alone can conceal changes in price and volume.

2. Elasticity and Managerial Interpretation

Price elasticity measures the responsiveness of quantity demanded to a percentage change in price:

For a differentiable demand function, point elasticity is:

The variables must be measured consistently, and the derivative holds other demand determinants constant. Price elasticity is usually negative. If its absolute value exceeds one, demand is elastic; if it is less than one, demand is inelastic.

For a discrete change, midpoint elasticity reduces dependence on the choice of base period:

Income elasticity and cross-price elasticity are defined analogously:

A positive cross-price elasticity commonly indicates substitutes; a negative value commonly indicates complements. Elasticities are not universal constants. They vary by customer segment, time horizon, price range, competitive response, and product availability.

For a simple price–quantity relationship, total revenue is:

This identity is always true, but the effect of a price change on revenue depends on the elasticity of demand and on whether competitors, product mix, and capacity also change. Profit analysis additionally requires cost, not merely revenue.

3. Evidence for Demand Estimation

3.1 Market research and stated preferences

Market research may use customer interviews, focus groups, surveys, conjoint analysis, panels, and test markets. Qualitative methods are valuable for discovering motivations, vocabulary, unmet needs, and objections. Quantitative surveys estimate incidence, preferences, awareness, and stated purchase intention.

Stated intention is not identical to actual purchase. Respondents may overstate socially desirable behavior, misunderstand prices, or fail to anticipate budget constraints. Good survey design uses a defined target population, an appropriate sampling frame, neutral wording, randomized question order where appropriate, pretesting, and transparent handling of nonresponse.

Conjoint analysis asks respondents to evaluate combinations of attributes and prices. It can estimate trade-offs, but results depend on realistic attribute levels, task design, sample representativeness, and the gap between hypothetical and actual choices.

3.2 Experiments

A randomized experiment changes a treatment variable and assigns units to treatment and control groups by chance. Randomization helps balance observed and unobserved characteristics in expectation. A simple average treatment effect is:

Here, Y(1) is the outcome if a unit receives treatment and Y(0) is the outcome for the same unit without treatment. Both potential outcomes cannot normally be observed for the same unit; this is the fundamental problem of causal inference. The experimental estimate compares average outcomes across randomly assigned groups:

where Ȳ_T and Ȳ_C are treatment and control sample means. Randomization supports a causal interpretation when assignment is implemented correctly, interference is limited, outcomes are measured consistently, and attrition is not differential in a way that changes the comparison.

Business experiments include A/B tests of prices, promotions, website layouts, recommendations, and messages. Managers should specify the primary outcome before examining results, determine an adequate sample size, monitor practical as well as statistical significance, and consider spillovers, novelty effects, seasonality, and long-run customer value.

3.3 Observational data and causal inference

In observational data, prices and promotions are often chosen in response to expected demand. A regression may show that high prices coincide with high sales because firms raise prices when demand is strong. This is reverse causality or simultaneity, not evidence that higher prices increase demand.

Causal analysis requires a credible counterfactual: what would have happened to the treated units without treatment? Useful designs include randomized experiments, natural experiments, difference-in-differences, regression discontinuity, instrumental variables, and matched or weighted comparisons. Each requires explicit assumptions.

Difference-in-differences compares changes over time in a treated group with changes in a comparison group:

Its central identifying assumption is parallel trends: absent treatment, the treated and comparison groups would have experienced similar changes. The assumption should be investigated using pre-treatment trends; it cannot be proven solely from the post-treatment estimate.

An instrumental variable must affect the potentially endogenous treatment but influence the outcome only through that treatment, conditional on the model. These are strong, context-specific assumptions, and instrumental-variable estimates may apply only to a particular subgroup affected by the instrument.

4. Regression-Based Demand Estimation

A commonly used linear specification is:

Q_t is quantity demanded in period t; P_t is own price; Y_t is income; A_t is advertising; P_s,t is a substitute's price; β₀ is the intercept; β_i are partial effects; and u_t contains unobserved influences. In a linear model, β₁ measures the change in units associated with a one-unit price change, holding included variables constant. Its unit is quantity units per currency unit.

A log–log model is often convenient:

When variables are positive and the specification is appropriate, β₁, β₂, and β₃ are elasticities. Log transformations can reduce skewness and represent proportional relationships, but they do not automatically solve endogeneity, omitted-variable bias, measurement error, or nonstationarity.

Ordinary least squares chooses coefficients that minimize the sum of squared residuals:

The method is interpretable under assumptions that include a correctly specified conditional mean, appropriate sampling or assignment, no perfect multicollinearity, and an exogeneity condition such as E(u_t | X_t) = 0 for unbiased conditional interpretation. In time-series applications, autocorrelation, structural breaks, changing variance, and nonstationarity require additional treatment. Robust or clustered standard errors may improve inference, but they do not repair a biased design.

A high R² does not establish causality or guarantee predictive accuracy. Managers should inspect residuals, validate out of sample, test economically plausible signs, assess multicollinearity, and report uncertainty. Forecast intervals are more informative than a single point estimate.

5. Time-Series Forecasting

Time-series forecasting uses the historical behavior of a variable. It is especially useful when demand has stable patterns and explanatory variables are unavailable or themselves difficult to forecast.

5.1 Components and baseline methods

Demand may contain level, trend, seasonality, cycles, irregular shocks, and calendar effects. A moving average smooths recent observations:

Here, A_t is actual demand at time t, F_t+1 is the next forecast, and k is the number of periods averaged. A larger k produces more smoothing but responds more slowly to change.

Simple exponential smoothing is:

where 0 < α < 1. A larger α gives greater weight to recent observations. This method is appropriate for a series with a changing level but no systematic trend or seasonality. Trend and seasonal data require extensions such as Holt’s trend method or Holt–Winters methods, with parameters estimated and validated rather than selected mechanically.

5.2 Autoregression and decomposition

An autoregressive model uses prior observations:

where p is the lag order and ε_t is an innovation with conditional mean approximately zero. Stationarity conditions depend on the model; an autoregressive process should not be used uncritically when trend or changing variance is present. Differencing, transformations, seasonal terms, or models with external predictors may be needed.

Forecasting should respect time order. Randomly splitting time-series observations can leak future information into training. Rolling-origin or expanding-window validation better simulates actual deployment.

6. Scenario and Judgmental Forecasting

A statistical forecast is conditional on assumptions. Scenario forecasting makes those assumptions explicit. A manager may construct baseline, upside, and downside scenarios using alternative paths for price, income, competitor entry, regulation, supply, and consumer behavior.

Scenario analysis is not the same as assigning arbitrary probabilities. If probabilities are credible, expected demand can be calculated:

where Q_s is demand under scenario s, p_s is its probability, and ∑p_s = 1. If probabilities are not defensible, managers should present a range and decision triggers rather than a falsely precise expected value.

Judgment is valuable when historical data are sparse, a product is new, or a structural break has occurred. It should be documented, separated from model output, and reviewed for anchoring, overconfidence, availability bias, and political pressure. Combining forecasts can improve robustness when component forecasts contain different information; weights should be validated rather than chosen only for convenience.

7. Forecast Accuracy and Decision Quality

Let actual demand be A_t and forecast demand be F_t, for t = 1,…,n. Common measures include:

MAE is expressed in demand units and is easy to interpret. RMSE penalizes large errors more heavily and is in demand units. MAPE is a percentage but is undefined when A_t = 0 and unstable when actual values are close to zero. Alternatives include weighted absolute percentage error or scale-free measures such as MASE, depending on the decision context.

Accuracy should be assessed against a naïve benchmark, such as the last-observation forecast or seasonal naïve forecast. Use a holdout period or rolling-origin evaluation, preserve the information available at the forecast date, and evaluate across relevant products and horizons. Statistical accuracy is not the same as business value: an error of 100 units may be trivial for one product and catastrophic for another. Inventory cost, stockout cost, capacity constraints, and asymmetric consequences should influence model selection.

Forecast bias can be monitored through mean error:

Persistent positive or negative error indicates systematic underforecasting or overforecasting. Accuracy reports should include uncertainty, segment-level performance, data revisions, and the operational consequences of error.

8. Data Quality, Privacy, and Responsible AI

Demand analytics inherits defects in its data. Important checks include completeness, validity, consistent units, duplicate records, timestamp accuracy, missingness mechanisms, outliers, changes in product codes, stockout censoring, and alignment between sales, price, promotion, and availability. A sales drop may represent weak demand, a stockout, a distribution failure, or a tracking change.

Privacy requires purpose limitation, data minimization, access control, retention limits, security, and lawful and transparent processing. Personally identifiable or sensitive data should not be used merely because they are available. Aggregation and de-identification reduce risk but do not guarantee anonymity, particularly when datasets can be linked.

AI and machine-learning models can discover nonlinearities and interactions, but they can also amplify historical discrimination, exploit proxy variables, conceal leakage, and fail under distribution shift. Responsible deployment requires:

• a documented business purpose and decision owner;
• data lineage, consent or lawful basis, and access controls;
• separation of training, validation, and test information;
• explainability appropriate to the decision's stakes;
• monitoring for drift, bias, instability, and forecast degradation;
• human review for consequential decisions;
• version control, reproducibility, audit logs, and rollback procedures;
• security against manipulation and adversarial inputs.

Generative AI may assist with feature documentation, code drafts, or scenario narratives, but generated content must be verified. It should not be treated as evidence, and confidential customer data should not be entered into an unapproved system. Model risk includes not only algorithmic error but also incorrect objectives, poor implementation, inappropriate use, and failure to act on warnings.

9. An Integrated Managerial Workflow

A disciplined demand project follows these stages:

1. Define the decision and target. Specify product, geography, horizon, frequency, unit, and cost of errors.
2. Construct a causal and commercial map. Identify price, income, substitutes, complements, promotion, availability, seasonality, and institutional factors.
3. Audit data and measurement. Document sources, missingness, stockouts, revisions, units, and privacy constraints.
4. Choose an identification or forecasting design. Use experiments where feasible; otherwise justify observational or time-series assumptions.
5. Build transparent baselines. Compare regression, naïve, smoothing, seasonal, and scenario methods as appropriate.
6. Validate realistically. Use holdouts or rolling windows and compare both statistical and economic performance.
7. Stress-test. Examine structural breaks, alternative assumptions, extreme scenarios, and subgroup performance.
8. Deploy with governance. Assign accountability, monitor drift and bias, record overrides, and establish review triggers.
9. Translate into action. Link the forecast to pricing, procurement, staffing, capacity, inventory, and contingency plans.

10. Illustrative Case: Forecasting Demand for a Subscription Meal Service

Background and problem

FreshRoute, a fictional subscription meal company, experienced rapid growth but frequent stockouts on vegetarian plans and excess waste on premium plans. Management needed a weekly forecast for procurement and staffing, while also estimating whether a proposed introductory discount would create incremental demand.

Analysis

The team first defined demand as completed orders by plan and delivery region, not website visits. It discovered that recorded sales were censored during stockouts. Historical prices, promotions, holidays, weather, delivery availability, and cancellations were joined at the weekly level. Customer-level data were aggregated for forecasting, with access restricted to authorized analysts.

For the discount, management ran a randomized regional experiment. Eligible new customers were randomly assigned either the standard offer or a discount. The primary outcome was completed orders after four weeks; cancellation and contribution margin were secondary outcomes. Randomization was checked, and results were reported with confidence intervals rather than only a point estimate.

For forecasting, a seasonal naïve benchmark, exponential smoothing model, and regression with holiday, promotion, price, and availability variables were compared using rolling-origin validation. Weeks with stockouts were flagged rather than interpreted as normal demand. Scenarios represented normal supply, a supplier disruption, and a successful advertising campaign.

Solution and outcome

The seasonal naïve model was more accurate for stable plans, while the regression model performed better for plans with promotions and changing availability. The company used a forecast ensemble only where validation showed improvement. Procurement received a baseline forecast plus an upside and downside range. A stockout indicator and service-level rule were added to prevent the model from learning that unavailable products had low demand.

The discount experiment showed increased trial orders but also higher cancellations. Management therefore evaluated incremental contribution margin rather than orders alone and redesigned the offer around a smaller discount with a longer commitment. Forecast monitoring tracked bias by plan, region, and customer segment.

Learning points

The case illustrates that demand estimation and forecasting are complementary; sales data may understate demand; experiments are preferable for causal questions; forecast validation must respect time; and operational decisions require ranges, constraints, and costs—not merely a single predicted number.

Challenges and Limitations

No demand model can eliminate uncertainty. Consumer preferences change, competitors respond, regulations shift, and shocks may lie outside historical experience. Observational estimates may be biased even when statistically precise. Experiments may have limited external validity or short-lived effects. Time-series models can fail after structural breaks. Scenario analysis can become storytelling without disciplined assumptions. More data can increase confidence without increasing truth if measurement remains defective.

Managers should therefore treat estimates as conditional evidence. They should state assumptions, distinguish association from causation, quantify uncertainty, compare alternatives, and preserve the ability to revise decisions as new information arrives.

Contemporary Developments and Best Practice

Modern demand systems increasingly combine causal experimentation, high-frequency transaction data, machine learning, probabilistic forecasting, and human judgment. Probabilistic forecasts report distributions or intervals, enabling decisions under asymmetric costs. Hierarchical methods reconcile forecasts across product, region, and enterprise levels. Real-time systems can detect demand shifts, but speed increases the importance of monitoring, access control, and change management.

Best practice is not “use AI.” It is to use the least complex method that meets the decision need, establish a credible benchmark, validate against realistic future information, communicate uncertainty, and govern the full data-to-decision process.

Chapter Summary

Demand estimation explains how quantity demanded relates to price, income, promotion, competition, expectations, and other determinants; forecasting predicts future demand under stated assumptions. Market research reveals needs but may suffer from hypothetical bias. Experiments support causal conclusions when randomization and implementation are credible, while observational methods require explicit counterfactual assumptions. Regression coefficients must be interpreted in units and within the limits of identification; elasticities are often more useful for managerial decisions.

Time-series methods exploit level, trend, seasonality, and dependence, whereas scenario methods make uncertain external conditions explicit. Forecasts should be evaluated out of sample against naïve benchmarks using metrics suited to the data and decision costs. Data quality, stockout censoring, privacy, fairness, model drift, explainability, and human accountability are central to responsible analytics and AI. The managerial objective is not maximum statistical sophistication but reliable, transparent, and actionable evidence.

Key Terms

Demand estimation: Measuring relationships that explain quantity demanded.

Business forecasting: Predicting a future business variable under specified information and assumptions.

Market potential: Maximum realistic demand under defined conditions.

Elasticity: Percentage responsiveness of one variable to a percentage change in another.

Price elasticity: Responsiveness of quantity demanded to own price.

Cross-price elasticity: Responsiveness of one product’s demand to another product’s price.

Conjoint analysis: A stated-preference method estimating trade-offs among product attributes.

Randomized experiment: Study assigning treatment by chance to estimate causal effects.

Counterfactual: Outcome that would have occurred under an alternative treatment condition.

Average treatment effect: Average difference between potential treated and untreated outcomes.

Endogeneity: Correlation between a predictor and the unobserved error component.

Ordinary least squares: Estimation method minimizing squared residuals.

Time series: Observations indexed in temporal order.

Seasonality: Systematic periodic variation within a year or other recurring cycle.

Exponential smoothing: Forecasting method weighting recent observations more heavily.

Scenario analysis: Evaluation of outcomes under alternative coherent assumptions.

MAE: Mean absolute forecast error in the variable’s units.

RMSE: Square root of mean squared forecast error.

Model risk: Risk of loss from incorrect model design, data, implementation, or use.

Data drift: Change in the distribution or relationship of data over time.

Review Questions

Short Answer Questions

1. Distinguish observed sales, effective demand, and latent demand.
2. What does a price elasticity of −1.6 imply?
3. Why can a high regression R² fail to establish causality?
4. State one assumption required for difference-in-differences analysis.
5. Why is MAPE problematic when actual demand is zero?

Long Answer Questions

1. Explain how market research, experiments, regression, and time-series methods answer different managerial questions.
2. Discuss the assumptions and limitations of a log–log demand model.
3. Compare moving averages, exponential smoothing, and autoregressive models.
4. Explain how forecast accuracy should be linked to inventory and capacity decisions.
5. Develop a governance framework for an AI-based demand forecasting system.

Analytical / Application Questions

1. A retailer observes that price and sales rose together during a holiday period. Explain at least three possible interpretations and design a study to estimate the causal price effect.
2. A product has actual weekly demand of 100, 120, and 80 units and forecasts of 90, 110, and 100 units. Calculate MAE and identify whether the model underforecasted or overforecasted overall using mean error.
3. Design baseline, upside, and downside demand scenarios for a new service facing uncertain competitor entry. Specify the drivers, decision triggers, and information that would update each scenario.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes demand estimation from forecasting?

A. Estimation explains relationships; forecasting predicts future outcomes under assumptions.

B. Estimation always uses surveys; forecasting always uses regression.

C. Estimation concerns only revenue; forecasting concerns only quantity.

D. Estimation is qualitative; forecasting is necessarily causal.

Correct Answer: A. Estimation explains relationships; forecasting predicts future outcomes under assumptions.

Question 2

If the absolute value of price elasticity is greater than one, demand is described as:

A. Perfectly inelastic

B. Inelastic

C. Elastic

D. Unit elastic

Correct Answer: C. Elastic

Question 3

Random assignment in an experiment primarily helps by:

A. Guaranteeing a large sample

B. Balancing observed and unobserved characteristics in expectation

C. Eliminating all measurement error

D. Ensuring every participant complies

Correct Answer: B. Balancing observed and unobserved characteristics in expectation

Question 4

In a log–log demand model, a coefficient on log price is commonly interpreted as the:

A. Price in currency units

B. Change in units for one currency increase

C. Price elasticity, under the model’s assumptions

D. Forecast error percentage

Correct Answer: C. Price elasticity, under the model’s assumptions

Question 5

Which method is most suitable as a simple benchmark for a seasonal weekly series?

A. Seasonal naïve forecast

B. Randomly shuffled train-test split

C. A survey average only

D. A model using future actual demand

Correct Answer: A. Seasonal naïve forecast

Question 6

Which metric is expressed in the same units as demand and penalizes large errors more strongly?

A. MAPE

B. RMSE

C. Mean error percentage

D. Cross-price elasticity

Correct Answer: B. RMSE

Question 7

The parallel-trends assumption is central to:

A. Moving averages

B. Difference-in-differences

C. Conjoint analysis

D. Simple exponential smoothing

Correct Answer: B. Difference-in-differences

Question 8

A stockout can cause recorded sales to:

A. Overstate unconstrained demand automatically

B. Understate demand because customers cannot purchase

C. Become a causal estimate of price elasticity

D. Remove seasonality from the data

Correct Answer: B. Understate demand because customers cannot purchase

Question 9

If scenario probabilities are credible and sum to one, expected demand is calculated by:

A. Choosing the highest scenario only

B. A probability-weighted sum of scenario demands

C. A simple average regardless of probabilities

D. Subtracting downside demand from upside demand

Correct Answer: B. A probability-weighted sum of scenario demands

Question 10

Which is a responsible AI practice in demand forecasting?

A. Deploying the model without a benchmark

B. Using confidential customer data in any public tool

C. Monitoring drift, bias, performance, and access controls

D. Treating generated explanations as verified evidence

Correct Answer: C. Monitoring drift, bias, performance, and access controls