Foundations of Stochastic Processes

Introduction

A stochastic process is a mathematical model for a phenomenon that evolves over time while containing uncertainty. In business analytics, stochastic processes support demand forecasting, credit-risk analysis, queueing, inventory control, algorithmic trading, reliability analysis, and revenue management. Unlike a single random variable, which describes uncertainty at one point or for one outcome, a stochastic process describes a collection of random variables indexed by time or another ordering variable.

Let {X_t:t∈T} denote a stochastic process, where t belongs to an index set T. Each X_t is a random variable. The value of the process at time t is uncertain before observation, but after observation it becomes one realized value. A complete realization over all times is called a sample path or trajectory.

The central analytical task is to connect four elements: the state space, the time index, the probabilistic dependence between observations, and the information available as time progresses. This chapter develops those foundations and shows how transition probabilities, autocorrelation, filtrations, stopping times, stationarity, ergodicity, and memory provide complementary ways to classify and analyze stochastic behavior.

Learning Objectives

By the end of this chapter, you should be able to:

1. Define a stochastic process and distinguish its index set, state space, sample space, and sample paths.
2. Classify processes by discrete or continuous time, discrete or continuous state, and path regularity.
3. Explain increments, independent increments, stationary increments, and common process families.
4. Interpret transition probabilities, transition matrices, and multi-step transitions.
5. Distinguish strict stationarity, weak stationarity, and ergodicity.
6. Explain autocovariance, autocorrelation, filtrations, adaptedness, and stopping times.
7. Compare independent, Markov, finite-memory, and long-memory processes.
8. Apply these concepts to a business analytics problem and identify modeling limitations.

1. The Architecture of a Stochastic Process

1.1 Probability space and random variables

A formal probability model consists of a sample space Ω, a collection of events, and a probability measure P. An outcome ω∈Ω represents one possible complete state of the world. A stochastic process is a family of measurable functions

where S is the state space. For a fixed time t, X_t is a random variable. For a fixed outcome ω, the mapping

is a sample path.

This distinction is essential. Probability statements concern variation across possible outcomes, whereas a time-series graph usually displays one observed sample path. A process can therefore be random even when the analyst sees only one trajectory.

1.2 State space and time index

The time index T may be discrete, such as {0,1,2,…}, or continuous, such as [0,∞). The state space may also be discrete, such as the number of customers in a queue, or continuous, such as daily revenue or a stock price.

A process may also have a vector state, for example X_t∈ℝ^k, when several quantities evolve jointly. A state representation is useful only if it retains the information needed for prediction and decision-making.

1.3 Sample paths and path regularity

Sample paths may be smooth, discontinuous, piecewise constant, or highly irregular. A random walk changes at discrete observation times. A counting process is nondecreasing and jumps by integer amounts. Brownian motion has continuous paths but is almost surely nowhere differentiable. These distinctions affect simulation, estimation, and the validity of calculus-based models.

Important path properties include:

• Continuity: X_t changes without jumps.
• Right continuity: the value immediately after time t approaches the value at t.
• Càdlàg paths: right-continuous paths with left limits, common in jump processes.
• Monotonicity: the path never decreases, as in cumulative arrivals.
• Bounded variation: total movement over an interval is finite; Brownian paths do not have this property.

A process definition must therefore specify not only distributions but, where relevant, path behavior.

2. Increments and Fundamental Process Families

For s<t, the increment over an interval is

The increment measures change, rather than level. A process has independent increments if increments over non-overlapping intervals are independent. It has stationary increments if the distribution of an increment depends only on interval length, so that

where ≡_d means equality in distribution. Independent increments and stationary increments are different properties: one concerns dependence, the other concerns distributional stability.

2.1 Random walk

A discrete-time random walk is commonly written as

where Y_i are successive increments. If the increments are independent and identically distributed with mean μ and variance σ², then

and, when X₀ is fixed,

The variance grows with time, so a random walk with nonzero innovation variance is generally not weakly stationary.

2.2 Counting processes and Poisson processes

A counting process N(t) records the number of events by time t. It normally satisfies N(0)=0, integer-valuedness, and nondecreasing paths. A homogeneous Poisson process with rate λ>0 has independent increments and

Thus,

The parameter λ has units of events per unit time. The equality of mean and variance is a special property of the homogeneous Poisson model, not a universal property of counting processes.

2.3 Brownian motion

A standard Brownian motion W_t satisfies W₀=0, has continuous paths, independent increments, and

A Brownian motion with drift μ and volatility σ can be represented informally by

where μ is the expected rate of change and σ controls uncertainty. This differential notation requires stochastic-calculus interpretation; it is not an ordinary derivative equation. Increments over a time interval of length h are normally distributed with mean μh and variance σ²h.

3. Transition Probabilities and Markov Structure

A transition probability describes the chance of moving from one state to another. For a discrete-state, discrete-time process, define

A process is a Markov process if its conditional distribution given the entire past depends only on the present state:

for every appropriate set A. This is the Markov property. It does not mean that observations are independent; the present state can carry substantial information about the future.

For a finite state space, the one-step transition matrix is P=(p_ij). Every entry is nonnegative and each row sums to one. The n-step transition probabilities are entries of Pⁿ:

This follows from the Chapman–Kolmogorov relationship. For an intermediate time, it can be written as

Worked example: customer-status transitions

Suppose customers are classified as Active or Inactive. The one-month transition matrix is

Rows represent the current state and columns the next state. An active customer has a 0.80 probability of remaining active and a 0.20 probability of becoming inactive. The probability that an active customer is inactive after two months is

The first term represents Active → Active → Inactive; the second represents Active → Inactive → Inactive. The calculation assumes the transition probabilities remain constant and the Markov property is appropriate.

A stationary distribution π satisfies

For this example, let π_A=a and π_I=1−a. Solving a=0.80a+0.30(1−a) gives a=0.60. Thus the long-run proportions are 60% Active and 40% Inactive, provided the chain is suitably irreducible and aperiodic. A stationary distribution describes long-run state proportions; it does not imply that each individual customer behaves deterministically.

4. Stationarity and Ergodicity

4.1 Strict stationarity

A process is strictly stationary if shifting all observation times by the same amount does not change any finite-dimensional joint distribution. For times t₁,…,t_k and shift h,

whenever both sides are defined. Strict stationarity concerns the entire distribution, including all higher-order relationships.

4.2 Weak stationarity

A process is weakly, or covariance, stationary when its first two moments satisfy:

for all t, and

depends only on the lag h, not on calendar time t. The variance is therefore constant and equals γ(0), assuming finite second moments.

The autocorrelation function is

provided γ(0)>0. It is dimensionless and measures linear dependence at lag h. Autocorrelation does not detect every form of dependence; nonlinear dependence can exist even when autocorrelation is zero.

For a weakly stationary process, the variance of a sample mean bar X_n = (1/n)∑_t=1ⁿ X_t is

This formula shows why serial dependence affects the precision of averages. Treating correlated observations as independent generally understates uncertainty.

4.3 Ergodicity

Ergodicity concerns whether time averages reveal population or ensemble properties. A process is ergodic for its mean when, under appropriate conditions,

with probability one as n→∞. Stationarity alone does not guarantee ergodicity. A stationary process may contain a permanent, unobserved regime component, causing one long trajectory to remain unrepresentative of the full population.

In practice, ergodicity supports learning from one business history. If demand is stationary and ergodic, a sufficiently long record may provide useful information about its stable distribution. Structural breaks, persistent regimes, strategic reactions, and changing customer populations weaken this justification.

5. Memory, Dependence, and Predictability

5.1 Independence and conditional independence

A sequence is independent if the joint distribution factors into the product of its marginal distributions. Independence is stronger than zero correlation. If variables are jointly normally distributed, zero covariance implies independence; outside the jointly Gaussian setting, it generally does not.

A process has the Markov property when the present summarizes the past for predicting the future. A process can be non-independent but Markov, as in an autoregressive model. It can also be transformed into a Markov process by expanding its state to include relevant lags.

5.2 Finite and infinite memory

An autoregressive model of order one is

where ε_t is a zero-mean innovation uncorrelated with the past. If |φ|<1, the model has a weakly stationary solution with

when Var(ε_t)=σ_ε² and the innovations are independent over time. Its autocorrelation is ρ(h)=φ^h for nonnegative integer lags. The model has a one-step Markov representation in X_t, although shocks influence future values through a decaying sequence.

Long memory means dependence declines slowly, often more slowly than exponential decay. It may arise from aggregation, heterogeneous customer behavior, volatility clustering, or regime persistence. Long-memory claims require careful diagnostics because trends, seasonality, and structural breaks can mimic slowly decaying dependence.

6. Filtrations, Information, and Stopping Times

A filtration is an increasing sequence of σ-algebras,

where F_t represents information available by time t. The process is adapted to the filtration if X_t is measurable with respect to F_t; informally, its value at time t is known using information available at that time.

A random time τ is a stopping time with respect to {F_t} if the event “stopping has occurred by time t” is observable at time t:

Examples include the first day cumulative sales exceed a threshold, the time a queue becomes empty, or the first time a trading rule signals exit. The time at which a future maximum occurs is generally not a stopping time because determining it requires future information.

Stopping-time concepts prevent look-ahead bias. A forecasting or trading rule may use information in F_t, but it must not use F_t+1 when making a decision at time t. In advanced probability, optional stopping results impose additional conditions, such as bounded stopping times or suitable integrability; one must not assume that stopping preserves expectations without checking those conditions.

7. A Structured Classification Framework

The following sequence helps analysts classify a process before selecting a model:

1. What is indexed? Calendar time, event number, transaction count, or spatial location?
2. Is the index discrete or continuous? This affects observation and mathematical tools.
3. What is the state space? Counts, categories, real values, vectors, or functions?
4. What are the paths? Continuous, jump-like, monotone, bounded, or irregular?
5. How are increments distributed? Are they independent or stationary?
6. Is the process stationary? Examine level, variance, seasonality, and dependence.
7. Is it ergodic enough for one-record inference? Consider regimes and structural change.
8. What memory is present? Test Markov, autoregressive, seasonal, and long-memory explanations.
9. What information is available at each decision time? Define the filtration and prevent leakage.

No single label fully characterizes a process. For example, a process may be continuous-time, continuous-state, nonstationary, Markov, and have continuous sample paths.

8. Illustrative Case: Retail Demand and Replenishment

Background

A regional retailer records hourly demand for a high-value product. Stockouts are costly, but excessive inventory ties up capital and increases obsolescence. Management wants to choose a replenishment trigger and estimate the probability of stockout before the next delivery.

Problem

The analyst initially models hourly demand as independent and identically distributed. However, demand is higher during business hours, rises before promotional events, and often remains elevated for several consecutive hours. Ignoring these features leads to unreliable safety-stock decisions.

Analysis

Let D_t be demand in hour t. The time index is discrete and the state space is nonnegative integers. The observed process is a sample path, while future hourly demand is uncertain. A first diagnostic examines the mean by hour of day. Systematic differences indicate seasonality and therefore nonstationarity in the raw series.

The analyst decomposes demand conceptually into

where m_t represents a predictable time-of-day and promotion effect, and X_t is a residual process. If the residual mean and variance are approximately stable, a stationary model may be reasonable for X_t, even though D_t itself is not stationary.

A lag plot and autocorrelation function reveal positive dependence at short lags. An autoregressive residual model or a Markov-modulated demand model may therefore be more appropriate than independent demand. If demand is represented by event arrivals, a Poisson process may be considered, but overdispersion—variance greater than the mean—would signal that a basic Poisson assumption is inadequate.

The replenishment rule must use only information available at the decision time. If τ is the first time projected inventory falls below the reorder point, it is a stopping time when the projection uses current and past information. A rule based on the minimum inventory over the next week is not a stopping-time rule because it uses future observations.

Approach and solution

The team uses a three-stage approach:

• Estimate time-of-day and promotion effects using historical data.
• Model residual dependence and validate one-step-ahead forecasts using rolling, time-ordered evaluation.
• Simulate inventory paths under lead-time uncertainty and compare service level, holding cost, and stockout cost.

Transition probabilities can be used if demand conditions are classified as Low, Normal, or High and the state is sufficiently informative. The matrix must be re-estimated when promotion policy or customer composition changes. Long-run proportions from a transition matrix should not be interpreted as forecasts during a known seasonal peak.

Outcome

The revised model reduces stockouts in simulation without simply increasing average inventory. More importantly, the team reports uncertainty intervals and stress scenarios rather than relying on a single expected-demand estimate. Monitoring is established for changes in autocorrelation, mean level, and transition behavior.

Learning points

The case illustrates that classification precedes modeling. State definition, stationarity, memory, and information timing directly affect operational decisions. A mathematically elegant model can still be unsuitable if its assumptions conflict with promotions, structural changes, or the actual decision process.

9. Limitations, Governance, and Contemporary Practice

Stochastic-process models are abstractions. Parameter estimates can be unstable when data are short, regimes change, or rare events matter most. Aggregating data can hide dependence, while excessive disaggregation can create sparse states and unreliable transition estimates. Model selection should therefore combine statistical diagnostics, domain knowledge, and out-of-sample validation.

Governance is especially important in credit, insurance, employment, and pricing applications. Historical transitions may encode discrimination or unequal access. A Markov model trained on past customer behavior can reproduce institutional bias even when protected attributes are excluded. Analysts should document data provenance, state definitions, assumptions, missing-data treatment, validation windows, and monitoring thresholds. Decisions affecting people should include human oversight and appeal mechanisms where appropriate.

Modern machine-learning systems often estimate conditional distributions or latent states rather than using classical process labels explicitly. Neural sequence models can capture complex dependence, but interpretability, leakage control, calibration, drift monitoring, and reproducibility remain essential. A sophisticated predictor does not remove the need to define the information set or distinguish association from a valid transition mechanism.

Best practice is to begin with a transparent baseline, such as an independent-increment or autoregressive model, then add complexity only when diagnostics and decision value justify it. Analysts should report whether conclusions concern one observed path, a stationary population, or a simulated ensemble.

Chapter Summary

A stochastic process is a time-indexed family of random variables. Its analysis requires explicit attention to the index set, state space, sample paths, and information structure. Processes may be discrete or continuous in time and state, and their paths may be continuous, discontinuous, monotone, or irregular.

Increments describe change between times. Independent increments concern dependence across intervals, while stationary increments concern the distribution of change. Markov processes summarize the predictive content of the past through the present state, and transition matrices provide a practical representation for finite-state models.

Strict stationarity preserves all finite-dimensional distributions under time shifts. Weak stationarity requires constant mean and lag-dependent covariance. Ergodicity determines whether long-run time averages can represent ensemble properties. Autocorrelation measures linear dependence, but zero autocorrelation does not always imply independence. Filtrations formalize available information, and stopping times identify decisions that can be made without looking into the future.

For business analytics, the most important discipline is assumption control. Seasonality, structural breaks, regime changes, dependence, data leakage, and ethical risks can invalidate an otherwise correct mathematical calculation. Effective modeling combines process classification, diagnostics, domain knowledge, transparent validation, and responsible governance.

Key Terms

Stochastic process: A family of random variables indexed by time or another ordered set.

State space: The set of values that a process can take.

Time index: The set over which a process is observed or defined.

Sample path: One realized trajectory of a stochastic process.

Increment: The change X_t−X_s between two times.

Independent increments: Non-overlapping increments that are independent.

Stationary increments: Increments whose distributions depend only on interval length.

Markov property: Conditional independence of the future from the past given the present.

Transition probability: The conditional probability of moving between states.

Transition matrix: A matrix containing one-step transition probabilities.

Strict stationarity: Invariance of all finite-dimensional distributions under time shifts.

Weak stationarity: Constant mean and covariance depending only on lag.

Autocovariance: Covariance between process values separated by a specified lag.

Autocorrelation: Autocovariance divided by the process variance.

Ergodicity: A property allowing time averages to represent population properties.

Memory: The extent to which past observations affect future behavior.

Filtration: An increasing representation of information available over time.

Adapted process: A process whose value at each time is known from the corresponding information set.

Stopping time: A random time whose occurrence can be determined using information available at that time.

Structural break: A change in the data-generating process or its parameters.

Review Questions

5 Short Answer Questions

1. What is the difference between a sample path and a random variable in a stochastic process?
2. Distinguish independent increments from stationary increments.
3. State the Markov property in words and mathematical form.
4. What conditions define weak stationarity?
5. Why is a future maximum generally not a stopping time?

5 Long Answer Questions

1. Explain how state space, time index, and path regularity classify stochastic processes. Give business examples for at least four classifications.
2. Compare strict stationarity, weak stationarity, and ergodicity. Explain why these properties should not be treated as equivalent.
3. Derive and interpret the two-step transition probability for a finite-state Markov chain using the Chapman–Kolmogorov relationship.
4. Explain how autocorrelation and memory affect the estimation of a sample mean and the evaluation of forecasts.
5. Discuss how filtrations and stopping times help prevent look-ahead bias in operational or financial analytics.

3 Analytical / Application Questions

1. A retailer observes hourly demand with a strong daily pattern and positive autocorrelation. Design a process-classification and modeling workflow, identifying which assumptions should be tested before forecasting.
2. A two-state customer chain has transition matrix P=[[0.9,0.1],[0.4,0.6]]. Find its stationary distribution and interpret the result operationally.
3. An analyst claims that zero sample autocorrelation proves demand observations are independent. Critically assess the claim, state conditions under which it could be valid, and propose additional diagnostics.

Multiple Choice Questions with Answers

Question 1

Which statement best defines a sample path?

A. The complete probability distribution of a process

B. One realized trajectory of a stochastic process

C. The set of all possible states at one time

D. A matrix of transition probabilities

Correct Answer: B. One realized trajectory of a stochastic process

Question 2

For a finite-state Markov chain, what does the entry (P²)ᵢⱼ represent?

A. The probability of state i at time zero

B. The unconditional mean of state j

C. The probability of moving from i to j in two steps

D. The variance of the transition from i to j

Correct Answer: C. The probability of moving from i to j in two steps

Question 3

Which condition is required for weak stationarity?

A. The process must have independent observations

B. The mean must be constant and covariance must depend only on lag

C. Every sample path must be continuous

D. The state space must be finite

Correct Answer: B. The mean must be constant and covariance must depend only on lag

Question 4

For a homogeneous Poisson process with rate λ, what is Var[N(t)]?

A. λ

B. t

C. λ²t

D. λt

Correct Answer: D. λt

Question 5

What does ergodicity generally permit an analyst to do?

A. Infer long-run population properties from one sufficiently long trajectory

B. Assume all observations are independent

C. Guarantee that the process is stationary

D. Remove all measurement error

Correct Answer: A. Infer long-run population properties from one sufficiently long trajectory

Question 6

Which expression defines the autocorrelation at lag h for a weakly stationary process?

A. γ(0)/γ(h)

B. γ(h)/γ(0)

C. γ(h)+γ(0)

D. γ(h)γ(0)

Correct Answer: B. γ(h)/γ(0)

Question 7

A stopping time τ must satisfy which condition?

A. {τ > t} is independent of all information

B. {τ = t} must be known before time zero

C. {τ ≤ t} belongs to the information available at time t

D. τ must be a fixed, nonrandom number

Correct Answer: C. {τ ≤ t} belongs to the information available at time t

Question 8

In the model Xₜ = c + φXₜ₋₁ + εₜ, which condition gives the usual weakly stationary AR(1) solution?

A. φ = 2

B. |φ| < 1

C. σ²ε = 0 only

D. c = 0 only

Correct Answer: B. |φ| < 1

Question 9

Which statement correctly distinguishes independence from zero correlation?

A. They are always equivalent

B. Zero correlation always implies independence

C. Independence implies zero correlation when moments exist, but the converse generally fails

D. Independence is weaker than zero correlation

Correct Answer: C. Independence implies zero correlation when moments exist, but the converse generally fails

Question 10

In a transition matrix, what must each row sum to?

A. 0

B. The number of states

C. The stationary mean

D. 1

Correct Answer: D. 1