Poisson Processes and Continuous-Time Markov Chains

Introduction and Context

Many business systems evolve through events occurring at unpredictable times. Examples include customer arrivals, insurance claims, machine failures, loan defaults, support requests, website transactions, and changes in inventory status. Stochastic-process models represent this temporal uncertainty while preserving useful structure for forecasting, capacity planning, pricing, reliability analysis, and risk management.

This chapter develops two closely related model families. Poisson processes describe event counts and arrival times, while continuous-time Markov chains (CTMCs) describe systems that move among discrete states over continuous time. The Poisson process is often the arrival mechanism that drives a queue or service system; a CTMC provides a broader framework for representing state transitions and long-run behavior.

Learning Objectives

By the end of this chapter, you should be able to:

1. State and assess the assumptions of homogeneous and nonhomogeneous Poisson processes.
2. Derive event-count, interarrival-time, and arrival-time distributions.
3. Apply compound Poisson models to aggregate losses and transaction totals.
4. Construct a CTMC from transition rates and interpret its generator.
5. Analyze holding times, transition probabilities, transition functions, and stationary distributions.
6. Select appropriate models for business applications and evaluate their limitations.

1. Poisson Processes

Let N(t) denote the number of events observed in the interval [0,t], with N(0)=0. A counting process is a Poisson process with rate λ > 0 if it satisfies:

• Independent increments: counts in disjoint time intervals are independent.
• Stationary increments: the distribution of the count over an interval depends only on its length.
• Poisson counts: for every t≥0,

Here, λ is the average event rate, measured in events per unit time. The product λt is dimensionless and is the expected number of events in an interval of length t.

An equivalent infinitesimal characterization is

where o(∆t) denotes a term whose ratio to ∆t tends to zero as ∆t→0. Thus, in a very short interval, one event has probability approximately λ∆t, while two or more events are negligible relative to ∆t.

The mean and variance are both

This equality is a diagnostic implication of the basic Poisson model, not a universal law. If observed data exhibit substantial overdispersion, dependence, clustering, or time-varying rates, another model may be needed.

Interarrival and Arrival-Time Distributions

Let T₁,T₂,… be successive interarrival times. Under a Poisson process, the T_i are independent and identically distributed exponential random variables with rate λ:

The exponential distribution is memoryless:

This means that the elapsed waiting time does not alter the distribution of the remaining wait. It is a modeling assumption, not a claim that real customers forget the past.

The time of the kth event is S_k=T₁+⋯+T_k. It has an Erlang distribution (also called a gamma distribution with integer shape):

The event-count and arrival-time views are equivalent: S_k≤t if and only if N(t)≥k.

Worked Example 1: Call Arrivals

A support center receives calls at a constant average rate of 12 per hour. Assuming a Poisson process, the number of calls in 15 minutes has parameter λt=12×0.25=3. Therefore,

The probability of no calls in the next 10 minutes is

The expected waiting time until the next call is 1/12 hour, or 5 minutes. These calculations support staffing and response-time planning, but only if the rate is reasonably stable and calls are approximately independent.

2. Nonhomogeneous Poisson Processes

A constant rate is often unrealistic. Website traffic may peak at lunchtime, claims may increase after a storm, and transaction rates may vary by day of week. A nonhomogeneous Poisson process (NHPP) has a nonnegative intensity function λ(t). Define the cumulative intensity

The count in [a,b] is Poisson with mean Λ(a,b):

Counts in disjoint intervals remain independent, but their distributions are not stationary unless λ(t) is constant. The probability of no event in an interval is

If T is the next-event time after time s, then

The instantaneous rate changes with time, so interarrival times are generally not identically distributed and are not ordinarily exponential.

A useful simulation method is time transformation. If E₁,E₂,… are independent exponential random variables with rate 1, event times can be generated by solving

provided the cumulative intensity can be inverted numerically or analytically.

3. Compound Poisson Processes

A Poisson process counts events, but many business questions concern the total amount generated by events. Let N(t) be Poisson with rate λ, and let independent, identically distributed event sizes X₁,X₂,… have mean μ_X and variance σ_X². The aggregate amount is

with S(t)=0 when no events occur. This is a compound Poisson process. Its mean and variance are

The variance formula follows from conditioning on the event count:

If event sizes are monetary losses, units must be consistent: λ is events per year, t is years, and X_i is currency per event; hence S(t) is currency.

The probability generating function for integer-valued nonnegative event sizes is

where G_X(z)=E[z^X]]. For continuous sizes, the analogous moment-generating function is

when the moment-generating function exists near zero.

4. Continuous-Time Markov Chains

A process X(t) with a finite or countable state space is a CTMC if, conditional on the present state, the future is independent of the past. Formally, for future states and times, the conditional distribution given the complete history up to time t depends only on X(t). This is the continuous-time Markov property.

A CTMC is usually specified by transition rates q_ij for i≠j. Over a short interval h,

and

where the total exit rate from state i is

The generator, or rate matrix, Q, is defined by

Consequently, every row sums to zero and off-diagonal entries are nonnegative. A valid generator must satisfy these structural conditions.

Holding Times and Embedded Chains

When the process enters state i, the time H_i until its next transition is exponential with rate q_i:

If q_i>0, the probability that the next state is j is

The matrix P=(p_ij) is the transition matrix of the embedded discrete-time jump chain. Thus a CTMC separates into a holding-time mechanism and a jump-destination mechanism. If q_i=0, state i is absorbing.

Transition Functions and Kolmogorov Equations

Let

and let P(t) be the transition-function matrix. Under standard finite-state conditions,

The matrix exponential is defined by

It satisfies the Chapman–Kolmogorov relation P(s+t)=P(s)P(t). The forward and backward Kolmogorov equations are

and

respectively. For finite time-homogeneous chains, both expressions equal QP(t)=P(t)Q. The distinction is conceptual: the forward equation evolves probability distributions, whereas the backward equation conditions on the initial state.

Worked Example 2: A Two-State Reliability Chain

Suppose a device is either operating (state 0) or failed (state 1). Failures occur at rate α, and repair occurs at rate β. The generator is

The stationary probabilities π=(π₀,π₁) satisfy

Solving gives

If α=0.02 per hour and β=0.18 per hour, long-run availability is π₀=0.18/0.20=0.90, and long-run unavailability is 0.10. The mean operating spell is 1/α=50 hours, while the mean repair spell is 1/β≈5.56 hours.

5. Stationary and Steady-State Behavior

A probability vector π is stationary if

If the chain starts with distribution π, its distribution remains π at all future times. A stationary distribution represents a long-run equilibrium only under suitable recurrence and communicating-class conditions. In a finite irreducible CTMC, a unique stationary distribution exists and the distribution converges to it under standard nonexplosion assumptions.

For a birth–death process with birth rates λ_n and death rates μ_n, transitions occur only between neighboring states. Its generator has

When a stationary distribution exists, detailed balance often gives

and therefore

with π₀ chosen so that the probabilities sum to one. Detailed balance is sufficient for stationarity in this setting, but not necessary for every CTMC.

6. Illustrative Case: Dynamic Capacity Planning for a Digital Support Center

Background. An online retailer receives support requests whose intensity varies by hour. Requests are classified as waiting, being served, or resolved. Managers must decide whether to schedule an additional service agent during the evening peak.

Problem. A constant-rate Poisson model predicts an average of 8 requests per hour during the day and 16 per hour from 18:00 to 21:00. Each active agent completes requests at an average rate of 10 per hour. Management wants both an arrival forecast and a long-run congestion assessment.

Analysis. An NHPP is appropriate for the arrivals because the rate changes by time interval. The expected number during the three-hour peak is

If arrivals are independent conditional on the intensity, the peak count is Poisson with mean 48. For a simplified capacity model with two agents, the number of requests in the system can be represented by an M/M/2 queue, a CTMC whose birth rate is 16 per hour and whose service rate is 10 per hour per busy agent. When two agents are busy, the total service-completion rate is 20 per hour. Because the arrival rate is below the maximum service capacity, 16/(2×10)=0.8, the basic stability condition is satisfied.

Approach and solution. The manager should estimate the intensity function from timestamped requests, verify whether arrivals are overdispersed or clustered, and estimate service-time variability. The CTMC can then be used to calculate state probabilities and performance measures such as probability that all agents are busy, expected number waiting, and probability of delay. If the data violate exponential service assumptions, a more general queueing model or simulation should replace the simple CTMC.

Outcome. The model supports a conditional recommendation: two agents may be adequate on average, but a service-level target such as “90% answered within two minutes” requires a tail-probability analysis, not merely comparison of average rates. A third agent may be justified during short high-volume periods even when the average system is stable.

Learning points. Arrival modeling and state modeling are distinct tasks. The NHPP describes when work arrives; the CTMC describes how system occupancy changes. Forecast uncertainty, abandonment, nonstationary service rates, and correlated arrivals should be incorporated before operational deployment.

7. Model Selection, Limitations, and Governance

Poisson models are attractive because they are mathematically tractable and interpretable. Nevertheless, their assumptions can fail in several ways:

• Overdispersion: observed count variance exceeds the mean.
• Underdispersion: operational scheduling can make counts more regular than Poisson counts.
• Self-excitation: one event increases the near-term probability of another, as in social-media activity or cyberattacks.
• Common shocks: weather, promotions, outages, or economic news create dependence across events.
• Unobserved heterogeneity: differing customer segments can produce mixtures of rates.
• Non-exponential durations: service, repair, and customer patience times may have heavy or structured tails.
• State aggregation: a single state can conceal materially different risk profiles.

Best practice is to examine rate plots, interarrival-time diagnostics, dispersion statistics, residuals, and out-of-sample predictive performance. For a homogeneous Poisson model, count data over equal-length intervals should have approximately equal means and variances. This is a useful diagnostic, not a definitive test.

In analytics applications, timestamps and state labels may contain personal or commercially sensitive information. Governance should address data minimization, access controls, retention, model documentation, and reproducibility. If a CTMC informs credit, insurance, employment, or health decisions, analysts must assess fairness and explainability. A mathematically elegant process model does not remove the need for regulatory compliance or human oversight.

Modern implementations often estimate intensities and transition rates using maximum likelihood, Bayesian methods, or penalized regression. Machine-learning systems can forecast time-varying rates, but forecasts should be calibrated and stress-tested. AI-generated predictions should not be accepted merely because they improve in-sample fit; temporal leakage, distribution shift, and rare-event performance require explicit evaluation.

Chapter Summary

A homogeneous Poisson process models independent, stationary event increments with count mean and variance λt. Its interarrival times are independent exponential variables, and the time of the kth event is Erlang distributed. A nonhomogeneous Poisson process replaces the constant rate with an intensity λ(t), making cumulative intensity the central quantity. A compound Poisson process attaches random sizes to arrivals and provides clear formulas for aggregate mean and variance.

A CTMC models transitions among states using a generator Q. Off-diagonal generator entries are transition rates, diagonal entries are negative total exit rates, and holding times are exponential. The transition function is P(t)=e^Qt, and stationary behavior is found from πQ=0 together with normalization. Poisson processes are especially useful for arrivals, while CTMCs represent evolving operational, reliability, financial, and service states. Sound application requires checking independence, stationarity, exponential-duration, stability, data-quality, and governance assumptions.

Key Terms

Poisson process: Counting process with independent stationary increments and Poisson counts.

Intensity/rate: Expected event frequency per unit time.

Independent increments: Counts on disjoint intervals are independent.

Stationary increments: Count distribution depends only on interval length.

Interarrival time: Time between consecutive events.

Memorylessness: Property that the residual exponential waiting time is unaffected by elapsed time.

Erlang distribution: Distribution of a sum of independent exponential variables with a common rate.

Nonhomogeneous Poisson process: Poisson process with a time-varying intensity.

Cumulative intensity: Integral of the intensity over a time interval.

Compound Poisson process: Poisson event count with random marks or event sizes.

Continuous-time Markov chain: State process satisfying the Markov property in continuous time.

Transition rate: Instantaneous rate of moving from one state to another.

Generator: Matrix of CTMC transition rates, with rows summing to zero.

Holding time: Duration spent in a state before the next jump.

Embedded jump chain: Discrete-time chain recording successive CTMC states.

Transition function: Matrix of probabilities of moving between states over a specified time.

Stationary distribution: Probability vector satisfying πQ=0.

Birth–death process: CTMC with transitions only to neighboring states.

Nonexplosion: Property that only finitely many transitions occur in every finite time interval.

Review Questions

Short Answer Questions

1. State the three defining properties of a homogeneous Poisson process.
2. Why are Poisson-process interarrival times exponentially distributed?
3. What is the role of cumulative intensity in an NHPP?
4. Interpret an off-diagonal element of a CTMC generator.
5. Distinguish a stationary distribution from a transient distribution.

Long Answer Questions

1. Derive the mean and variance of a Poisson count and explain their modeling implications.
2. Explain the relationship between Poisson counts, exponential interarrival times, and Erlang arrival times.
3. Derive the mean and variance of a compound Poisson aggregate using conditional expectation and variance.
4. Explain how a CTMC generator determines holding times and jump destinations.
5. Discuss assumptions and diagnostic procedures for applying Poisson and CTMC models to business data.

Analytical / Application Questions

1. A claims process has rate 3 per day. Calculate the probability of exactly four claims in two days and the expected time until the fifth claim.
2. An NHPP has intensity 4 during the first hour and 10 during the next two hours. Find the probability of no events over the full three-hour period.
3. A repairable system has failure rate 0.01 per hour and repair rate 0.09 per hour. Construct its generator and calculate long-run availability.

Multiple Choice Questions with Answers

Question 1

For a Poisson process with rate 6 events per hour, what is the probability of no events in 30 minutes?

A. e^-1

B. e^-2

C. 1-e^-2

D. e^-6

Correct Answer: B. e^-2

Question 2

Which property distinguishes a nonhomogeneous Poisson process from a homogeneous one?

A. It permits at most one event overall

B. Its event sizes must be random

C. Its intensity may vary with time

D. Its increments cannot be independent

Correct Answer: C. Its intensity may vary with time

Question 3

If interarrival times are exponential with rate λ, their mean is

A. λ

B. λ²

C. 1/λ

D. e^-λ

Correct Answer: C. 1/λ

Question 4

For a compound Poisson process, which expression is the aggregate variance over time t?

A. λtμ_X

B. λtσ_X²

C. λt(σ_X²+μ_X²)

D. (λtμ_X)²

Correct Answer: C. λt(σ_X²+μ_X²)

Question 5

In a CTMC generator, the diagonal entry q_ii equals

A. The probability of remaining forever in state i

B. The negative total rate of leaving state i

C. The rate of entering state i from all states

D. Always zero

Correct Answer: B. The negative total rate of leaving state i

Question 6

If the total exit rate from a CTMC state is 0.25 per hour, the mean holding time is

A. 0.25 hours

B. 2 hours

C. 4 hours

D. 25 hours

Correct Answer: C. 4 hours

Question 7

For a finite CTMC, the transition-function matrix is generally written as

A. P(t)=Q/t

B. P(t)=e^Qt

C. P(t)=t^Q

D. P(t)=Q^t

Correct Answer: B. P(t)=e^Qt

Question 8

A stationary distribution π of a CTMC satisfies

A. πQ=1

B. Qπ=π

C. πQ=0

D. π_i=0

Correct Answer: C. πQ=0

Question 9

A two-state system fails at rate 0.02 per hour and is repaired at rate 0.18 per hour. Its long-run availability is

A. 0.10

B. 0.18

C. 0.20

D. 0.90

Correct Answer: D. 0.90

Question 10

Which observation most directly suggests that a homogeneous Poisson model may be inadequate?

A. Counts have approximately equal mean and variance

B. Interarrival times appear independent

C. The event rate changes systematically by hour

D. The observation interval has a fixed length

Correct Answer: C. The event rate changes systematically by hour