Healthcare Data, Econometrics and Decision Analytics

Introduction

Healthcare management increasingly depends on the ability to convert complex data into defensible decisions. Hospitals, insurers, public agencies and pharmaceutical organizations use data to measure need, allocate resources, evaluate interventions, forecast demand and assess value. Yet healthcare data are not neutral representations of reality. They are generated by clinical practices, payment systems, institutional incentives, documentation habits and unequal access to care. Sound analytics therefore requires both technical competence and critical judgment.

This chapter examines a connected set of methods: descriptive analytics, regression, causal inference, outcomes research, forecasting, decision trees, Markov models, simulation and uncertainty analysis. The methods differ in purpose. Descriptive analytics summarizes what has occurred; predictive analytics estimates what may occur; causal inference investigates what would happen under an intervention; and decision analysis compares actions under competing outcomes, costs and risks.

A useful analytical workflow is:

1. Define the decision and the target population.
2. Specify the outcome, time horizon and perspective.
3. Identify the data-generating process and possible biases.
4. Clean, link and describe the data.
5. Select a model consistent with the question and outcome.
6. Estimate effects or predictions with uncertainty intervals.
7. Conduct sensitivity and subgroup analyses.
8. Translate results into an implementable decision.
9. Monitor performance after implementation.

Learning Objectives

By the end of this chapter, the reader should be able to:

• distinguish descriptive, predictive and causal questions;
• assess the strengths and limitations of common healthcare datasets;
• calculate and interpret important descriptive and regression measures;
• recognize confounding, selection bias, measurement error and missing-data problems;
• design basic outcomes research and causal-inference analyses;
• apply forecasting, decision trees, Markov models and simulation;
• interpret uncertainty, probabilistic sensitivity analysis and value of information;
• evaluate ethical, governance and managerial implications of healthcare analytics.

1. Healthcare Data as Evidence

1.1 Major data sources

Healthcare datasets differ in purpose, coverage, granularity and validity. Electronic health records contain diagnoses, observations, medications, laboratory results and clinical notes, but documentation may be incomplete or influenced by reimbursement. Administrative claims are well suited to utilization and expenditure analysis, but diagnosis codes may be optimized for billing rather than clinical truth. Registries often provide carefully defined clinical variables and outcomes, although they may cover only selected institutions or patient groups.

Other sources include surveys, mortality records, pharmacy databases, laboratory systems, patient-generated data, wearable devices, social-care records and qualitative interviews. Linking sources can improve completeness, but linkage errors, inconsistent identifiers and privacy risks must be managed.

1.2 Measurement and data quality

A variable should be defined operationally before analysis. For example, “readmission” requires a specified interval, such as an unplanned admission within 30 days of discharge, and rules concerning transfers and planned procedures. Reliability concerns consistency of measurement; validity concerns whether the measurement represents the intended construct. A blood-pressure device may be reliable but systematically inaccurate, whereas a valid clinical concept may be measured inconsistently across sites.

Important quality dimensions include completeness, accuracy, timeliness, consistency, uniqueness and provenance. Data dictionaries, version control, reproducible code and an audit trail are essential for professional work. Analysts should distinguish missing completely at random, missing at random and missing not at random. The last category is especially important when missingness is related to disease severity, income or treatment response.

1.3 Descriptive analytics

Descriptive analysis begins with denominators. A count without its population at risk can be misleading. Common measures include:

Here, x is the number of events and n is the relevant population. If x = 80 infections occur among n = 2,000 admissions, p = 0.04, or 4%. The denominator must match the question: admissions, patient-days, procedures or people.

A rate incorporates time:

where E is the number of events and T is total person-time. If 30 events occur during 1,500 patient-years, the rate is 0.02 events per patient-year, or 20 per 1,000 patient-years. Rates are not automatically risks; a rate uses person-time, whereas risk is the proportion experiencing an event during a defined period.

For a sample mean and variance:

The mean is sensitive to extreme values, while the median is often preferable for skewed costs or length of stay. A confidence interval expresses sampling uncertainty, not the probability that a fixed parameter is random. For a large-sample mean, a 95% confidence interval is commonly approximated by:

where SE is the standard error. The approximation requires an appropriate sampling framework and is less reliable in small or highly non-normal samples without suitable methods.

2. Econometric Thinking in Healthcare

Econometrics applies statistical models to economic and policy questions while recognizing that behavior, institutions and treatment assignment are not random. Healthcare applications include demand, moral hazard, provider incentives, insurance choice, treatment effects, costs and productivity.

2.1 Regression models

The ordinary least-squares linear model is:

Here, yᵢ is the outcome for person i, x variables are predictors, β coefficients are parameters and εᵢ is the error term. βⱼ represents the expected change in y associated with a one-unit change in xⱼ, holding included variables constant. The interpretation is causal only under additional assumptions; regression adjustment alone does not guarantee causality.

Key assumptions for standard inference include correct specification of the conditional mean, independent observations or appropriately modeled dependence, and valid estimation of uncertainty. Homoscedasticity is not required for unbiased coefficients, but heteroscedasticity affects conventional standard errors; robust standard errors may be appropriate. Clustering is needed when observations within hospitals, practices or geographic areas are correlated.

Binary outcomes are often modeled using logistic regression:

where pᵢ = Pr(yᵢ = 1 | xᵢ). The quantity exp(βⱼ) is an odds ratio for a one-unit increase in xⱼ, conditional on the model. Odds ratios are not risk ratios, especially when outcomes are common. For managerial communication, predicted probabilities and absolute risk differences are frequently more meaningful.

Healthcare costs are often right-skewed and contain zeros. Alternatives include generalized linear models with a log link and appropriate variance distribution, two-part models for probability of any cost and positive cost, quantile regression, or carefully justified transformations. Length-of-stay models may require count, survival or zero-truncated methods. Model choice should reflect the data-generating process rather than convenience.

2.2 Confounding and causal inference

A confounder is associated with both exposure and outcome and is not a consequence of the exposure. For example, severity may influence both receipt of intensive treatment and mortality. Comparing treated and untreated patients without addressing severity may produce confounding by indication.

The potential-outcomes framework defines Y(1) as the outcome if a person receives treatment and Y(0) as the outcome under control. The average treatment effect is:

Individual causal effects cannot generally be observed simultaneously; this is the fundamental problem of causal inference. Identification commonly requires consistency, exchangeability conditional on measured covariates, and positivity. Consistency means the treatment and outcome are well defined; exchangeability means no unmeasured confounding after adjustment; positivity means every relevant subgroup has a nonzero probability of receiving each treatment.

Methods include randomization, regression adjustment, matching, inverse-probability weighting, propensity scores, instrumental variables, regression discontinuity and difference-in-differences. Each relies on assumptions that must be defended, not merely named.

A difference-in-differences estimator compares changes over time in a treated group with changes in a comparison group:

The central assumption is parallel trends: absent the intervention, the groups would have experienced similar average changes. Pre-intervention trends can be examined, but cannot prove the assumption. Time-varying confounding, spillovers and concurrent policies may invalidate the design.

Instrumental-variable analysis can address unmeasured confounding when an instrument affects treatment, affects the outcome only through treatment, and is independent of unmeasured causes of the outcome. These assumptions are difficult to verify. The resulting local average treatment effect generally applies to compliers affected by the instrument, not necessarily to all patients.

Directed acyclic graphs can clarify which variables should be adjusted for. Adjusting for a mediator can remove part of a treatment effect, while adjusting for a collider can create bias. A post-treatment variable should not be included automatically as a “control.”

2.3 Bias, uncertainty and missing data

Selection bias occurs when inclusion or retention depends on exposure, outcome or their causes. Information bias results from inaccurate measurement. Survivorship bias excludes those who leave observation or die. Ecological fallacy occurs when group-level associations are incorrectly applied to individuals. Simpson’s paradox describes reversal of an association after stratification.

Missing-data strategies include complete-case analysis, explicit missingness categories where substantively justified, multiple imputation and inverse-probability methods. Multiple imputation is defensible when the imputation model includes variables predicting missingness and outcomes, but it does not eliminate bias when missingness is strongly informative and unmeasured.

Precision is not the same as validity. A very large biased dataset can produce a narrow confidence interval around a wrong estimate. Analysts should report effect sizes, uncertainty intervals, model diagnostics, sensitivity analyses and clinically meaningful thresholds.

3. Outcomes Research and Evaluation

Outcomes research examines the effects of healthcare on survival, function, symptoms, safety, experience, equity and cost. A strong study specifies a population, intervention or exposure, comparator, outcomes, time horizon and analytic perspective.

Patient-reported outcomes capture health status directly from patients. Quality-adjusted life-years combine length and quality of life:

where uₜ is utility on a scale commonly anchored at 0 for death and 1 for full health, and Δt is time spent in the health state. Utilities may be elicited using standard gamble, time trade-off or validated instruments. QALYs are useful for comparison, but they involve value judgments and may not fully represent equity, caregiver burden or capabilities.

Economic evaluation depends on perspective. A healthcare-system perspective may include payer costs; a societal perspective may additionally include patient time, productivity and informal care. The incremental cost-effectiveness ratio is:

where C denotes cost and E denotes effectiveness for intervention 1 and comparator 0. The ICER is meaningful when the intervention is not dominated and the denominator is positive or negative in a properly interpreted comparison. Costs and outcomes occurring in future periods are commonly discounted:

where Vₜ is a future value at time t, d is the discount rate per period and PV is present value. The chosen rate and treatment of health outcomes should follow applicable national guidance.

4. Forecasting Healthcare Demand

Forecasting supports workforce planning, bed capacity, inventory and emergency preparedness. A time series may contain level, trend, seasonality, cycles and irregular shocks. Forecasting should separate training data from test data and avoid leakage of future information.

A simple moving average is:

where m is the window length. Exponential smoothing assigns greater weight to recent observations:

where 0 ≤ α ≤ 1. Larger α responds more rapidly to changes but may be less smooth. Seasonal demand may require seasonal indices, regression with calendar variables, or models such as ARIMA. Forecast accuracy should be evaluated out of sample using measures such as mean absolute error:

MAE is expressed in the outcome’s units. Mean absolute percentage error can be unstable when actual values are zero or close to zero. Prediction intervals should accompany point forecasts, and managers should understand that structural breaks—such as a pandemic, policy change or new competitor—can invalidate historical patterns.

5. Decision Trees and Markov Models

5.1 Decision trees

Decision trees represent sequences of choices and uncertain events over a finite horizon. Decision nodes are usually shown as squares, chance nodes as circles and terminal outcomes as triangles. Expected value is:

where pⱼ is the probability of outcome j and vⱼ is its value. Probabilities at a chance node must sum to 1. Rollback analysis calculates expected values from terminal branches toward the initial decision.

Illustrative example: a screening strategy costs $100 per person. It identifies disease with probability 0.10, and early treatment produces a benefit valued at $2,000. Assume perfect detection for simplicity and no benefit otherwise. The expected net value is:

This is a simplified value calculation, not a complete cost-effectiveness analysis. Real decisions must include false positives, downstream harms, treatment costs, time preferences and uncertainty.

5.2 Markov models

Markov models are useful when disease processes involve recurring transitions among health states. The Markov property assumes that the future depends on the current state, not the full prior history. A state-transition probability matrix P contains probabilities pᵢⱼ of moving from state i to state j in one cycle:

Each row of P sums to 1. If vₜ is a row vector of state occupancy, then:

State rewards can represent costs or utilities. If r is a column vector of rewards per cycle, expected reward in cycle t is vₜr, and total undiscounted reward over T cycles is:

A Markov model may be inappropriate when history strongly affects future risk, treatment wanes, or events occur at variable times. Tunnel states, semi-Markov models or individual-level simulation can address some limitations. Cycle length, half-cycle correction, transition calibration and competing risks require careful justification.

6. Simulation and Uncertainty Analysis

Simulation represents variability and uncertainty explicitly. In discrete-event simulation, patients, arrivals, services and queues are modeled as events over time. In Monte Carlo simulation, uncertain parameters are sampled repeatedly from specified probability distributions. For example, a probability may be assigned a beta distribution and a cost a gamma or lognormal distribution, subject to parameter justification.

Variability describes real differences among patients or arrivals; parameter uncertainty concerns imperfect knowledge of a quantity. They should not be conflated. A probabilistic sensitivity analysis propagates parameter uncertainty through the model and reports distributions of costs, effects and decisions. A cost-effectiveness acceptability curve shows the probability that an intervention is cost-effective across willingness-to-pay thresholds, but it is not a substitute for clinical judgment.

One-way sensitivity analysis varies one input at a time. Scenario analysis changes a coherent group of assumptions, such as a short versus long treatment effect. Threshold analysis identifies the value at which the preferred decision changes. Value-of-information analysis estimates whether collecting additional evidence could improve decisions. Expected value of perfect information is conceptually:

where a denotes an action, θ uncertain parameters and NB net benefit. The expression is nonnegative under consistent definitions and indicates the upper bound on the value of eliminating parameter uncertainty.

7. Applied Case: Reducing Thirty-Day Readmissions

The following is an illustrative case, constructed for educational purposes.

Background and problem

A 300-bed hospital observes a 30-day readmission rate of 14% among patients discharged after heart-failure admissions. Leaders are considering a nurse-led transition program involving medication reconciliation, telephone follow-up and rapid outpatient review. The program costs approximately $180 per discharged patient. The hospital’s initial proposal is based on a simple comparison: readmission is 11% among patients who voluntarily received an early follow-up call and 16% among those who did not.

Analytical concerns

Patients accepting follow-up may be more motivated, have better transportation or be clinically different. The crude difference of 5 percentage points may therefore be confounded. Analysts define the cohort, index discharge, readmission outcome, program exposure and follow-up period in advance. They examine missing contact information, competing death events and transfers to other hospitals.

A risk-adjusted logistic model includes age, prior admissions, renal disease, baseline severity, insurance category and discharge destination. Because patients are clustered within clinical teams, uncertainty is calculated with team-level clustering. A propensity-score weighted analysis and a negative-control outcome are used as robustness checks. The team also considers a stepped-wedge implementation, in which units adopt the program at different randomly assigned times, enabling a stronger evaluation than voluntary uptake.

Decision model

A short-term decision tree represents program participation, readmission and death. Readmission costs, program costs and patient-reported outcomes are included. For longer-term consequences, a Markov model uses states of alive without readmission, alive after readmission and dead, with monthly cycles for one year. Transition probabilities are estimated from the evaluation dataset, while uncertain parameters are sampled in a probabilistic analysis.

Illustrative outcome

Suppose the adjusted analysis estimates a 2.5-percentage-point reduction in readmissions, with a 95% confidence interval from 0.5 to 4.5 percentage points. The result is potentially important, but it does not prove that every hospital or patient will experience the same effect. The economic model estimates an average net saving of $65 per patient under the base case, but the probabilistic analysis finds savings in 68% of simulations. The leadership team therefore approves a staged implementation with predefined monitoring thresholds rather than immediate universal expansion.

Learning points

The case demonstrates that a crude association is not automatically an intervention effect. A credible decision requires a well-defined outcome, attention to competing events and missingness, adjustment based on a causal framework, uncertainty analysis and prospective monitoring. It also shows why statistical significance alone is insufficient: managers must consider effect size, affordability, operational feasibility, equity and patient experience.

8. Governance, Ethics and Responsible Analytics

Healthcare analytics involves sensitive information and power asymmetries. Data use should satisfy applicable privacy, security, research ethics and data-protection requirements. Governance should address lawful basis, purpose limitation, data minimization, access controls, retention, breach response and secondary use. De-identification reduces risk but does not guarantee anonymity, particularly when datasets are linkable.

Fairness requires examining whether data quality and model performance differ across age, sex, ethnicity, disability, geography, language, socioeconomic status and other relevant groups. A model can have similar overall accuracy but unequal false-negative rates. Fairness metrics may conflict, and no metric resolves the underlying normative question. Analysts should involve affected communities and report limitations transparently.

Artificial intelligence can support risk prediction, imaging, documentation and resource planning. However, predictive accuracy does not establish clinical usefulness. Model drift may arise when coding, disease prevalence, treatment pathways or patient populations change. External validation, calibration, prospective impact evaluation, human oversight and a clear escalation route are essential. Automation bias occurs when professionals accept algorithmic recommendations without adequate review.

9. Best Practices for Professional Use

• Begin with a decision, not with an available dataset.
• Define estimands explicitly: whose outcome, under which intervention, over what period and from which perspective?
• Pre-specify outcomes, subgroup analyses and primary models where feasible.
• Use causal diagrams and subject-matter expertise to guide adjustment.
• Report absolute effects as well as relative measures.
• Validate models internally and externally; assess calibration, discrimination and clinical utility.
• Separate training, validation and test data and prevent information leakage.
• Quantify both sampling uncertainty and parameter uncertainty.
• Test robustness to missing data, alternative specifications and plausible bias.
• Communicate assumptions, not merely results.
• Include clinicians, patients, managers, statisticians, informaticians and ethicists in governance.
• Monitor implementation outcomes and unintended consequences.

Challenges and Limitations

No method can recover information that was never measured. EHR and claims data often contain exposure misclassification, incomplete outcomes and changing coding practices. Regression can adjust only for measured variables under its assumptions. Forecasts fail during structural change. Decision trees can oversimplify recurrent disease, while Markov models can hide history dependence. Simulation results may appear precise despite uncertain or poorly calibrated inputs. Economic thresholds are policy judgments, not universal scientific constants.

Consequently, analytic quality depends on alignment between question, design, data and decision. Transparent assumptions and humility about generalizability are central features of advanced healthcare management.

Chapter Summary

Healthcare analytics progresses from describing events to predicting outcomes, estimating causal effects and selecting actions under uncertainty. Reliable analysis begins with clearly defined populations, outcomes, denominators and data-generating processes. Regression models quantify associations but require assumptions for causal interpretation. Confounding, selection, measurement error and missing data can bias even technically sophisticated analyses.

Outcomes research incorporates clinical, patient-reported, economic and equity outcomes. Decision trees support finite sequences of uncertain events; Markov models represent recurring transitions among health states; simulation propagates variability and parameter uncertainty. Forecasting supports operational planning but must be tested out of sample and accompanied by prediction intervals. Sensitivity analysis, probabilistic analysis and value-of-information methods show how conclusions depend on uncertain assumptions.

The responsible manager combines statistical evidence with clinical expertise, implementation feasibility, ethical governance and patient perspectives. No model replaces judgment; a model makes assumptions visible so that judgment can be disciplined and reviewed.

Key Terms

Confounding: Distortion of an exposure–outcome relationship by a common cause.

Causal estimand: Precisely defined quantity representing an intervention effect.

Confidence interval: Interval procedure expressing sampling uncertainty under specified assumptions.

Difference-in-differences: Design comparing changes over time between intervention and comparison groups.

Electronic health record: Digital longitudinal record of clinical care.

Exchangeability: Condition that treated and untreated groups are comparable conditional on measured variables.

Forecasting: Estimation of future observations using historical and contextual information.

Health utility: Preference-based value assigned to a health state.

Instrumental variable: Variable used to identify causal effects under strong validity assumptions.

Markov model: State-transition model based on current-state information.

Missing data: Values not recorded or unavailable for analysis.

Monte Carlo simulation: Repeated random sampling to propagate uncertainty through a model.

Odds ratio: Ratio of odds between exposure groups or per unit change in a model.

Outcomes research: Study of effects of healthcare on patient, clinical, economic and system outcomes.

Positivity: Presence of a nonzero probability of each relevant treatment option in each covariate-defined group.

Propensity score: Probability of receiving treatment conditional on observed covariates.

QALY: Quality-adjusted life-year combining health-related quality and time.

Sensitivity analysis: Examination of how results change under alternative inputs or assumptions.

Review Questions

Short Answer Questions

1. Why must a healthcare rate have a clearly specified denominator?
2. Distinguish an odds ratio from a risk ratio.
3. What is confounding by indication?
4. State the Markov property.
5. Why should forecasts be evaluated on data not used for model fitting?

Long Answer Questions

1. Explain how selection bias, measurement error and missing data can affect an outcomes study. Give one mitigation strategy for each.
2. Compare regression adjustment, propensity-score methods and instrumental-variable analysis, including their major assumptions.
3. Explain the structure and appropriate use of a healthcare cost-effectiveness model using QALYs.
4. Compare decision trees and Markov models in terms of time horizon, recurrence and memory.
5. Discuss ethical and governance requirements for implementing an artificial-intelligence risk-prediction model in a hospital.

Analytical / Application Questions

1. A hospital reports 24 readmissions among 300 discharges. Calculate the readmission proportion and explain why the result cannot be interpreted causally as evidence about a discharge intervention.
2. Design a difference-in-differences evaluation of a nurse-led transition program. Specify the treatment group, comparison group, pre- and post-periods, outcome and parallel-trends assessment.
3. Construct a sensitivity-analysis plan for a Markov model evaluating a chronic-disease intervention. Distinguish parameter uncertainty, structural uncertainty and patient-level variability.

Multiple Choice Questions with Answers

Question 1

Which measure requires person-time in its denominator?

A. Risk

B. Rate

C. Odds

D. Proportion

Correct Answer: B. Rate

Question 2

In logistic regression, exp(βⱼ) is generally interpreted as the:

A. Risk difference

B. Odds ratio for a one-unit increase in xⱼ

C. Mean outcome change in original units

D. Hazard rate

Correct Answer: B. Odds ratio for a one-unit increase in xⱼ

Question 3

Which assumption is central to a difference-in-differences design?

A. Perfect measurement of every covariate

B. Parallel trends absent the intervention

C. Equal sample sizes in both groups

D. No seasonal variation

Correct Answer: B. Parallel trends absent the intervention

Question 4

If 80 events occur among 2,000 admissions, the event proportion is:

A. 0.4%

B. 2%

C. 4%

D. 40%

Correct Answer: C. 4%

Question 5

Which condition means that each relevant subgroup has a nonzero probability of receiving each treatment option?

A. Consistency

B. Positivity

C. Reliability

D. Calibration

Correct Answer: B. Positivity

Question 6

In a Markov model, the next state is assumed to depend primarily on the:

A. Full treatment history

B. Current state

C. Study sample size

D. Confidence interval width

Correct Answer: B. Current state

Question 7

A moving-average forecast primarily smooths a time series by:

A. Using the average of recent observations

B. Ignoring all prior observations

C. Randomly assigning future values

D. Replacing outcomes with medians across hospitals

Correct Answer: A. Using the average of recent observations

Question 8

Which statement about a confidence interval is most accurate?

A. It proves that the parameter has a 95% probability of being inside the interval

B. It expresses sampling uncertainty under specified assumptions

C. It measures clinical importance directly

D. It eliminates confounding

Correct Answer: B. It expresses sampling uncertainty under specified assumptions

Question 9

For a screening strategy with a 0.10 probability of a $2,000 benefit and a $100 cost in every case, the expected net value is:

A. $10

B. $100

C. $110

D. $200

Correct Answer: C. $110

Question 10

What is a major purpose of probabilistic sensitivity analysis?

A. To conceal model assumptions

B. To propagate parameter uncertainty through the model

C. To guarantee a causal estimate

D. To replace external validation

Correct Answer: B. To propagate parameter uncertainty through the model