Decision Analysis, Analytics and Evidence

Introduction

Strategic leaders make decisions under conditions of uncertainty, limited resources, competing interests and incomplete information. Intuition, experience and professional judgement remain valuable, but they become more reliable when combined with explicit evidence and transparent analytical methods. Decision analysis provides a disciplined way to identify alternatives, compare consequences, make assumptions visible and communicate why a decision was reached.

This chapter examines practical tools for managerial decision making: decision trees, weighted criteria, cost–benefit analysis, break-even analysis, expected value, sensitivity analysis, dashboards and key performance indicators (KPIs). These methods do not remove uncertainty or guarantee a correct outcome. Their value lies in improving the quality, consistency and accountability of reasoning.

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain how evidence and analytical reasoning support strategic leadership.
2. Construct and interpret a basic decision tree.
3. Apply weighted-criteria analysis to compare alternatives.
4. Conduct cost–benefit and break-even analyses.
5. Calculate and interpret expected value.
6. Use sensitivity analysis to identify critical assumptions.
7. Design dashboards and KPIs that support managerial action.
8. Recognise limitations, ethical risks and governance requirements in evidence-based decisions.

1. Evidence and the Decision Process

A managerial decision can be represented as a sequence of connected activities:

1. Define the decision problem.
2. Establish objectives and constraints.
3. Identify feasible alternatives.
4. Gather relevant evidence.
5. Analyse consequences, costs and risks.
6. Consult affected stakeholders.
7. Decide and allocate responsibility.
8. Implement, monitor and learn.

A common error is to begin with a preferred solution and then search selectively for supporting evidence. This is confirmation bias. A stronger process defines the decision and evaluation criteria before reviewing results. Leaders should also distinguish facts, assumptions, forecasts, interpretations and value judgements.

Types of evidence

Evidence may include operational data, financial records, customer research, experiments, expert judgement, benchmarking and qualitative stakeholder accounts. Each type answers different questions. Transaction data may show what happened, but interviews may help explain why it happened. A forecast estimates what may happen, while an audit examines whether controls operated as intended.

Evidence should be assessed for relevance, reliability, timeliness, completeness and comparability. A large dataset is not automatically useful: it may contain systematic measurement error or omit important groups. Similarly, correlation does not establish causation. If sales and advertising rise together, this does not prove that advertising alone caused the sales increase.

2. Decision Trees

A decision tree represents choices, uncertain events and outcomes in chronological order. It is particularly useful when a decision involves stages, probabilities or the option to revise action after new information becomes available.

A decision node is usually shown as a square; a chance node as a circle; and an outcome as an endpoint. At a chance node, probabilities should be mutually exclusive and collectively exhaustive: they must not overlap and must sum to 1.00.

For an outcome with monetary payoff V_i and probability p_i, expected monetary value (EMV) is:

Here, p_i is the probability of outcome i, V_i is its payoff, and the summation covers all possible outcomes. Probabilities are dimensionless and the EMV has the same monetary unit as V_i. This calculation assumes that the probabilities are meaningful and that the decision maker is willing to use an average monetary value as a decision criterion.

Decision trees are solved from right to left, a process called rollback. At chance nodes, calculate the probability-weighted average. At decision nodes, select the alternative with the preferred value, subject to strategic, legal and ethical constraints.

Worked example: service expansion

A college is considering a new evening programme. It may launch immediately or conduct a pilot. The immediate launch costs £40,000 and produces a net payoff of £120,000 if demand is high, with probability 0.60, or £20,000 if demand is low, with probability 0.40.

The EMV of immediate launch is:

EMV = (0.60 × £120,000) + (0.40 × £20,000) = £80,000

A pilot costs £15,000. After the pilot, management estimates a 70% chance of high demand and a 30% chance of low demand. If the pilot indicates high demand, the programme is launched with an expected net payoff of £100,000; if it indicates low demand, management cancels and receives no further payoff. The pilot EMV is:

EMV = (0.70 × £100,000) + (0.30 × £0) − £15,000 = £55,000

On EMV alone, immediate launch is preferred (£80,000 versus £55,000). However, the pilot may reduce reputational risk, improve learning and protect students from a poor-quality programme. The tree therefore informs judgement; it does not replace it.

3. Weighted-Criteria Analysis

Many strategic decisions cannot be reduced to money. Weighted-criteria analysis compares alternatives against several objectives, such as cost, quality, implementation speed, risk and stakeholder acceptability.

The method is:

1. Define criteria that reflect the decision objectives.
2. Assign weights whose total is 1.00 or 100%.
3. Score each alternative using a consistent scale.
4. Multiply each score by its criterion weight.
5. Add the weighted scores.
6. Test whether the conclusion changes when weights or scores change.

The weighted score for alternative j is:

w_i is the weight of criterion i, and r_ij is the rating of alternative j on that criterion. If weights sum to 1 and ratings use a 0–100 scale, S_j also lies between 0 and 100.

Worked comparison

A department compares three software systems using four criteria.

System A scores: (0.35×80)+(0.25×90)+(0.20×70)+(0.20×75)=80.25.

System B scores: (0.35×90)+(0.25×60)+(0.20×80)+(0.20×85)=79.25.

System C scores: (0.35×75)+(0.25×85)+(0.20×60)+(0.20×70)=75.25.

System A ranks first, although System B has stronger functionality and support. The outcome depends on the chosen weights and the quality of the ratings. Ratings should be supported by evidence rather than created to justify a predetermined option. Criteria that represent minimum safety, legal or accessibility requirements should be treated as thresholds, not merely traded off against other scores.

4. Cost–Benefit Analysis

Cost–benefit analysis (CBA) compares the expected benefits and costs of an option, usually in monetary terms. It is useful for investment, policy and programme decisions, but monetisation can be difficult for benefits such as wellbeing, trust, inclusion or environmental quality.

The basic net benefit is:

A benefit–cost ratio is:

A BCR greater than 1 indicates that benefits exceed costs under the stated assumptions. The ratio must use comparable units and the same time period.

For multi-year decisions, future amounts may be discounted because money available today can be invested and because future outcomes are uncertain. Present value is:

X_t is the amount received or paid in period t, r is the discount rate per period, and PV_t is its present value. The net present value (NPV) is:

I₀ is the initial investment. This formulation assumes that the discount rate is appropriate, cash flows are estimated consistently and the timing of flows is known. A positive NPV means the discounted benefits exceed the discounted costs, but non-financial impacts still require explicit consideration.

5. Break-Even Analysis

Break-even analysis identifies the activity level at which total revenue equals total cost. It is useful for capacity planning, pricing and assessing commercial risk.

With constant unit price and variable cost:

Q_BE is break-even quantity, F is fixed cost, P is selling price per unit, and V is variable cost per unit. The denominator is the contribution per unit. The formula assumes a constant price, constant unit variable cost and a single product or a constant sales mix.

Worked example

A training unit has fixed costs of £60,000. It charges £500 per participant and incurs variable cost of £200 per participant.

Q_BE = £60,000 ÷ (£500 − £200) = 200 participants.

At 200 participants, contribution equals fixed cost. If the unit expects 260 participants, its operating surplus before other costs is:

(260 × £300) − £60,000 = £18,000.

Break-even is not a forecast. It is a threshold. Demand uncertainty, quality effects from expanding capacity and cash-flow timing must also be examined.

6. Sensitivity Analysis

Sensitivity analysis tests how a result changes when an assumption changes. It helps leaders identify which variables deserve further research and where contingency plans are needed.

Suppose an investment has net benefit:

NB = (Q × (P − V)) − F.

If price, volume, variable cost or fixed cost changes, the result changes. A one-way sensitivity analysis varies one assumption while holding others constant. A scenario analysis changes several assumptions together, for example a favourable, central and adverse scenario. A break-even sensitivity asks how far an assumption can move before the decision changes.

A useful measure is the percentage change in output divided by the percentage change in an input:

This is a comparative indicator, not a universal causal law. It is most useful when the tested changes are plausible and the underlying model remains valid.

Sensitivity analysis should include probability uncertainty where possible. It should also expose model risk: the possibility that the structure of the model, not merely its numerical inputs, is wrong. Leaders should ask whether omitted variables, threshold effects or behavioural responses could alter the conclusion.

7. Dashboards and Key Performance Indicators

A dashboard is a visual management interface that presents selected information for monitoring and action. A KPI is a measure linked to an important objective. Not every metric is a KPI. A KPI should help answer a management question and support a decision.

Good KPI design follows a chain:

Strategic objective → critical success factor → KPI → target → owner → action.

For example:

A rate should identify its numerator, denominator, population and period. For example:

Dashboards should distinguish leading indicators from lagging indicators. A lagging indicator, such as annual profit, reports an outcome. A leading indicator, such as response time or training completion, may signal future performance. A balanced dashboard commonly includes financial, customer or stakeholder, internal-process, people and risk measures.

Dashboard design principles

Use a small number of consequential measures. Display trends rather than isolated numbers. Show actual performance against target, with clear definitions and update dates. Permit appropriate drill-down while protecting confidential data. Use colour carefully: red should signal an agreed exception, not merely a low number. Every red indicator should have an owner and a management response.

A dashboard can create dysfunctional behaviour if people optimise the measure rather than the underlying objective. For example, a call centre that rewards short calls may reduce service quality. Combining speed, resolution quality and customer outcomes reduces this risk.

8. Integrated Applied Case: Regional Health Appointment System

Background

The fictional Northvale Health Partnership must replace an outdated appointment system. The existing system causes long waiting times, duplicated records and high administrative workload. Three options are under consideration: upgrade the current system, purchase a cloud system or develop a bespoke platform.

Problem

The board must choose an option within a £1.2 million budget while protecting patient confidentiality, maintaining service continuity and improving access for people with disabilities.

Analysis

The team first defines mandatory criteria: legal compliance, accessibility and secure data migration. Options failing any mandatory criterion are rejected. The remaining options are assessed using weighted criteria: patient benefit 35%, total cost 25%, implementation risk 20% and staff usability 20%.

The cloud system receives the highest weighted score. However, CBA identifies annual benefits of £420,000 and annual operating and implementation costs equivalent to £350,000 in the first year. Its first-year net benefit is therefore £70,000, but benefits are uncertain because adoption may be slower than expected. Break-even analysis shows that the system needs 14,000 successfully completed digital appointments annually to cover relevant fixed and variable costs.

A decision tree is then constructed. Under high adoption, probability 0.65, the estimated five-year NPV is £900,000. Under low adoption, probability 0.35, NPV is £100,000. The expected NPV is:

(0.65 × £900,000) + (0.35 × £100,000) = £620,000.

Sensitivity analysis shows that the result is most affected by adoption and migration cost. The board therefore approves the cloud option conditionally, requiring a staged pilot, an independent security review, accessibility testing and a migration contingency budget.

Outcome and learning points

After six months, the pilot shows improved appointment completion but lower adoption among some older patients. The partnership retains telephone booking and funds digital support rather than treating digital uptake as the only success measure. Its dashboard includes appointment completion, waiting time, failed migration records, accessibility complaints, staff workload and security incidents.

The case illustrates that a high analytical score is not sufficient. Decision quality depends on valid criteria, stakeholder participation, safeguards, implementation capacity and monitoring after the decision.

9. Ethical, Governance and Technology Considerations

Evidence-based leadership has ethical responsibilities. Leaders should disclose material assumptions, conflicts of interest, data limitations and uncertainty. They should avoid presenting estimates as facts and should explain who bears costs and who receives benefits.

Data governance requires lawful collection, purpose limitation, access controls, retention rules, quality assurance and auditability. Sensitive personal data should be minimised and protected. KPI systems can unintentionally discriminate when historic data reflect unequal treatment. An apparently neutral algorithm may reproduce that bias.

Artificial intelligence can support forecasting, anomaly detection and scenario analysis, but managers remain accountable for decisions. AI-generated recommendations should be tested for accuracy, bias, explainability, privacy and robustness under changed conditions. Human review is especially important for employment, education, health, financial access and other high-impact decisions.

A sound governance process records the decision owner, evidence used, alternatives considered, assumptions, approval limits, implementation controls and review date. Post-decision review should ask not only whether the outcome was good, but whether the reasoning was reasonable given the information available at the time. This avoids unfair hindsight bias.

10. Best Practices and Common Failures

Best practices include:

• Start with a clear decision question and explicit objectives.
• Separate mandatory requirements from desirable preferences.
• Use multiple evidence sources and document data quality.
• Make assumptions, probabilities and uncertainty visible.
• Use the simplest model that adequately represents the decision.
• Test results with sensitivity and scenario analysis.
• Include affected stakeholders, especially under-represented groups.
• Assign KPI owners and predefine management responses.
• Review outcomes and update models from experience.

Common failures include false precision, double-counting benefits, ignoring opportunity cost, using arbitrary weights, confusing activity with impact, measuring what is easy rather than what matters and treating EMV as an instruction rather than an aid to judgement. Analytical tools improve transparency only when leaders are willing to question their own assumptions.

Chapter Summary

Decision analysis strengthens strategic leadership by making alternatives, assumptions, consequences and trade-offs explicit. Decision trees organise sequential choices and uncertainty; expected value provides a probability-weighted comparison when its assumptions are appropriate. Weighted-criteria analysis supports multi-objective decisions, while cost–benefit and break-even analyses clarify financial feasibility and thresholds. Sensitivity analysis identifies assumptions that can change a recommendation. Dashboards and KPIs connect strategy to monitoring and action, but they must be carefully defined to avoid dysfunctional behaviour. Ethical evidence-based leadership requires data governance, stakeholder inclusion, transparency, proportionality and continuing review. No analytical method replaces judgement; rather, good analysis makes judgement more disciplined and accountable.

Key Terms

Decision analysis: A structured process for comparing alternatives under objectives, constraints and uncertainty.

Evidence: Information used to support, challenge or evaluate a decision.

Decision tree: A diagram showing decisions, chance events and possible outcomes over time.

Decision node: A point at which a decision maker selects among alternatives.

Chance node: A point at which uncertain outcomes occur.

Expected monetary value: The probability-weighted average of possible monetary outcomes.

Weighted-criteria analysis: A method that combines criterion ratings using assigned weights.

Criterion: A dimension used to evaluate an alternative.

Cost–benefit analysis: Comparison of expected benefits and costs, commonly in monetary terms.

Net benefit: Total benefits minus total costs.

Benefit–cost ratio: Benefits divided by costs.

Present value: The current value of a future amount after discounting.

Net present value: Discounted benefits minus discounted costs and investment.

Break-even point: The activity level at which total revenue equals total cost.

Contribution per unit: Price per unit minus variable cost per unit.

Sensitivity analysis: Testing how results change when assumptions change.

Scenario analysis: Examination of outcomes under coherent alternative conditions.

Dashboard: A visual display of selected performance information.

Key performance indicator: A strategically important measure linked to a target and action.

Model risk: The possibility that a model is inappropriate, incomplete or incorrectly specified.

Review Questions

Short Answer Questions

1. What is the purpose of a decision tree?
2. Why must probabilities at a chance node be mutually exclusive and collectively exhaustive?
3. State the break-even quantity formula and define its variables.
4. What distinguishes a KPI from an ordinary operational metric?
5. Give two limitations of cost–benefit analysis.

Long Answer Questions

1. Explain how a leader could combine weighted-criteria analysis with cost–benefit analysis when selecting a major technology system.
2. Describe the process of solving a decision tree by rollback, including the role of expected value.
3. Explain why sensitivity analysis is important even when a decision has a positive NPV.
4. Discuss how dashboard design can create unintended or dysfunctional behaviour.
5. Evaluate the ethical responsibilities of managers using AI-supported evidence in high-impact decisions.

Analytical / Application Questions

1. A project has fixed costs of £90,000, a price of £750 per unit and variable cost of £450 per unit. Calculate the break-even quantity and explain two assumptions behind your calculation.
2. An investment has a 0.4 probability of producing £300,000 and a 0.6 probability of producing £80,000. Calculate its expected monetary value and identify one non-financial factor that should also be considered.
3. Design a small dashboard for a university programme seeking to improve retention. Specify four KPIs, their definitions, targets and the management action associated with an adverse result.

Multiple Choice Questions with Answers

Question 1

Which condition is required for probabilities at a chance node in a decision tree?

A. They must all be equal

B. They must be mutually exclusive and sum to 1

C. They must be monetary values

D. They must describe only favourable outcomes

Correct Answer: B. They must be mutually exclusive and sum to 1

Question 2

A project has a 0.5 probability of a £100,000 payoff and a 0.5 probability of a £20,000 payoff. What is its expected monetary value?

A. £40,000

B. £50,000

C. £60,000

D. £120,000

Correct Answer: C. £60,000

Question 3

In weighted-criteria analysis, what should the weights normally represent?

A. The number of alternatives

B. The relative importance of the criteria

C. The monetary profit of each option

D. The probability of failure

Correct Answer: B. The relative importance of the criteria

Question 4

A product has fixed costs of £50,000, a price of £100 and variable cost of £60. What is the break-even quantity?

A. 500 units

B. 800 units

C. 1,250 units

D. 5,000 units

Correct Answer: D. 5,000 units

Question 5

What does a positive NPV generally indicate, assuming the model and discount rate are appropriate?

A. Discounted benefits exceed discounted costs

B. The project has no risk

C. All stakeholders will benefit equally

D. The project will definitely succeed

Correct Answer: A. Discounted benefits exceed discounted costs

Question 6

What is the main purpose of sensitivity analysis?

A. To remove all uncertainty

B. To identify how changes in assumptions affect results

C. To replace stakeholder consultation

D. To guarantee a positive outcome

Correct Answer: B. To identify how changes in assumptions affect results

Question 7

Which is most clearly a leading indicator?

A. Annual profit after the year ends

B. Last year’s customer complaints

C. Training completion before a system launch

D. Final market share after five years

Correct Answer: C. Training completion before a system launch

Question 8

Which practice improves KPI quality?

A. Using measures without definitions

B. Changing targets whenever performance is poor

C. Assigning an owner and linking the KPI to an action

D. Displaying every available metric

Correct Answer: C. Assigning an owner and linking the KPI to an action

Question 9

What is a major ethical risk in using historical data for automated decisions?

A. Data always become more accurate over time

B. Historical bias may be reproduced by the system

C. Automation eliminates the need for governance

D. The system cannot process numerical information

Correct Answer: B. Historical bias may be reproduced by the system

Question 10

A benefit–cost ratio greater than 1 means, under the stated assumptions, that:

A. Costs exceed benefits

B. Benefits equal zero

C. Benefits exceed costs

D. The decision has no uncertainty

Correct Answer: C. Benefits exceed costs