Evidence-Based Decisions and Stakeholder Communication

Introduction

User research creates value only when evidence changes what an organisation decides to do. A repository of interview quotations, usability findings, survey results, and analytics is not itself a design outcome. The professional task is to interpret evidence responsibly, identify the most consequential opportunities, recommend action, and communicate the reasoning clearly to people who control resources, priorities, risk, and implementation.

Evidence-based design decisions connect four activities:

1. Evidence collection: gathering observations, behavioural data, reported experiences, and contextual information.
2. Evidence interpretation: identifying patterns while recognising uncertainty, bias, and limitations.
3. Decision translation: converting findings into prioritised recommendations, experiments, and roadmap commitments.
4. Stakeholder communication: presenting an understandable, credible account of what should happen next and why.

This chapter focuses on the strategic stage between research and delivery. It treats research communication not as decoration or advocacy, but as a form of accountable reasoning. A strong recommendation makes its evidence visible, explains its importance, acknowledges constraints, and specifies how success will be measured.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish raw data, findings, insights, implications, and recommendations;
• assess the strength and limitations of mixed research evidence;
• prioritise opportunities using transparent criteria;
• convert recommendations into experiments, product-roadmap items, and measurable outcomes;
• communicate research through reports, presentations, and evidence-based stories;
• negotiate stakeholder constraints without distorting user evidence;
• apply ethical, inclusive, and governance principles to research communication.

1. From Research Data to Design Decisions

Research outputs have different levels of abstraction. Confusing them produces weak decisions.

A useful research statement links these levels: evidence → interpretation → consequence → action → measurement. Each arrow requires judgement. Evidence does not mechanically dictate a single design solution. For example, abandonment during verification could result from confusing language, lack of trust, document availability, technical failure, or an inappropriate target audience. A responsible researcher separates what was observed from what is inferred.

1.1 Claims, evidence, and confidence

A research claim should answer three questions:

• What exactly was observed?
• How was it observed?
• How confidently can it be generalised to the relevant population or situation?

Confidence is not the same as sample size. A small usability study may provide strong evidence of a severe interaction failure, particularly when several participants independently reproduce it. Conversely, a large but poorly designed survey may provide weak evidence if its respondents are self-selected or its questions are leading.

A practical evidence record contains:

• the claim;
• supporting sources;
• the research method and sample;
• alternative explanations;
• affected user groups;
• severity and frequency indicators;
• confidence level;
• recommended next action.

Confidence should be expressed cautiously—for example, high, moderate, or low—with a justification. It is better to state “moderate confidence based on converging interviews and analytics, but limited evidence from non-English users” than to imply false precision.

1.2 Triangulation

Triangulation means examining a question through more than one source, method, researcher, or theoretical perspective. Common forms include:

• Method triangulation: combining interviews, usability testing, and analytics.
• Source triangulation: comparing new users, experienced users, support staff, and operational records.
• Researcher triangulation: having more than one analyst review the evidence.
• Time triangulation: examining whether a pattern persists across periods or releases.

Triangulation does not mean that all sources must agree. Disagreement can reveal context. For instance, interviews may show that users value a feature, while analytics show low usage. This could indicate that the feature is hard to discover, that stated preference differs from behaviour, or that the feature matters only in exceptional circumstances. The discrepancy becomes a question for further investigation rather than a reason to select whichever source is more convenient.

2. Turning Findings into Priorities

A product team normally has more valid opportunities than it can address immediately. Prioritisation makes trade-offs explicit. It should not be presented as a purely objective calculation: the criteria, weights, and evidence quality all contain judgement. Its value lies in transparency and repeatability.

2.1 Priority dimensions

Useful dimensions include:

• User impact: the extent to which the issue affects task success, comprehension, safety, access, or satisfaction.
• Reach: the proportion or number of relevant users or occasions affected.
• Strategic alignment: connection with organisational goals or product direction.
• Evidence confidence: strength and convergence of supporting evidence.
• Urgency: consequences of delaying action, including legal, safety, reputational, or contractual risk.
• Effort and feasibility: estimated design, engineering, operational, and change-management requirements.
• Equity impact: whether the issue disproportionately harms people with disabilities, low digital confidence, limited language proficiency, or other forms of marginalisation.

A simple qualitative matrix may classify each criterion as low, medium, or high. This is appropriate when evidence is early or estimates are uncertain. A weighted scoring model can be useful when a team needs a more explicit comparison.

2.2 Weighted prioritisation

Suppose an organisation scores an opportunity on five criteria, each from 1 to 5. A weighted score is:

Here, P is the priority score, wᵢ is the weight assigned to criterion i, and sᵢ is the opportunity's score on that criterion. The weights are dimensionless and should sum to 1. The model assumes that scores are sufficiently comparable for a weighted sum to be meaningful and that the team has agreed on the direction of each scale.

For example, consider an onboarding problem scored as follows:

The total is P = 1.50 + 0.80 + 0.60 + 0.45 + 0.40 = 3.75. This score does not prove that the opportunity must be selected. It provides a structured basis for discussion. A low-feasibility item may still be necessary if it concerns safety or legal compliance. Similarly, a high score based on weak evidence may justify discovery work rather than full implementation.

A common mistake is to treat the score as more authoritative than its assumptions. Teams should record who assigned scores, what each scale means, and which estimates are uncertain. Sensitivity analysis—checking whether the ranking changes when a weight is adjusted—can reveal fragile decisions.

2.3 Impact–effort and risk

An impact–effort map is useful for conversation but should not be the sole prioritisation method. “Impact” must be tied to a defined user or business outcome, while “effort” should include dependencies, operational change, accessibility work, content, research, and maintenance. The map can be enhanced by identifying:

• quick experiments that reduce uncertainty;
• foundational work needed before visible improvements;
• high-risk items requiring legal, security, or accessibility review;
• opportunities whose value is concentrated in a vulnerable user group.

Prioritisation should distinguish importance from confidence. A potentially severe problem with low confidence may deserve rapid investigation, while a minor problem with high confidence may wait. This distinction prevents teams from ignoring unknowns simply because they are not yet quantified.

3. Recommendations That Can Be Acted Upon

A recommendation is stronger when it names the problem, affected users, proposed response, rationale, constraints, and validation plan. The following structure is practical:

Because [evidence and user need], we recommend [specific action] for [target context or audience], so that [intended outcome]. We will validate this by [measure or experiment], while monitoring [risk or unintended effect].

For example:

Because first-time applicants misinterpret the document requirement and abandon verification, we recommend displaying an illustrated checklist before upload, so that applicants understand what to prepare. We will validate this through task success and abandonment rates, while monitoring whether the checklist increases page length or creates confusion for returning applicants.

This format avoids the weak recommendation “improve verification.” It also prevents a solution from being presented without a reason or evaluation method.

3.1 Recommendation quality criteria

A professional recommendation is:

• traceable: linked to evidence and a research question;
• specific: describes a decision or action rather than a general aspiration;
• feasible: recognises technical, operational, and organisational conditions;
• inclusive: considers affected groups and accessibility;
• testable: identifies how the team will learn whether it helped;
• proportionate: does not claim more certainty than the evidence supports;
• owned: identifies the decision-maker or team responsible for next action.

Recommendations should be separated into three categories when useful:

1. Now: actions supported by sufficient evidence and feasible within current work.
2. Next: actions needing design exploration, technical discovery, or additional validation.
3. Later or watch: lower-confidence or lower-impact opportunities that should be monitored.

4. From Recommendations to Product Roadmaps

A roadmap is a communication and coordination instrument, not merely a list of feature dates. Research should influence the roadmap through outcomes and learning goals rather than through unexamined feature requests.

4.1 Features, outcomes, and outputs

An output is what the team produces, such as a new form or help article. An outcome is the change in user or organisational condition, such as improved completion or reduced support contact. A feature may be delivered without producing its intended outcome; therefore, roadmaps should record both the planned output and the expected outcome.

A concise roadmap item includes:

• user problem or opportunity;
• target users and context;
• desired outcome;
• evidence and confidence;
• proposed discovery or delivery activity;
• dependencies and constraints;
• owner;
• success and guardrail measures;
• review date.

4.2 Discovery and delivery horizons

A useful roadmap distinguishes different kinds of work:

• Explore: investigate an uncertain problem through research or prototyping.
• Validate: test a proposed interaction, content approach, or service change.
• Deliver: build and release a solution with appropriate quality checks.
• Measure and learn: examine outcomes after release and decide whether to adapt.

This structure resists the assumption that every research finding should immediately become a feature. Sometimes the correct roadmap response is a prototype, a content audit, a policy discussion, instrumentation, or a decision not to proceed.

4.3 Dependencies and constraints

Stakeholders may impose constraints involving budget, staffing, architecture, procurement, brand, regulation, security, data protection, or launch dates. Constraints are not automatically unreasonable, but they should be made visible. A useful decision log records:

• the decision;
• alternatives considered;
• evidence used;
• constraints;
• risks accepted;
• decision owner;
• date and review condition.

When a preferred recommendation cannot be implemented, the researcher should explain the consequence rather than silently weakening the user need. For example: “The full redesign is deferred because of platform migration. As an interim measure, we recommend clearer error content and support guidance; this reduces confusion but is unlikely to address the underlying navigation problem.”

5. Measuring Outcomes Responsibly

Measurement connects design work to learning. A balanced measurement plan commonly includes a primary outcome, supporting measures, and guardrails.

5.1 Metric types

• Behavioural metrics: completion rate, abandonment, time on task, error frequency, repeat attempts.
• Perceptual metrics: confidence, perceived ease, trust, satisfaction, or clarity.
• Operational metrics: support contacts, processing time, rework, escalation, or cost.
• Equity metrics: differences in outcomes across relevant user groups, where collection is ethical and lawful.
• Guardrail metrics: measures intended to detect harm, degradation, or unintended incentives.

A completion rate can be defined as:

The denominator must be defined carefully. “Eligible attempts” might mean users who began the task, reached a particular step, or met a specified condition. Changing the denominator can make trends appear better or worse without any real behavioural change.

For a proportion measured before and after a change, the absolute percentage-point change is:

where p₀ is the baseline proportion and p₁ is the post-change proportion. If completion increases from 40% to 50%, ∆p = 0.50 − 0.40 = 0.10, or 10 percentage points. Relative change would be 0.10 ÷ 0.40 = 0.25, or 25%. These are different statements and should not be confused.

Metrics alone do not establish causality. A rise in completion may result from seasonality, marketing, a policy change, or a different user mix. Stronger causal claims require appropriate comparison designs, such as controlled experiments or carefully justified quasi-experimental methods. At certificate level, the key practice is to avoid claiming that a design caused an outcome merely because the outcome changed after release.

5.2 Measurement plans

A measurement plan should specify:

Avoid “metric theatre”: collecting dashboards that no one uses to make decisions. Every important metric should have an owner and a consequence attached to its interpretation.

6. Communicating Evidence to Stakeholders

Stakeholders differ in authority, expertise, incentives, and tolerance for uncertainty. Executives may need implications and choices; engineers may need constraints and acceptance conditions; support teams may need operational guidance; legal and compliance colleagues may need traceability; designers may need behavioural detail. The evidence should remain consistent, but the emphasis and format can change.

6.1 The research report

A report supports careful reading and future reference. A compact structure is:

1. executive summary;
2. decision required;
3. research purpose and questions;
4. methods, participants, and limitations;
5. findings organised by theme or journey stage;
6. evidence excerpts, behavioural examples, or data displays;
7. implications;
8. prioritised recommendations;
9. measurement and validation plan;
10. appendices containing detail, instruments, and traceability.

The executive summary should state what matters, why it matters, what is recommended, and what decision is needed. It should not hide important limitations in an appendix.

6.2 The presentation

A presentation is a guided decision experience. A strong sequence is:

• establish the decision context;
• introduce the users and situation;
• show the most consequential evidence;
• explain the interpretation;
• present options and trade-offs;
• recommend an action;
• define measures, owners, and next steps;
• invite a specific decision.

Slides should not be overloaded with transcripts or unexplained charts. One claim should be supported by visible evidence. Quotations should be contextualised: identify the relevant situation, avoid selective editing that changes meaning, and protect identity.

6.3 Evidence-based storytelling

Storytelling is not a replacement for rigour. It is a way to make evidence intelligible and memorable. A responsible research story often follows this pattern:

1. Context: who is trying to do what?
2. Tension: what obstacle or uncertainty appears?
3. Evidence: what did research show?
4. Meaning: why does the pattern matter?
5. Choice: what options exist?
6. Action: what is recommended?
7. Verification: how will the team know whether it worked?

A single participant quote can humanise a finding but cannot establish prevalence. Conversely, a percentage can establish scale but may not explain lived experience. Combining a carefully selected story with transparent aggregate evidence is usually stronger than either alone.

7. Managing Disagreement and Stakeholder Constraints

Stakeholder disagreement is normal. It may reflect different objectives rather than rejection of research. A product manager may prioritise growth, a service manager may prioritise operational capacity, and an accessibility specialist may prioritise legal and inclusive access. The researcher’s role is to make the conflict explicit and help the group reason about consequences.

Useful practices include:

• distinguish disagreement about evidence from disagreement about priorities;
• ask what decision would change if the evidence were stronger;
• identify assumptions and testable uncertainties;
• present multiple options with consequences;
• use decision criteria agreed before reviewing alternatives;
• document accepted risks and unresolved questions;
• avoid overstating findings to win an argument;
• invite domain experts and affected users into appropriate stages of discussion.

A stakeholder who asks for “positive findings only” may be expressing concern about morale, reputation, or political risk. The ethical response is to reframe the report around opportunities while retaining material negative evidence. Omitting inconvenient evidence undermines informed decision-making and can create greater organisational risk later.

8. Ethical and Governance Considerations

Evidence-based communication must protect people as well as improve products. Important responsibilities include:

• Privacy: remove or mask identifying details and avoid unnecessary personal data in reports.
• Consent: ensure participants understand how recordings, quotations, and images will be used.
• Confidentiality: limit access to sensitive research materials and follow retention policies.
• Accessibility: provide presentations and reports in formats usable by people with disabilities.
• Fair representation: do not portray a small or marginalised group as a problem to be fixed.
• Uncertainty disclosure: state sample limitations, missing groups, and plausible alternative explanations.
• Data governance: document provenance, access, storage, retention, and permitted uses.
• Responsible automation: review AI-generated summaries for fabrication, omission, stereotyping, and confidentiality breaches.

AI tools may assist with transcription, clustering, translation, or draft synthesis, but they do not replace researcher judgement. Sensitive transcripts should not be uploaded to an unapproved service. Any automated output should be checked against source material, and the use of automation should be documented when it affects interpretation or decisions.

9. Illustrative Case: Improving a Public-Service Application

This is a fictional illustrative case designed for learning purposes.

Background

A municipal service allows residents to apply online for a household energy-support grant. Completion has declined, and the service team receives frequent calls asking which documents are required.

Problem

Analytics show that many applicants leave during the evidence-upload stage. Support staff report that applicants often submit the wrong documents. The team initially proposes adding a prominent “Apply now” button, but this solution is not supported by the evidence.

Analysis

Researchers conduct eight moderated usability sessions with a mix of first-time and returning applicants, review 12 months of funnel data, interview four support advisers, and inspect the content with an accessibility specialist. The evidence converges on three issues:

• applicants cannot tell which documents are acceptable before beginning;
• error messages describe technical failure but not how to recover;
• users with limited English proficiency rely heavily on support advisers.

The evidence is strong for a comprehension problem in the tested flow, but less certain about its prevalence among all eligible residents because the qualitative sample is small and recruitment was online.

Approach and solution

The team prioritises a staged response:

1. Now: rewrite requirements in plain language, provide document examples, and improve error recovery content.
2. Next: prototype a pre-check tool that lets applicants confirm their documents before upload.
3. Measure: establish baseline completion, upload-error, and support-contact measures; monitor outcomes by language preference where lawful and appropriate.
4. Later: investigate whether policy or eligibility rules themselves create unnecessary burden.

The recommendation is not simply “make the form easier.” It specifies an immediate content intervention, a discovery activity, and a longer-term policy question.

Outcome

After the content change, the team observes a rise in successful uploads over the first four weeks and fewer support contacts mentioning document confusion. Because the period coincides with a seasonal campaign, the team reports the result as promising rather than conclusive. A later controlled comparison of the pre-check tool provides stronger evidence about its effect.

Learning points

The case demonstrates that:

• different sources answer different questions;
• prioritisation should account for confidence and feasibility;
• a roadmap can contain content, research, and policy work—not only features;
• measurement must define denominators and acknowledge confounding factors;
• communicating uncertainty increases credibility rather than weakening the recommendation.

10. Best-Practice Checklist

Before presenting a research-informed decision, check that:

• the decision required is explicit;
• observations are distinguished from interpretations;
• evidence sources and limitations are traceable;
• affected user groups are identified;
• alternatives and trade-offs are visible;
• priority criteria are transparent;
• recommendations specify owners and next actions;
• expected outcomes and guardrails are measurable;
• accessibility, privacy, and equity have been considered;
• stakeholder constraints are recorded rather than concealed;
• the communication format suits the audience;
• uncertainty and contradictory evidence are acknowledged;
• the team has a plan to learn after implementation.

Chapter Summary

Evidence-based design work transforms research data into accountable decisions. The essential chain is evidence, finding, insight, implication, recommendation, action, and outcome. Triangulation strengthens interpretation, but disagreement among sources can reveal important contextual differences. Prioritisation should consider impact, reach, alignment, confidence, urgency, feasibility, and equity rather than relying on a feature’s popularity.

Weighted scoring can make assumptions visible, but it does not turn judgement into objective truth. Recommendations should be specific, traceable, feasible, inclusive, testable, and assigned to an owner. Roadmaps are stronger when they express outcomes and distinguish exploration, validation, delivery, and learning. Measurement plans require clear denominators, baselines, populations, time windows, and guardrails; a change after release is not automatically proof of causation.

Reports support detailed traceability, presentations support collective decisions, and storytelling makes evidence meaningful without replacing methodological rigour. Stakeholder constraints and disagreement should be surfaced and documented. Finally, privacy, consent, accessibility, fairness, uncertainty, and responsible use of AI are central requirements of professional research communication.

Key Terms

Evidence: Information used to support or challenge a research claim.

Data: Recorded observations, responses, measurements, or artefacts.

Finding: A recurring or significant pattern identified in data.

Insight: An interpretation that explains why a finding matters.

Implication: The consequence of an insight for design, service, or strategy.

Recommendation: A proposed action justified by evidence and reasoning.

Triangulation: Examination of a question through multiple sources, methods, researchers, or time periods.

Confidence: The justified degree of trust in a research claim.

Prioritisation: The process of ordering opportunities or actions according to agreed criteria.

Weighted score: A combined score produced by multiplying criterion weights by criterion ratings and summing the results.

Roadmap: A coordinated view of planned exploration, delivery, outcomes, dependencies, and learning.

Output: A product or activity produced by a team.

Outcome: A meaningful change resulting, or expected to result, from an intervention.

Guardrail metric: A measure used to detect harm or undesirable trade-offs.

Baseline: The condition measured before a change is introduced.

Denominator: The defined total against which a rate or proportion is calculated.

Decision log: A record of decisions, evidence, constraints, owners, and review conditions.

Stakeholder: A person or group affected by, responsible for, or able to influence a decision.

Research governance: Policies and practices controlling ethical, secure, lawful, and accountable research use.

Review Questions

Short Answer Questions

1. Distinguish a finding from an insight and give an example of each.
2. What is triangulation, and why can disagreement between sources be useful?
3. Name four dimensions that may be used to prioritise a design opportunity.
4. Why should a roadmap distinguish outputs from outcomes?
5. What information should be included in a decision log?

Long Answer Questions

1. Explain how raw research data can be transformed into a defensible design recommendation. Include the role of uncertainty.
2. Describe how a weighted prioritisation model works, including its assumptions and limitations.
3. Compare the purposes of a research report, a stakeholder presentation, and an evidence-based story.
4. Explain why measurement plans should include baseline measures, denominators, and guardrail metrics.
5. Discuss the ethical responsibilities involved in communicating participant quotations and sensitive research data.

Analytical / Application Questions

1. A team has strong interview evidence that users distrust a payment flow, but analytics show high completion. What explanations could account for this pattern, and what research or measurement would you propose next?
2. A high-impact accessibility problem is expensive to fix and has moderate evidence confidence. How should it be represented in the roadmap and prioritisation discussion?
3. Design a recommendation and measurement plan for a university enrolment form with a high abandonment rate. State the evidence needed, proposed action, primary outcome, and at least one guardrail metric.

Multiple Choice Questions with Answers

Question 1

Which sequence best represents the translation of research into action?

A. Recommendation → data → insight → outcome

B. Evidence → finding → insight → implication → recommendation

C. Outcome → stakeholder opinion → method → data

D. Roadmap → quotation → sample → strategy

Correct Answer: B. Evidence → finding → insight → implication → recommendation

Question 2

In a weighted prioritisation model, what does the condition ∑wᵢ = 1 mean?

A. All opportunity scores must equal 1

B. All criteria must have identical ratings

C. The criterion weights sum to one

D. The model can contain only one criterion

Correct Answer: C. The criterion weights sum to one

Question 3

An increase from 40% completion to 50% completion represents what absolute change?

A. 5 percentage points

B. 10 percentage points

C. 25 percentage points

D. 40 percentage points

Correct Answer: B. 10 percentage points

Question 4

Which is the best example of a guardrail metric?

A. The number of design meetings held

B. The number of prototype screens created

C. A measure detecting whether support contacts increase after launch

D. The number of stakeholders invited to a workshop

Correct Answer: C. A measure detecting whether support contacts increase after launch

Question 5

What is the principal purpose of triangulation?

A. To guarantee that every research source agrees

B. To replace qualitative research with analytics

C. To examine a question through multiple sources or methods

D. To increase the sample size automatically

Correct Answer: C. To examine a question through multiple sources or methods

Question 6

Which statement best distinguishes an output from an outcome?

A. An output is always quantitative, while an outcome is always qualitative

B. An output is what a team produces, while an outcome is the change it seeks to create

C. An output is a research limitation, while an outcome is a stakeholder constraint

D. An output occurs after measurement, while an outcome occurs before research

Correct Answer: B. An output is what a team produces, while an outcome is the change it seeks to create

Question 7

What should a responsible researcher do when a stakeholder requests that negative findings be omitted?

A. Delete them to preserve stakeholder confidence

B. Present them as positive findings without qualification

C. Retain material evidence and frame it around opportunities and risks

D. Replace them with data from another project

Correct Answer: C. Retain material evidence and frame it around opportunities and risks

Question 8

Which statement about a post-release metric change is most accurate?

A. It always proves that the design caused the change

B. It is irrelevant unless the sample is a census

C. It may be promising evidence, but other explanations should be considered

D. It should never be used in a roadmap review

Correct Answer: C. It may be promising evidence, but other explanations should be considered

Question 9

Which roadmap activity is most appropriate when the problem is important but the solution is uncertain?

A. Immediate full-scale implementation without testing

B. Exploration or validation before committing to delivery

C. Permanent removal from the roadmap

D. Changing the metric until the problem disappears

Correct Answer: B. Exploration or validation before committing to delivery

Question 10

Why is a single participant quotation insufficient to establish prevalence?

A. Quotations cannot describe user experience

B. One quotation provides an individual perspective but not evidence of how widespread a pattern is

C. Participants are never allowed to speak in reports

D. Qualitative evidence is always less valid than quantitative evidence

Correct Answer: B. One quotation provides an individual perspective but not evidence of how widespread a pattern is