Research Strategy and Study Planning

Introduction

User research is most valuable when it connects evidence to a decision. A study should not begin simply because a team is curious about users or because a research method is fashionable. It should begin with a meaningful product, service, or business decision; a clearly defined uncertainty; and a practical plan for collecting and interpreting evidence.

Research strategy is the process of translating assumptions and organizational goals into answerable research questions, selecting suitable methods and participants, scheduling the work, and defining how findings will influence design. Study planning turns that strategy into an operational document that specifies what will happen, when it will happen, who will participate, how data will be recorded, and what deliverables will be produced.

In user experience practice, research is commonly divided into two broad purposes. Generative research explores users’ lives, needs, motivations, behaviours, and problems in order to generate opportunities. Evaluative research examines an existing or proposed design to determine whether it is understandable, usable, useful, desirable, or accessible. The distinction is useful, although real projects may combine both purposes.

Learning Objectives

By the end of this chapter, you should be able to:

• connect business goals, assumptions, decisions, and research questions;
• distinguish research questions, hypotheses, and interview questions;
• select appropriate generative and evaluative methods;
• define participant criteria and choose a defensible sampling approach;
• prepare a practical research plan, timeline, and set of deliverables;
• identify ethical, accessibility, privacy, and bias-related risks; and
• explain how research evidence should influence product decisions.

From Business Goals to Research Questions

A business goal describes a desired organizational outcome, such as increasing subscription conversion, reducing customer-support costs, or improving retention. A research question asks what the team needs to learn in order to make a decision related to that goal. These are not interchangeable.

For example:

• Business goal: Increase completion of an online account-opening process.
• Assumption: People abandon the process because identity verification is confusing.
• Decision: Decide whether to redesign identity verification and, if so, what to change first.
• Research question: How do prospective customers understand and complete identity verification, and where do they experience uncertainty or loss of confidence?

A strong research question is focused, answerable through evidence, neutral in wording, and connected to a decision. “Do users like our new verification screen?” is weak because it presupposes that liking is the relevant outcome and does not specify what decision will follow. “Can first-time applicants understand what documents are required and complete verification without assistance?” is more useful because it identifies a population, behaviour, and observable outcome.

The research-chain model

A practical chain is:

Goal → assumption → decision → research question → method → evidence → insight → action

The chain prevents two common errors. The first is conducting research without knowing what decision it supports. The second is collecting interesting information that cannot be translated into product action.

A research brief should therefore state:

1. what the organization is trying to achieve;
2. what is currently unknown;
3. what assumptions or risks require examination;
4. what decision will be made after the study;
5. what evidence would increase or decrease confidence; and
6. who owns the decision.

Research questions and subquestions

A broad question may need to be divided into subquestions. For instance:

Primary question: How do small-business owners manage monthly expense reporting?

Subquestions:

• What triggers the reporting task?
• Which information sources do they use?
• What workarounds have they developed?
• Which errors or delays are most costly?
• How do they judge whether a report is complete and accurate?

Subquestions should be related rather than an unstructured list of everything the team wants to know. A study with too many questions becomes difficult to recruit for, conduct, analyse, and complete on time.

Assumptions, Hypotheses, and Research Questions

An assumption is a belief treated as true for planning purposes. It may concern users, their context, a problem, or the likely effect of a design. Examples include “new customers understand the term ‘beneficiary’” or “frequent users prefer shortcuts to guided instructions.” Assumptions are not automatically facts; they are propositions requiring an appropriate level of confidence.

A hypothesis is a specific, testable expectation about a relationship or outcome. In UX research, hypotheses need not always be statistical. A team might write: “If the form groups questions by topic and explains unfamiliar terms inline, first-time applicants will make fewer interpretation errors.” This hypothesis identifies an intervention and an expected effect.

A good hypothesis specifies:

• the relevant population;
• the condition or design change;
• the expected behaviour or outcome; and
• where appropriate, a comparison condition.

Hypotheses are especially useful in evaluative studies and experiments. Exploratory research should not be forced into prematurely precise hypotheses. If the team does not yet understand the problem space, an open question is often more appropriate than a prediction.

Avoid leading questions that merely seek confirmation. “How helpful was the simplified form?” assumes that the form was helpful. Better questions include “What, if anything, made this form easier or harder to complete?” and “What would you change?”

Generative and Evaluative Research

Generative research

Generative research helps a team understand people and discover opportunities before or during solution development. Common methods include:

• contextual interviews;
• field observation;
• diary studies;
• semi-structured interviews;
• focus groups for attitudes and social discussion;
• journey mapping activities;
• participatory design; and
• analysis of existing customer or service data.

Generative methods are suitable when the team needs to understand goals, routines, language, constraints, workarounds, and unmet needs. They are less suitable for estimating the prevalence of a behaviour in a large population unless combined with quantitative research.

Evaluative research

Evaluative research assesses a design, prototype, service, or interaction. Common methods include:

• moderated usability testing;
• unmoderated usability testing;
• accessibility evaluation with representative users;
• heuristic evaluation by specialists;
• first-click testing;
• tree testing;
• card sorting;
• surveys measuring perceptions or reported outcomes; and
• controlled experiments such as A/B tests.

Evaluative research is appropriate when the team has a concrete artefact or experience to examine. It can identify breakdowns, compare alternatives, and provide evidence for refinement. It cannot, by itself, explain every underlying user motivation or establish that a design will succeed in all contexts.

Choosing between methods

Method selection should follow the question, not researcher preference. The following guide is useful:

Methods may be combined through mixed-methods research. For example, interviews may identify why customers abandon a task, while analytics estimates how often abandonment occurs. The methods answer different questions and should not be treated as interchangeable.

Developing a Study Plan

A study plan is a shared agreement about the research. It should be detailed enough for another trained researcher to understand the procedure, yet flexible enough to accommodate legitimate discoveries.

Essential components

A practical plan normally includes:

1. Title and owner: A concise name and the person responsible for delivery.
2. Background: Relevant product, service, market, or organizational context.
3. Problem and decision: The uncertainty and decision the research will support.
4. Objectives and questions: What the study must learn.
5. Method and rationale: Why the selected method fits the questions.
6. Participants: Required characteristics, exclusions, accessibility needs, and recruitment source.
7. Procedure: Session structure, tasks, prompts, duration, location, and recording arrangements.
8. Analysis approach: How observations, quotations, measures, or other evidence will be organized and interpreted.
9. Timeline: Recruitment, fieldwork, analysis, readout, and decision dates.
10. Deliverables: The specific outputs and their intended audiences.
11. Risks and ethics: Consent, privacy, compensation, safeguarding, and data security.
12. Success criteria: Conditions indicating that the study has answered its decision-critical questions.

Scope and prioritization

Research plans should distinguish essential questions from desirable questions. A useful prioritization method is to label each question as:

• Critical: Without an answer, the decision cannot be made responsibly.
• Important: The answer would materially improve the decision.
• Useful: Interesting or beneficial, but not necessary for this study.

This prevents a short study from becoming an unrealistic investigation of the entire customer experience. It also helps negotiate scope with stakeholders.

Discussion guide and task design

An interview guide should organize a conversation, not function as a rigid script. It commonly includes an introduction, warm-up questions, questions about actual behaviour, probes for examples, and a closing section. Questions about past behaviour are often more informative than general opinions. “Tell me about the last time you changed your delivery address” is usually stronger than “Would you like an easier address-change feature?”

Usability tasks should describe a goal without revealing the interface path. “You have moved and need your next statement sent to your new address. Show me what you would do” is preferable to “Click Settings, then Address.” The researcher should record what participants do, say, misunderstand, and require assistance to complete.

A pilot session is a small trial conducted before full fieldwork. It can reveal ambiguous wording, unrealistic task durations, technical problems, and missing consent procedures. Pilot findings should be used to improve the study, not casually mixed with the final dataset when the procedure changed substantially.

Participants and Sampling

Defining the target population

The target population is the broader group to which the research is relevant. A study may concern first-time renters, independent retailers, people using screen readers, or customer-support agents. Participant criteria should reflect meaningful differences in goals, experience, access needs, and context.

A recruitment screener should establish inclusion and exclusion criteria without disclosing so much about the hypothesis that participants alter their behaviour. Typical criteria include:

• relationship to the product or service;
• frequency and recency of relevant behaviour;
• role or decision authority;
• device, environment, or assistive technology;
• geographic or language requirements; and
• conflicts of interest, such as employment by a competitor.

Demographic characteristics should be collected only when relevant and with appropriate sensitivity. They should not be used as a substitute for behavioural relevance.

Sampling approaches

Purposive sampling recruits people who meet deliberately selected criteria. It is common in qualitative UX research because the aim is depth and relevance.

Convenience sampling recruits people who are easy to access. It is efficient but can produce systematic bias, so its limitations should be made explicit.

Quota sampling sets target numbers for selected characteristics, such as experience level or customer type. It can improve coverage without claiming statistical representativeness.

Stratified sampling divides a population into defined strata and samples within each. It is more structured and may be appropriate for quantitative studies when population information is available.

Random sampling gives eligible members a known selection process, supporting stronger population estimates. It is often difficult in product research because a complete sampling frame may not exist.

Qualitative sample size is not determined by a universal formula. Researchers commonly stop when additional sessions produce little new information relevant to the study questions, while considering the diversity and complexity of the population. This is sometimes described as information power or saturation, but neither should be used mechanically. A narrowly defined, homogeneous study may require fewer participants than a study spanning several user groups and contexts.

Analysis and Evidence Quality

Analysis begins during fieldwork. Researchers should record observations promptly, separate what participants did from what the researcher inferred, and note contradictions as well as confirmations.

A basic qualitative workflow is:

1. organize recordings, notes, and participant identifiers securely;
2. transcribe or summarize relevant material;
3. code observations using descriptive labels;
4. group related codes into patterns or themes;
5. compare themes across participant segments and contexts;
6. examine disconfirming evidence;
7. interpret implications for the research questions; and
8. connect findings to decisions and recommendations.

A finding is evidence-based information, such as “six of eight participants searched for delivery options under account settings.” An insight is an interpretation that explains significance, such as “Customers treat delivery preferences as part of order management rather than account administration.” A recommendation proposes action. These categories should not be collapsed: evidence, interpretation, and action require different levels of justification.

For quantitative measures, define the calculation before collecting data. For example, a task-completion rate can be expressed as:

The numerator and denominator must be defined consistently, and the measure should state whether partial completion, assistance, or technical failure counts as success. A small usability sample can reveal problems, but a completion percentage from that sample should not automatically be generalized to the entire customer population.

Researchers should distinguish severity from frequency. A problem observed once may be critical if it prevents access for a particular group; a frequently observed inconvenience may be less harmful. Severity can be assessed using criteria such as impact, persistence, frequency, and the availability of workarounds, but the rating system should be documented rather than presented as objective fact.

Timeline, Deliverables, and Stakeholder Communication

A typical small study may include the following sequence:

The timeline should include contingency time for recruitment failure, participant cancellation, technical issues, translation, accessibility support, and stakeholder review. Deliverables should be designed for use. A long report may be appropriate for an evidence archive, while a product team may need a concise decision summary containing findings, evidence, implications, confidence, and recommended next steps.

A research readout should not imply certainty beyond the method. Useful wording includes “In this sample,” “Participants commonly reported,” or “The evidence suggests.” Stakeholders should know which findings are strongly supported, which are tentative, and which questions remain unanswered.

Ethics, Privacy, and Inclusion

Research participants are people, not merely sources of data. Ethical practice requires informed consent, voluntary participation, the right to withdraw, fair treatment, and proportionate compensation. Participants should understand the purpose of the study, what participation involves, how recordings or personal information will be used, and who will have access.

Personal data should be minimized. Collect only information necessary for the research, separate identifying details from research notes where practical, restrict access, establish retention periods, and use secure storage. Sensitive topics may require additional safeguards, referral information, or review by an institutional ethics process.

Accessibility is part of research quality. Recruitment, scheduling, consent materials, prototypes, and session environments should accommodate relevant access needs. Researchers should not infer that a design works for disabled people merely because no disabled participants were included. When assistive technology or language is central to the experience, it should be represented in the participant sample and test setup.

Power differences require care. Employees, students, patients, or financially dependent participants may feel unable to refuse. Researchers should clarify that participation is voluntary and that responses will not affect employment, grades, benefits, or service eligibility.

Bias and Limitations

Research can reproduce the biases of its design. Recruitment channels may exclude people with limited internet access. Researcher wording may lead participants. Stakeholders may selectively remember evidence that supports a preferred solution. Participants may describe ideal behaviour rather than actual behaviour, particularly when discussing socially sensitive issues.

Bias cannot be eliminated completely, but it can be managed through neutral prompts, diverse recruitment channels, transparent inclusion criteria, reflexive notes, pilot testing, multiple analysts where feasible, and deliberate examination of contradictory evidence. Triangulation—comparing different methods, sources, or researchers—can strengthen interpretation, but agreement between sources does not guarantee truth.

A research study also has limits of transferability. Findings from eight experienced users in a moderated prototype test may be highly useful for identifying interaction problems, yet they do not establish market prevalence or long-term retention. Each conclusion should be matched to the strength and scope of its evidence.

Technology and AI in Research Planning

Digital recruitment systems, remote-conferencing tools, transcription software, analytics platforms, and collaborative repositories can improve efficiency. However, automation does not remove the need for researcher judgment. Transcription may misrecognize accents or technical terms; automated sentiment analysis may oversimplify meaning; and dashboards may make behavioural data appear more objective than it is.

Generative AI can assist with drafting recruitment language, organizing an initial codebook, or identifying gaps in a discussion guide. It should not be treated as an independent analyst or allowed to expose confidential recordings and personal data to an unapproved service. Researchers must verify AI-generated summaries against source evidence, disclose relevant use where required, and retain responsibility for interpretation.

Contemporary practice increasingly combines qualitative research with product analytics, remote testing, accessibility testing, and continuous discovery. The principle remains unchanged: technology should serve a well-defined question and decision rather than determine the research strategy.

Applied Case: Improving a Community Clinic Booking Service

This is an illustrative case created for educational purposes.

Background

A community clinic found that many patients telephoned reception to book appointments even though online booking was available. The service team wanted to reduce avoidable calls without excluding patients who required assistance.

Problem and assumptions

The team assumed that patients did not use online booking because the interface was confusing. Other plausible explanations included limited digital access, uncertainty about appointment types, concern about choosing incorrectly, language barriers, and the need for reassurance about urgent symptoms.

The decision was whether to redesign the booking flow, provide additional support, change appointment information, or pursue a different intervention.

Research strategy

The team selected a two-stage plan. First, generative research would explore how patients decide whether and how to book. The researchers planned eight semi-structured interviews, including patients who had booked online, patients who had telephoned, older adults, and people using a language other than the clinic’s primary language. Second, evaluative usability sessions would test a revised prototype with six representative participants.

The primary generative question was: “How do patients choose a booking route and determine which appointment type is appropriate?” Subquestions examined triggers, previous experiences, confidence, access constraints, and the role of reception staff.

The recruitment plan used purposive quotas rather than a convenience sample drawn only from existing digital users. Participants received clear consent information and compensation that was not dependent on completing every task.

Analysis

Interviews showed that the original assumption was incomplete. Some patients found the interface confusing, but many avoided it because appointment categories used clinical terms they did not understand. Several participants also believed that choosing the wrong appointment type would delay care. Participants with limited digital confidence valued the possibility of speaking to a person, even when they could technically complete the online process.

Usability testing of the revised prototype found that plain-language descriptions improved selection, but participants still needed a prominent explanation of when to call reception. The team therefore avoided treating online conversion as the only success measure.

Outcome

The service introduced plain-language appointment categories, examples of common reasons for each type, a visible route to reception support, and a warning directing urgent cases to the appropriate service. The team planned to monitor online completion, misbookings, support calls, and feedback rather than optimizing one metric in isolation.

Learning points

The case demonstrates why research should test assumptions rather than merely confirm them. It also shows the value of combining generative and evaluative methods, sampling different user circumstances, and defining success in terms of user and service outcomes. A reduction in telephone calls alone could have indicated exclusion rather than improvement.

Best-Practice Checklist

Before beginning a study, ask:

• What decision will this research support?
• Which assumption or uncertainty matters most?
• Are the questions neutral, focused, and answerable?
• Does the method fit the question and maturity of the design?
• Are participants relevant and sufficiently diverse for the decision?
• Have accessibility, consent, privacy, and safeguarding been addressed?
• Are tasks and prompts free from unnecessary leading language?
• What evidence will count as an answer?
• What are the limitations and likely biases?
• Who will act on the findings, and by when?

Chapter Summary

Research strategy connects organizational goals to evidence-based design decisions. A strong plan distinguishes goals, assumptions, decisions, research questions, hypotheses, methods, evidence, insights, and actions. Generative research explores behaviour, needs, contexts, and opportunities; evaluative research examines an existing or proposed experience. Method selection should follow the question, while participant criteria and sampling should reflect the users and contexts relevant to the decision.

A complete study plan defines scope, procedure, recruitment, analysis, timeline, deliverables, and ethical safeguards. Researchers should distinguish findings from interpretations and recommendations, communicate uncertainty, and avoid overgeneralizing from small or non-representative samples. Inclusive recruitment, privacy protection, accessibility, careful use of technology, and examination of bias are essential to responsible practice. The ultimate measure of a study is not the quantity of data collected but whether credible evidence improves a real decision.

Key Terms

Research strategy: A plan for connecting goals, questions, methods, evidence, and decisions.

Study plan: An operational document specifying how research will be conducted.

Business goal: A desired organizational outcome.

Assumption: A belief treated as true for planning purposes.

Hypothesis: A specific, testable expectation about an outcome or relationship.

Research question: A focused question that can be answered through evidence.

Generative research: Research intended to understand people and discover opportunities.

Evaluative research: Research intended to assess a design, service, or experience.

Mixed-methods research: A planned combination of qualitative and quantitative approaches.

Purposive sampling: Selecting participants because they meet relevant criteria.

Convenience sampling: Recruiting participants who are readily accessible.

Quota sampling: Recruiting to specified targets for selected characteristics.

Target population: The broader group to which research relevance is claimed.

Research finding: Evidence-based information derived from collected data.

Insight: An interpretation that explains the significance of evidence.

Triangulation: Comparing evidence from multiple sources, methods, or researchers.

Pilot study: A preliminary trial used to improve the research procedure.

Completion rate: The percentage of attempts that meet a defined success criterion.

Informed consent: Voluntary agreement based on understandable information about participation.

Research deliverable: An output designed to communicate evidence and support action.

Review Questions

Short Answer Questions

1. What is the difference between a business goal and a research question?
2. Define generative research and give two suitable methods.
3. What makes a hypothesis useful in an evaluative study?
4. Why should a usability task avoid revealing the interface path?
5. State two ethical requirements for user research.

Long Answer Questions

1. Explain the chain from organizational goal to research action, using an example.
2. Compare generative and evaluative research in terms of purpose, timing, methods, and outputs.
3. Describe how participant criteria and sampling approach should be developed for a UX study.
4. Explain the main components of a professional study plan.
5. Discuss how researchers can manage bias, privacy risks, and limitations in a study.

Analytical / Application Questions

1. A product team believes users abandon checkout because the payment button is hard to find. Formulate a neutral research question, one hypothesis, and a suitable two-method research plan.
2. A clinic wants to reduce telephone bookings. Explain why optimizing call reduction alone could be harmful, and propose appropriate participants, methods, and outcome measures.
3. A prototype test shows that four of five participants fail to find an important setting. Explain what can reasonably be concluded, what cannot be concluded, and what further research or design action is justified.

Multiple Choice Questions with Answers

Question 1

Which sequence best represents a sound research strategy?

A. Method → goal → participant → finding → question

B. Goal → assumption → decision → question → method → evidence → action

C. Participant → prototype → business goal → hypothesis → recruitment

D. Finding → deliverable → method → goal → decision

Correct Answer: B. Goal → assumption → decision → question → method → evidence → action

Question 2

Which question is most suitable for generative research?

A. Which of two button colours produces more clicks?

B. Can users complete this prototype task without assistance?

C. How do small-business owners currently prepare expense reports?

D. What is the statistical conversion rate of version A?

Correct Answer: C. How do small-business owners currently prepare expense reports?

Question 3

What is a defining feature of a useful hypothesis?

A. It guarantees the preferred design will succeed.

B. It is specific and testable.

C. It contains only participants’ opinions.

D. It avoids identifying an expected outcome.

Correct Answer: B. It is specific and testable.

Question 4

Which sampling approach deliberately recruits people who meet relevant research criteria?

A. Purposive sampling

B. Random guessing

C. Unrestricted convenience sampling

D. Post hoc sampling

Correct Answer: A. Purposive sampling

Question 5

Which task is least leading in a usability study?

A. Click the account menu and change your address.

B. Show me how you would change your address after moving.

C. Use the helpful address feature to update your details.

D. Find the address button and select the correct option.

Correct Answer: B. Show me how you would change your address after moving.

Question 6

A study has 10 task attempts, of which 7 meet the predefined success criterion. What is the completion rate?

A. 7%

B. 30%

C. 70%

D. 170%

Correct Answer: C. 70%

Question 7

Which statement correctly distinguishes a finding from an insight?

A. A finding is an action plan; an insight is a raw recording.

B. A finding is evidence-based information; an insight interprets its significance.

C. A finding is always quantitative; an insight is always qualitative.

D. A finding is a hypothesis; an insight is a recruitment criterion.

Correct Answer: B. A finding is evidence-based information; an insight interprets its significance.

Question 8

Why is a pilot session conducted before full fieldwork?

A. To guarantee statistically representative results

B. To replace the need for participant consent

C. To identify problems in materials, tasks, timing, or technology

D. To eliminate the need for analysis

Correct Answer: C. To identify problems in materials, tasks, timing, or technology

Question 9

Which practice best supports ethical research?

A. Collecting all available personal information

B. Hiding recording and data-use details from participants

C. Making participation voluntary and explaining data use clearly

D. Recruiting only people who cannot refuse

Correct Answer: C. Making participation voluntary and explaining data use clearly

Question 10

What is the main danger of treating a small usability sample as a population estimate?

A. The sample may reveal too many design issues.

B. The result may be overgeneralized beyond the evidence.

C. The participants will necessarily be randomly selected.

D. Qualitative evidence cannot identify any problems.

Correct Answer: B. The result may be overgeneralized beyond the evidence.