User Research and Evidence Gathering

Introduction

User research is the systematic study of people, their goals, behaviours, contexts, needs, and difficulties in relation to a product or service. In UI/UX design, research replaces assumptions with evidence. It helps a team understand not only what users say, but also what they do, why they do it, and what conditions shape their decisions.

Research is not a single activity conducted at the beginning of a project. It is a continuing process that informs problem definition, information architecture, interaction design, visual design, usability evaluation, and post-launch improvement. A useful research programme combines methods because every method reveals only part of the user experience. Interviews may reveal beliefs and motivations; observation may reveal actual behaviour; surveys may indicate prevalence; and competitive analysis may reveal conventions and opportunities.

The purpose of research is not to collect the largest possible amount of data. It is to gather relevant, credible, and ethically obtained evidence that supports a design decision.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish primary, secondary, qualitative, and quantitative research;
• select appropriate methods for specific UX questions;
• plan ethical interviews, observation, contextual inquiry, and surveys;
• recruit participants using relevant and inclusive criteria;
• identify and reduce common sources of research bias;
• conduct a structured competitive analysis;
• analyse and synthesise qualitative and quantitative findings; and
• communicate evidence as actionable design insights.

1. Research Questions and Evidence

A research project should begin with a decision or uncertainty, not with a favourite method. A broad question such as “What do users think of our app?” is difficult to answer because it has no defined population, context, or intended decision. A stronger question is: “How do first-time customers currently compare delivery options, and what information prevents them from selecting a service confidently?”

A well-formed research question identifies:

1. The population: whose experience is being studied.
2. The behaviour or phenomenon: what people do, understand, or experience.
3. The context: where and under what conditions it occurs.
4. The decision: how the evidence will influence design or strategy.

Research evidence can be classified in two useful ways.

These categories overlap. A survey is primary research and may contain qualitative open-text responses as well as quantitative ratings. Product analytics is often secondary research when it has already been collected, although a new analytics study can be primary research for a particular question.

Exploratory and evaluative research

Exploratory research is used when the problem, users, or relevant behaviours are not yet well understood. Interviews, observation, diary studies, and contextual inquiry are common exploratory methods.

Evaluative research examines an existing concept, interface, or service. Usability testing, heuristic evaluation, surveys, and comparative testing are common evaluative methods.

The distinction is practical rather than absolute. An interview can evaluate an existing experience, while a survey can explore a broad problem. The researcher must match the method to the question.

2. Choosing Research Methods

A method should be selected according to the type of evidence required, the available resources, and the consequences of being wrong.

No method provides a complete account of reality. Interviews are valuable for understanding participants’ perspectives, but self-reports may differ from actual behaviour. Observation reveals behaviour, but the researcher may misunderstand its purpose. Surveys can include many respondents, but poorly written questions create precise-looking results that measure the wrong thing.

A mixed-method approach therefore uses methods sequentially or in combination. For example, interviews may identify possible needs, a survey may estimate how common those needs are, and usability testing may determine whether a proposed solution addresses them.

3. Interviews

A UX interview is a planned conversation designed to learn about a participant’s experiences, goals, practices, and interpretations. The most useful format for early research is often the semi-structured interview: the researcher prepares a guide but can ask appropriate follow-up questions.

Planning an interview

An interview plan normally includes:

• research objectives;
• participant criteria;
• session length and location or platform;
• consent and recording procedures;
• topic areas and questions;
• a plan for protecting personal data; and
• a process for storing and analysing notes.

Questions should be open, neutral, specific, and grounded in experience. “Tell me about the last time you booked a medical appointment” is generally stronger than “Do you find the booking system easy to use?” The first invites a concrete account; the second encourages a general opinion and may imply that ease is expected.

Useful prompts include:

• “What happened next?”
• “How did you decide that?”
• “What alternatives did you consider?”
• “What made that difficult?”
• “Can you show me how you normally do this?”

Avoid leading questions, double questions, technical jargon, and questions that assume an experience has occurred. Ask about past behaviour before asking participants to predict future behaviour. People are often poor at forecasting what they will do, especially when the proposed product is unfamiliar.

Conducting interviews

Begin by explaining the purpose without overstating the product or implying a correct answer. Establish that the participant is evaluating neither their own ability nor the researcher. Obtain consent before recording. During the interview, listen more than you speak, tolerate pauses, and distinguish facts, interpretations, and emotions in your notes.

A practical interview structure is:

1. introduction and consent;
2. background and relevant experience;
3. recent concrete episode;
4. goals, difficulties, and workarounds;
5. reactions to carefully selected prompts or artefacts; and
6. closing questions and explanation of next steps.

The researcher should not become an advocate for a design. Defending an idea suppresses criticism and reduces the credibility of the evidence.

4. Observation and Contextual Inquiry

Observation studies what people do in their natural or realistic setting. It is especially valuable when work is habitual, complex, collaborative, safety-critical, or difficult to describe accurately.

A contextual inquiry combines observation with questions asked at appropriate moments. The researcher watches the participant perform a real activity and asks for clarification without taking control of the task. For example, while observing a receptionist schedule an appointment, the researcher might ask, “What tells you that this slot is suitable?” rather than asking abstractly how scheduling works.

A sound observation plan defines:

• the setting and activities to observe;
• the role of the researcher;
• what may and may not be recorded;
• the observation period and sampling strategy;
• safety and privacy requirements; and
• how interruptions will be handled.

Field notes should separate observation from interpretation. “Participant reread the dosage label twice” is an observation. “Participant was confused” is an interpretation that requires supporting evidence. Notes may be organised under people, tools, tasks, information, environment, interruptions, and workarounds.

Observation also has limitations. The presence of a researcher may change behaviour, known as reactivity. Access may be restricted in private or sensitive settings. A short visit can overrepresent unusual events. Researchers must not secretly record people or expose confidential information merely for convenience.

5. Surveys

Surveys are useful when the researcher needs responses from a larger population, wishes to describe opinions or behaviours numerically, or needs to compare defined groups. They are less suitable for discovering complex processes that respondents may not remember accurately.

A survey should have a clear target population and a recruitment method. Questions must use familiar language, specify a time period where relevant, and avoid combining multiple ideas. “How satisfied are you with the speed and clarity of the checkout?” is a double-barrelled question because speed and clarity may receive different ratings.

Common response formats include:

• binary responses, such as yes/no;
• multiple choice;
• ordered categories, such as never to always;
• rating scales; and
• open-text responses.

A rating scale should have balanced, clearly labelled options. The researcher should avoid assuming that a numerical rating is an interval measurement when respondents may interpret the distances between categories differently. Reporting the distribution of responses is often more informative than reporting only an average.

Survey quality depends on wording, order, accessibility, sampling, and response rate. Leading introductions can influence answers. A long survey may attract only highly motivated respondents. If a survey is distributed through one social-media community, its results may not represent all users.

Where a proportion is reported, define the denominator. If 36 of 60 respondents select an option, the observed proportion is:

Here, x is the number selecting the option and n is the number of valid responses. This percentage describes the respondents in the sample; it does not automatically prove that 60% of the entire user population would respond similarly. Generalisation depends on sampling quality and uncertainty.

6. Competitive and Secondary Research

Competitive analysis examines comparable products, services, or substitutes. The purpose is not to copy competitors but to understand user expectations, established conventions, market positioning, and opportunities for differentiation.

A useful analysis defines the comparison criteria before examining products. Criteria might include onboarding, navigation, search, accessibility, pricing transparency, support, error recovery, privacy communication, and mobile performance. The same task should be attempted across products where possible. Evidence should distinguish direct observation from interpretation.

Secondary research includes academic literature, government publications, industry reports, existing customer-support records, analytics, app-store reviews, and organisational documents. Researchers should evaluate the source’s date, purpose, method, population, definitions, and possible commercial or political interests. A report claiming that users “prefer simplicity” is weak evidence unless it explains how simplicity and preference were measured.

A source review can be recorded in a compact evidence table:

7. Recruitment and Sampling

Recruitment determines whose experiences enter the study. The researcher should define inclusion and exclusion criteria based on the research question rather than recruiting only convenient friends or colleagues.

Important criteria may include experience level, frequency of behaviour, device ownership, accessibility needs, geographic context, language, or role in a workflow. Purposive sampling deliberately selects people with relevant characteristics. Convenience sampling selects people who are easiest to access. Convenience sampling may be acceptable for an early classroom exercise, but its limitations must be stated.

A recruitment screener should ask only questions necessary to determine eligibility. It should not reveal so much about the hypothesis that participants tailor their answers. Incentives should compensate time and inconvenience without becoming coercive. Recruitment materials must not promise outcomes that the study cannot guarantee.

Researchers should consider diversity and inclusion from the start. A sample that excludes people using screen readers, older devices, low bandwidth, minority languages, or assistive technologies may produce a design that works only for privileged users.

Sample size is a design decision, not a universal fixed number. Qualitative studies often recruit until additional sessions produce little new insight, sometimes called thematic sufficiency. Quantitative surveys require attention to sampling error, subgroup sizes, and the precision needed for the decision. A small convenience survey can be useful for directional learning but should not be described as representative.

8. Ethics, Privacy, and Research Governance

Ethical research respects autonomy, welfare, dignity, and privacy. Participants should know what the study involves, what data will be collected, whether sessions will be recorded, how data will be used, and whom to contact with questions. Consent should be voluntary and understandable. Participants may decline questions or withdraw, subject to clearly explained limits once data have been anonymised or incorporated into an aggregate analysis.

Researchers should collect the minimum personal data necessary. Names can often be replaced by participant codes. Recordings, transcripts, contact details, and consent forms should be stored separately with appropriate access controls and retention periods. Sensitive information should not be copied into presentations unnecessarily.

Special care is required when studying children, employees, patients, financially vulnerable people, or people in dependent relationships with the researcher. Organisational permission does not replace individual consent. If research is conducted in a university or regulated setting, institutional ethics review may be required.

Ethical practice also includes accessibility. Provide reasonable accommodations, use accessible materials, and avoid treating disability as a defect to be corrected. Confidentiality must be explained honestly: a researcher should not promise absolute secrecy if safeguarding or legal obligations apply.

9. Bias and Research Quality

Bias is a systematic influence that can distort recruitment, questioning, observation, analysis, or reporting. Common forms include:

• Selection bias: the sample differs systematically from the intended population.
• Non-response bias: people who do not respond differ from those who do.
• Leading-question bias: wording encourages a particular answer.
• Confirmation bias: researchers notice evidence supporting an existing belief.
• Recall bias: participants remember past events inaccurately.
• Social-desirability bias: participants give answers they believe are acceptable.
• Observer bias: expectations influence what the researcher records or interprets.
• Novelty or Hawthorne effects: behaviour changes because participants know they are being studied.

Bias cannot be eliminated completely, but it can be made visible and reduced. Use neutral wording, recruit beyond personal networks, pilot the research guide, record disconfirming evidence, involve more than one analyst where feasible, and maintain an audit trail of important decisions. Triangulation—comparing evidence from different methods, participants, or researchers—helps identify findings that depend on one narrow source.

Quality is not the same as certainty. A credible finding states its evidence and limits. “Four of six observed participants used a workaround when filtering” is more transparent than “Users cannot filter products.”

10. Analysis and Synthesis

Analysis turns raw evidence into organised observations. Synthesis connects those observations into patterns, explanations, and implications for design.

Qualitative analysis

A practical workflow is:

1. familiarise yourself with transcripts, notes, and recordings;
2. label meaningful segments with descriptive codes;
3. compare codes across participants and contexts;
4. group related codes into candidate themes;
5. test themes against contradictory evidence;
6. define each theme precisely; and
7. connect themes to design opportunities or further questions.

A code should describe the evidence rather than jump immediately to a solution. “Copies order number into another system” is a behavioural code. “Needs an export button” is a design proposal. Keeping these separate helps the team consider multiple solutions.

A theme is a patterned and meaningful idea supported by multiple pieces of evidence. A quotation illustrates a theme; it does not prove that the theme is universal. Researchers should preserve negative cases and distinguish frequency from importance. A rare problem may be critical if it creates serious harm.

Quantitative analysis

Begin with data quality: remove accidental duplicates, document missing values, and define valid responses. Use counts and percentages for categorical data. For a numerical measure, the arithmetic mean is:

where xᵢ is each observed value, n is the number of valid observations, and x̄ is the sample mean. The mean is sensitive to extreme values, so the median—the middle ordered value—may better represent a skewed distribution.

A simple worked example illustrates the distinction. Suppose task ratings are 2, 3, 3, 4, and 8 on a numerical scale. The mean is (2 + 3 + 3 + 4 + 8) ÷ 5 = 4, while the median is 3. The unusually high value raises the mean; reporting both values communicates the distribution more honestly. These statistics describe the sample and should not be treated as proof of a causal effect.

Cross-tabulation can compare groups, but small subgroup counts should be labelled clearly. Correlation or association does not establish that one design feature caused an outcome. In certificate-level UX work, careful description is usually more valuable than sophisticated modelling applied to weak data.

From findings to insights

A finding is an evidence-based observation. An insight explains why the finding matters in the user’s situation. A design implication indicates what the team might investigate or change.

For example:

• Finding: In five of seven sessions, participants looked outside the checkout page to confirm delivery dates.
• Insight: Delivery timing is a confidence requirement, not merely supporting information.
• Implication: Explore presenting delivery estimates at the decision point and test whether this reduces abandonment.

Avoid writing personas, journey maps, or “user needs” as if they were evidence themselves. They are communication models derived from evidence and should retain a clear connection to sources.

11. An Illustrative Case: Community Clinic Booking

Background

A fictional community clinic believes that patients abandon its online appointment process because the interface is unattractive. The design team is asked to redesign the booking screen.

Problem

Rather than accepting the stated explanation, the team defines a research question: “How do patients select appointment times, and what prevents them from completing an online booking?”

Approach

The team conducts eight semi-structured interviews with recent patients, observes six participants completing a realistic booking task, reviews three months of support messages, and analyses an existing completion funnel. Recruitment includes people who book frequently and infrequently, people using mobile and desktop devices, and participants with different accessibility needs. The team obtains consent, avoids collecting medical details not required for the study, and stores recordings under participant codes.

Analysis

Interviews suggest that participants want a convenient appointment, but observation reveals that several people cannot interpret the difference between “requested,” “confirmed,” and “held” appointments. Support records show repeated questions about whether a referral is required. Funnel data indicates that completion falls sharply after the eligibility questions, although the data cannot explain why.

The team therefore rejects the initial conclusion that visual attractiveness is the primary problem. The evidence points to uncertainty about eligibility and appointment status. It also reveals that the mobile calendar hides important explanatory text.

Approach and outcome

The team proposes clearer status labels, eligibility guidance before users invest time in the form, and a mobile layout that keeps essential explanations visible. These are treated as hypotheses rather than final answers. A subsequent usability test is planned to evaluate whether the changes improve comprehension and completion without increasing errors.

Learning points

The case demonstrates the value of method combination. Interviews exposed expectations, observation exposed behaviour, support records exposed recurring problems, and funnel data identified where to investigate. None of the sources alone was sufficient. The team also avoided overstating the evidence: a drop in a funnel identified a location of difficulty, not its cause.

12. Contemporary Practice and AI

Digital products create new sources of evidence, including event analytics, session recordings, accessibility reports, search logs, and customer-support text. These sources can reveal patterns at scale, but they also raise privacy and interpretation concerns. Behavioural data records what happened in a system, not necessarily what a person intended.

AI tools can assist with transcription, translation, clustering, summarisation, and detection of recurring terms. They should be used as support rather than as an unquestioned analyst. Researchers must check transcription accuracy, protect confidential data, inspect for demographic or linguistic bias, and preserve human review. An automatically generated theme may merge distinct experiences or give undue weight to frequently repeated but low-importance comments. Participants should not be told that AI guarantees objectivity.

A responsible workflow documents what data entered an AI system, what processing occurred, who reviewed the output, and how errors were corrected. Do not upload identifiable research data to an unapproved service.

13. Best-Practice Research Workflow

A disciplined evidence-gathering project can follow this sequence:

1. define the product decision and research questions;
2. review existing evidence before collecting more;
3. select complementary methods;
4. define inclusion criteria and recruitment channels;
5. prepare consent materials and data-management procedures;
6. pilot the guide, survey, or observation protocol;
7. collect evidence consistently while recording deviations;
8. analyse observations and numerical data separately before integrating them;
9. look actively for contradictory evidence;
10. communicate findings with examples, scope, confidence, and limitations; and
11. connect insights to design decisions and plan follow-up evaluation.

The central professional habit is epistemic discipline: stating what is known, what is inferred, and what remains uncertain.

Challenges and Limitations

UX research is constrained by time, access, budget, recruitment difficulty, organisational politics, and imperfect data. Participants may not represent all users. Research settings may differ from real life. Stakeholders may prefer positive findings. Quantitative data may be abundant but poorly defined, while qualitative data may be rich but difficult to generalise.

These limitations do not make research useless. They make transparent method and careful interpretation essential. A small, well-designed study can expose a serious usability problem, but it should not be used to claim population-wide prevalence. Conversely, a large dataset can show where behaviour changes without explaining users’ reasons. Good practice matches the strength of the claim to the strength of the evidence.

Chapter Summary

User research provides evidence for understanding people, contexts, behaviours, and needs. Interviews reveal perspectives and language; observation and contextual inquiry reveal situated behaviour; surveys support structured comparison; competitive analysis reveals conventions and market opportunities; and secondary research supplies existing knowledge. Effective research begins with decisions and questions, not methods alone.

Ethical research requires informed consent, privacy protection, voluntary participation, accessibility, and special care for vulnerable groups. Recruitment should reflect the research question and include relevant diversity. Bias can be reduced through neutral wording, pilot work, triangulation, reflexivity, and explicit reporting of limitations.

Qualitative analysis organises evidence into codes and themes, while quantitative analysis describes distributions and comparisons without overstating generalisation or causation. Strong synthesis connects findings to insights and testable design implications. Research is most valuable when it changes a decision and when the confidence of the claim matches the quality of the evidence.

Key Terms

Primary research: Evidence collected directly for a current research purpose.

Secondary research: Existing evidence originally collected for another purpose.

Qualitative research: Research focused on meanings, experiences, explanations, and context.

Quantitative research: Research using numerical observations and structured measurement.

Exploratory research: Research used to understand an unfamiliar problem or domain.

Evaluative research: Research used to assess an existing product, concept, or experience.

Semi-structured interview: An interview guided by prepared topics but allowing follow-up questions.

Observation: Systematic study of behaviour in a real or realistic setting.

Contextual inquiry: Observation combined with questions asked during or near the activity being studied.

Survey: A structured instrument for collecting responses from a defined group.

Competitive analysis: Systematic comparison of competing or substitute products and services.

Purposive sampling: Selecting participants because they possess relevant characteristics.

Selection bias: Systematic distortion caused by who is included or excluded from a study.

Triangulation: Comparing evidence from multiple methods, sources, or researchers.

Code: A label applied to a meaningful segment of qualitative data.

Theme: A significant pattern supported by multiple pieces of qualitative evidence.

Finding: An evidence-based observation produced by analysis.

Insight: An explanation of why a finding matters in context.

Informed consent: Voluntary agreement based on understandable information about a study.

Thematic sufficiency: The point at which additional qualitative data produces little new understanding.

Review Questions

Short Answer Questions

1. What is the difference between primary and secondary research?
2. Why is a semi-structured interview often useful in exploratory UX research?
3. State two limitations of surveys.
4. What is selection bias?
5. Why should observation notes distinguish observation from interpretation?

Long Answer Questions

1. Explain how interviews, observation, surveys, and analytics can complement one another in a mixed-method UX study.
2. Describe the major ethical requirements for recruiting and studying UX research participants.
3. Explain how a researcher can reduce leading-question bias and confirmation bias.
4. Describe a practical process for coding and synthesising qualitative research data.
5. Explain why competitive analysis should not be treated as an instruction to copy competitors.

Analytical / Application Questions

1. A team claims that users want a redesigned colour scheme because six interviewees mentioned that the current interface looks old. What additional evidence would you gather before accepting this conclusion, and why?
2. Design a short research plan for understanding why customers abandon an online checkout. Specify the research questions, participants, methods, ethical safeguards, and intended outputs.
3. A survey finds that 60% of 60 respondents prefer a new navigation design, while observation shows repeated navigation errors. Analyse how these findings might both be valid and explain what follow-up research is needed.

Multiple Choice Questions with Answers

Question 1

Which research method is most appropriate for learning how a person performs a task in their normal environment?

A. Contextual inquiry

B. Competitive analysis

C. A branding workshop

D. A market-size forecast

Correct Answer: A. Contextual inquiry

Question 2

What is a major weakness of asking only, “Do you find the checkout easy to use?” in an interview?

A. It is too specific to a past event

B. It encourages a general opinion rather than a concrete account

C. It prevents the participant from expressing any opinion

D. It produces an observation rather than self-report

Correct Answer: B. It encourages a general opinion rather than a concrete account

Question 3

If 36 of 60 valid survey respondents select an option, what proportion of the sample selected it?

A. 36%

B. 40%

C. 60%

D. 96%

Correct Answer: C. 60%

Question 4

Which practice best protects research quality during qualitative analysis?

A. Ignoring evidence that conflicts with the initial hypothesis

B. Converting every participant quote directly into a design requirement

C. Separating observations from interpretations and checking themes against contradictory evidence

D. Using only the most memorable participant quotation

Correct Answer: C. Separating observations from interpretations and checking themes against contradictory evidence

Question 5

What does purposive sampling mean?

A. Selecting participants because they have characteristics relevant to the research question

B. Selecting participants entirely by random lottery from the whole population

C. Selecting only friends and colleagues

D. Selecting participants after the results are known

Correct Answer: A. Selecting participants because they have characteristics relevant to the research question

Question 6

Which statement about a survey percentage is most accurate?

A. It always represents the entire user population

B. It describes the respondents unless the sampling supports broader generalisation

C. It proves that the measured factor caused the response

D. It eliminates the need to report the sample size

Correct Answer: B. It describes the respondents unless the sampling supports broader generalisation

Question 7

A researcher records, “The participant reread the dosage label twice.” This is best classified as:

A. A direct observation

B. A confirmed motivation

C. A design solution

D. A population estimate

Correct Answer: A. A direct observation

Question 8

Which is an example of non-response bias?

A. A question uses technical jargon

B. Participants change behaviour because they are observed

C. People who decline a survey differ systematically from those who complete it

D. A researcher compares two competitors using different criteria

Correct Answer: C. People who decline a survey differ systematically from those who complete it

Question 9

Why should competitive analysis use predefined comparison criteria?

A. To ensure every competitor receives the same logo

B. To support consistent comparison rather than relying only on impressions

C. To guarantee that the researcher finds a market leader

D. To prevent researchers from observing product behaviour

Correct Answer: B. To support consistent comparison rather than relying only on impressions

Question 10

How should an AI-generated research summary be used responsibly?

A. As a final conclusion requiring no human checking

B. Only after identifiable data has been uploaded publicly

C. As an assistive output that is checked for errors, omissions, and bias

D. As a replacement for participant consent

Correct Answer: C. As an assistive output that is checked for errors, omissions, and bias