Advanced Research Methods in IT

Chapter 5
Research Design, Validity, and Inference

Doctorate in Information Technology
Advanced Research Methods in IT   •   Doctorate

Chapter 5: Research Design, Validity, and Inference

1. Introduction

Research design is the logical architecture of a study. It connects a research problem and its questions to data, methods, analysis, and conclusions. In information technology (IT), design decisions are especially important because IT phenomena are simultaneously technical, organizational, social, and economic. A study of a cybersecurity intervention, for example, may involve system logs, employee behavior, managerial policy, financial loss, and regulatory requirements. A weak design can produce precise measurements that do not answer the research question, while a strong design makes the evidential basis of a conclusion explicit.

This chapter examines how doctoral researchers align research questions with exploratory, descriptive, explanatory, predictive, evaluative, longitudinal, cross-sectional, experimental, and quasi-experimental designs. It also addresses validity, bias, causal reasoning, generalization, and inference. The central argument is that no design is universally superior. The appropriate design depends on the question, the state of knowledge, the feasibility of intervention, the time horizon, the unit of analysis, and the claims the researcher intends to make.

2. Learning Objectives

After completing this chapter, the reader should be able to:

1. Distinguish major research purposes and design types in IT research.

2. Align research questions, variables, units of analysis, and designs.

3. Explain construct, internal, external, statistical conclusion, and ecological validity.

4. Identify common sources of bias and threats to inference.

5. Select appropriate experimental, quasi-experimental, longitudinal, and cross-sectional strategies.

6. Develop a defensible research design and analysis plan.

7. Apply design principles to practical and managerial IT problems.

8. Evaluate ethical, governance, and reproducibility requirements.

3. Meaning and Concept

Research design is a plan for generating and interpreting evidence. It specifies what will be studied, from whom or from what, when observations will be made, how variables will be measured, whether conditions will be manipulated, and how alternative explanations will be addressed. Design is therefore broader than a method. A survey, interview, experiment, or machine-learning model is a method; the design explains why that method is suitable for the question and how it contributes to valid inference.

A research question expresses the intended relationship between inquiry and evidence. Exploratory questions ask what is occurring or how a phenomenon is experienced. Descriptive questions ask about characteristics, frequencies, or distributions. Explanatory questions ask why or through what mechanisms something occurs. Predictive questions ask how accurately future or unknown outcomes can be estimated. Evaluative questions ask whether an intervention, policy, system, or program achieved its intended effects.

The distinction between description, prediction, and explanation is essential. A model may predict customer churn accurately without identifying its causes. Conversely, a theoretically informative explanatory model may have modest predictive performance. Similarly, correlation may support a useful forecast but does not by itself establish that changing one variable will change another. Doctoral research must therefore state clearly what kind of claim is being made.

4. Fundamental Principles

Alignment

The research problem, questions, design, measurements, sampling strategy, analysis, and conclusions must be aligned. A question about causal impact requires stronger control of confounding than a question about prevalence. A question about lived experience may require qualitative inquiry rather than a standardized instrument. Misalignment is one of the most frequent causes of weak research.

Temporal precedence and comparison

Causal claims generally require evidence that the presumed cause precedes the effect and that the outcome differs from a credible comparison condition. Randomized experiments provide this through assignment and control. Quasi-experiments seek comparable evidence using naturally occurring policies, thresholds, time discontinuities, or matched groups.

Measurement quality

Constructs such as trust, usability, digital maturity, resilience, and algorithmic fairness are not directly observable in the same way as response time or storage capacity. Researchers must operationalize them through indicators and justify reliability, validity, and sensitivity to change.

Transparency and falsifiability

A design should make it possible for evidence to challenge the proposed explanation. Hypotheses, exclusion criteria, outcomes, analyses, and deviations from the original plan should be documented. Transparency reduces undisclosed flexibility and selective reporting.

Proportionality

Design complexity should correspond to the importance and risk of the claim. A high-stakes study of an automated medical decision system requires stronger validation and governance than a low-risk usability pilot. Resources should be concentrated on the threats most capable of changing the conclusion.

Reflexivity and context

Researchers bring assumptions, professional interests, and relationships to the field. In organizational and qualitative IT studies, reflexivity requires acknowledging how access, authority, identity, and interpretation may influence evidence. Context is not merely a nuisance; it can explain why an intervention works in one setting and not another.

5. Major Components / Elements

A complete design normally includes the following elements.

Research purpose and questions. The purpose may be exploratory, descriptive, explanatory, predictive, or evaluative. A study can have more than one purpose, but the hierarchy should be explicit.

Unit of analysis. This is the entity about which conclusions are made: an individual, team, organization, software project, transaction, device, country, or time period. The unit of observation may differ. For example, many log records may be observed, while the conclusion concerns organizations.

Conceptual framework. A framework identifies constructs, presumed relationships, mechanisms, moderators, and contextual conditions. It prevents data collection from becoming an unstructured search for interesting associations.

Population and sampling. The target population is the set to which the researcher intends to generalize. The sampling frame is the accessible representation of that population. Probability sampling supports statistical generalization, whereas purposive and theoretical sampling support information-rich and conceptually relevant cases.

Operationalization. Abstract constructs are translated into variables, indicators, coding rules, and instruments. Operational definitions should include scale, timing, source, and procedures for handling missing or anomalous values.

Design structure. The researcher specifies whether observations are cross-sectional or longitudinal, whether conditions are manipulated, and how comparison groups or cases are selected.

Analysis plan. The plan should connect each question to an analysis technique, assumptions, effect measures, uncertainty estimates, and sensitivity analyses. Statistical significance alone is insufficient; practical significance and precision also matter.

Inference and scope. The researcher must state whether conclusions concern the sample, a wider population, a mechanism, a prediction task, or a causal effect. Claims should not exceed the design's evidential capacity.

6. Process / Framework / Methodology

Step 1: Define the decision or knowledge problem

Begin with the practical or theoretical problem, not with a preferred technique. A manager may want to know whether a new identity-management system reduces account compromise. The researcher should clarify whether the need is to understand user resistance, estimate incident rates, explain causal impact, predict risk, or evaluate return on investment.

Step 2: Classify the question

Exploratory questions are appropriate when concepts, mechanisms, or relevant variables are uncertain. Descriptive questions characterize a phenomenon. Explanatory questions test relationships or mechanisms. Predictive questions assess out-of-sample performance. Evaluative questions judge an intervention against explicit criteria. These categories can be combined in a staged program, but each requires a suitable evidential standard.

Step 3: Choose the time perspective

A cross-sectional design observes units at one point or short period and is efficient for prevalence and association. A longitudinal design observes the same units or comparable populations over time. Panel studies follow the same units; cohort studies follow units sharing a defining experience; repeated cross-sectional studies sample different units from the same population at multiple times. Longitudinal designs help establish temporal order and change, but attrition and time-varying confounding must be managed.

Step 4: Decide whether intervention is possible

In an experiment, the researcher assigns participants or units to conditions, ideally at random. Randomization balances measured and unmeasured characteristics in expectation and supports causal inference. In IT settings, cluster randomization may assign departments, schools, or organizations rather than individuals to avoid contamination.

When randomization is infeasible or unethical, a quasi-experimental design may exploit a natural policy change, eligibility threshold, phased rollout, interrupted time series, difference-in-differences, instrumental variable, or matched comparison. These designs require explicit assumptions and diagnostic tests.

Step 5: Develop a measurement plan

Define primary outcomes before collecting data. Specify instruments, data sources, timing, coding, quality checks, and procedures for missing data. If a survey measures perceived security, the researcher should assess content coverage, response interpretation, internal consistency, and relationships with relevant external indicators. If system logs measure availability, maintenance windows and monitoring changes must be documented.

Step 6: Plan sampling and power

Sampling should reflect the intended population and the likely structure of the data. Clustered observations violate independence assumptions if analyzed as though they came from unrelated individuals. Power analysis should consider the minimum substantively meaningful effect, expected variance, attrition, clustering, multiple comparisons, and design restrictions. Large samples cannot repair systematic bias.

Step 7: Anticipate threats to validity

Before data collection, construct a threat matrix. For each major claim, identify possible confounders, selection processes, measurement errors, spillovers, history effects, maturation, attrition, and model dependence. Specify mitigation measures such as randomization, blinding, preregistration, validation samples, negative controls, robustness checks, or triangulation.

Step 8: Analyze and communicate uncertainty

Use estimates with confidence or credible intervals, not isolated p-values. Examine assumptions, influential observations, missingness, heterogeneity, and sensitivity to alternative specifications. For predictive work, separate training, validation, and test data and report performance on data not used for model tuning. For qualitative work, document coding procedures, rival interpretations, negative cases, and evidence supporting thematic claims.

7. Important Theories and Models

The validity framework

Construct validity asks whether the operationalization represents the intended concept. Internal validity asks whether the observed relationship is attributable to the proposed explanation rather than bias or confounding. External validity concerns transfer or generalization to other populations, settings, times, or technologies. Statistical conclusion validity concerns whether the analytical procedures support the claimed relationship, including adequate power, appropriate models, and reliable estimates.

Causal diagrams

Directed acyclic graphs (DAGs) represent assumed causal relationships. They help distinguish confounders, mediators, colliders, and sources of selection bias. For example, organizational size may influence both adoption of an AI governance program and incident reporting. Adjusting for size may be appropriate. A variable caused by both treatment and an unmeasured factor may be a collider; conditioning on it can create a spurious association.

Logic models and theory of change

A logic model links inputs, activities, outputs, short-term outcomes, and long-term impacts. A cybersecurity awareness program may use training resources to produce completed modules, improved knowledge, safer behavior, and fewer successful phishing incidents. The model clarifies intermediate mechanisms and indicates which outcomes should be measured at which times.

Design-based causal inference

The potential-outcomes framework defines a causal effect as the difference between an outcome under treatment and the outcome that would have occurred for the same unit under control. Only one potential outcome is observed for each unit, creating the fundamental problem of causal inference. Randomization, credible comparison groups, and design-based assumptions are used to estimate the missing counterfactual.

Generalizability and transportability

A result may be internally credible but not applicable elsewhere. Transportability analysis asks how evidence from a study population can inform a target population with different users, infrastructure, institutions, or threat environments. Contextual moderators should be measured rather than treated as incidental details.

8. Practical Applications

An exploratory design is suitable when an organization does not know why developers avoid a secure coding platform. Interviews, observation, diary studies, and artifact analysis can identify perceived barriers and emergent practices. A descriptive design can then estimate how widespread those barriers are across teams.

An explanatory design may test whether psychological safety influences incident reporting through trust in management. Mediation analysis can be informative, but it requires temporal and theoretical justification and should not be interpreted as proof of mechanism from cross-sectional data alone.

A predictive design may estimate the probability that a cloud resource will experience a service interruption. Evaluation should use temporally realistic validation, because randomly splitting records can leak future information into the training set. Performance should be reported across relevant subgroups, not only as an overall average.

An evaluative design may assess a service-desk chatbot. Outcomes could include resolution time, first-contact resolution, user satisfaction, escalation rate, accessibility, and unintended workload transfer to human agents. A cost-benefit conclusion should include implementation and maintenance costs, not merely technical accuracy.

9. Examples

A cross-sectional survey of 1,000 employees can estimate the prevalence of multifactor-authentication fatigue and examine associations with job role. It cannot confidently determine whether fatigue caused insecure behavior because exposure and outcome are measured at the same time.

A longitudinal panel can measure digital transformation maturity and productivity for three years. It can establish temporal ordering more effectively, although organizations that drop out may differ systematically from those that remain.

A randomized experiment can assign users to two interface designs and compare completion rates. If users share accounts or discuss the interfaces, contamination may reduce the observed difference. Cluster assignment may be preferable.

A difference-in-differences study can compare organizations adopting a privacy control at different times with organizations not yet adopting it. The design depends on the parallel-trends assumption: absent adoption, both groups would have followed similar outcome trends. Pre-intervention trends and placebo tests should be examined.

A regression-discontinuity design can evaluate access to an innovation grant assigned to organizations above a score threshold. Its estimate is most credible near the threshold and may not generalize to organizations far from it.

10. Technology and Digital Applications

Digital traces, application programming interfaces, sensor streams, cloud logs, and platform records permit fine-grained longitudinal research. They can improve temporal resolution, reveal system behavior, and reduce recall error. However, trace data are not automatically objective. Logging policies, missing events, changing software versions, and unequal digital visibility can create systematic measurement bias.

Computational experiments can test recommender systems, security controls, or interface variants. A/B testing requires stable assignment, well-defined outcomes, safeguards against interference, and monitoring for adverse effects. Machine-learning research requires separation of development and evaluation data, leakage prevention, calibration assessment, subgroup analysis, and documentation of data provenance.

Digital tools also support reproducible research through version control, data dictionaries, workflow automation, containerized environments, and preregistered protocols. Reproducibility tools improve auditability but do not solve invalid constructs, biased samples, or unethical data collection.

11. Strategic / Managerial Importance

Research design affects investment decisions, risk governance, compliance, and organizational legitimacy. Managers need to know whether an observed improvement reflects an intervention or merely a favorable market period. They also need to know whether a predictive model remains reliable after a platform change or whether a pilot result applies to all business units.

A transparent design enables decision-makers to distinguish evidence of effectiveness from evidence of feasibility. It also clarifies uncertainty and the cost of being wrong. In high-risk settings, governance should require independent validation, monitoring after deployment, documented assumptions, and escalation procedures when performance deteriorates.

12. Detailed Case Study

The following case is fictional and illustrative.

Background

Northbridge Services, a regional financial-technology provider, introduced adaptive authentication intended to reduce account takeover while preserving customer convenience. The system increased authentication requirements when a transaction appeared risky.

Problem

After six months, reported account-takeover incidents declined by 18 percent, but customer complaints and abandonment of online transactions increased. Executives disagreed about whether the system was effective and whether the decline reflected the intervention or a simultaneous fraud-prevention campaign.

Analysis

The initial comparison used incidents in the six months before and after implementation. This simple pre-post design was vulnerable to history effects, seasonal fraud patterns, changes in reporting, and regression to the mean. The research team constructed a quasi-experimental design using a phased rollout. Branches scheduled for later implementation served as a comparison group. Outcomes included confirmed account takeover, false-positive challenge rate, transaction completion, complaint volume, and support costs.

The team examined pre-intervention trends, adjusted for branch and calendar effects, and conducted subgroup analyses by customer age, accessibility needs, and transaction type. Logs were audited to determine whether changes in detection rules altered the definition of an incident.

Strategy / Solution

A difference-in-differences model estimated the change in outcomes for early-adopting branches relative to not-yet-adopting branches. A qualitative component interviewed customers and support staff to explain abandonment and identify burdens not visible in logs. The protocol defined primary outcomes in advance, documented missing data, and included a fairness review. The team avoided adjusting for post-treatment customer satisfaction when estimating the total effect because satisfaction could be a mediator.

Outcome

The evidence indicated a genuine reduction in confirmed account takeover, but the benefit was concentrated in high-risk transactions. False-positive challenges increased for customers using assistive technologies and for international travelers. After interface changes, clearer explanations, and an alternative verification channel, abandonment declined while security gains remained substantially intact.

Learning Points

The case demonstrates that evaluation should measure benefits, costs, and distributional effects. A pre-post comparison was insufficient because it did not establish a credible counterfactual. Combining quasi-experimental evidence with qualitative inquiry improved both causal interpretation and implementation design. It also shows that an average effect can conceal important subgroup harms.

13. Advantages / Benefits

A carefully designed study improves causal clarity, reduces wasted data collection, supports reproducibility, and makes conclusions proportionate to evidence. It helps organizations compare alternatives, identify mechanisms, anticipate implementation barriers, and monitor unintended effects. Explicit validity assessment also increases stakeholder trust because limitations are acknowledged rather than hidden.

14. Limitations / Challenges

Perfect designs rarely exist. Randomization may be expensive, politically difficult, or unethical. Longitudinal studies face attrition, changing technologies, and time-varying confounding. Administrative data may be incomplete or generated for operational rather than research purposes. Quasi-experimental designs rely on assumptions that cannot always be verified. Qualitative interpretation may be influenced by researcher position, while quantitative models may create false confidence through complex but poorly justified specifications.

Generalization is particularly difficult in IT because systems, standards, threats, and user populations change quickly. A model validated on one platform may fail after deployment elsewhere. Researchers must therefore distinguish limitations that merely reduce precision from those that threaten the direction or interpretation of the result.

15. Ethical and Governance Considerations

Participants must receive appropriate information about purposes, risks, data use, withdrawal, and confidentiality. In workplace studies, voluntary consent is difficult when managers control access; researchers should prevent supervisors from seeing individual responses. Security logs and behavioral traces can reveal sensitive activities and should be minimized, protected, retained only as long as justified, and governed by access controls.

Researchers should assess re-identification risk, secondary use, algorithmic discrimination, and possible harms from publishing vulnerabilities. Experimental interventions require monitoring for adverse effects and stopping rules when risks exceed anticipated benefits. Governance should assign responsibility for data stewardship, model validation, incident response, and post-study retention. Ethical review is not a substitute for methodological validity: collecting large amounts of ethically approved but irrelevant data remains poor research.

16. Current and Emerging Trends

Digital experimentation is increasingly integrated with operational platforms, creating opportunities for rapid evaluation but also risks of hidden experimentation and consent ambiguity. Causal machine learning is being used to estimate heterogeneous treatment effects, although flexible models do not remove confounding. Synthetic data can support development and sharing, but privacy and fidelity must be assessed.

Researchers increasingly combine log analysis, surveys, interviews, and experiments to triangulate technical and human outcomes. Continuous monitoring is becoming essential because software updates, adversarial behavior, and population shifts can cause concept drift. Open science practices—including preregistration, registered reports, data and code documentation, and transparent reporting of null results—are expanding in applied IT research.

17. Best Practices

1. Begin with the claim and question, not the fashionable method.

2. Define the unit of analysis and intended population explicitly.

3. Distinguish description, prediction, association, explanation, and causal effect.

4. Specify primary outcomes and analysis decisions before examining results when feasible.

5. Use diagrams or logic models to clarify mechanisms and confounders.

6. Measure implementation fidelity, unintended consequences, and subgroup effects.

7. Separate model development from final evaluation data.

8. Report effect sizes, uncertainty, assumptions, and sensitivity analyses.

9. Use qualitative evidence to investigate mechanisms and anomalous findings.

10. Document data provenance, version changes, exclusions, and deviations from protocol.

11. Treat privacy, fairness, security, and accessibility as design requirements.

12. Limit conclusions to the population, time, and conditions supported by evidence.

18. Future Scope

Future IT research will increasingly require designs capable of studying adaptive systems whose behavior changes in response to users and attackers. Sequential experiments, adaptive trials, causal reinforcement-learning methods, and continuously updated evaluation frameworks may become more common. Privacy-preserving analysis will enable research on sensitive digital traces, while federated and distributed designs may support multi-organization inference without centralizing raw data.

A major frontier is the integration of technical performance with social and institutional outcomes. Future studies should evaluate not only whether an algorithm works, but for whom, under what conditions, at what cost, and with what implications for autonomy and accountability. Doctoral researchers can contribute by developing designs that are simultaneously rigorous, context-sensitive, ethically governed, and operationally useful.

19. Chapter Summary

Research design provides the bridge between questions and defensible conclusions. Exploratory, descriptive, explanatory, predictive, and evaluative purposes require different forms of evidence. Cross-sectional designs are efficient for describing current conditions, whereas longitudinal designs clarify change and temporal order. Experiments provide strong causal evidence when randomization is feasible; quasi-experiments can approximate causal inference when it is not. Validity concerns the meaning of measures, the credibility of causal claims, the quality of statistical conclusions, and the scope of generalization. Bias can arise through sampling, measurement, attrition, confounding, data leakage, selective reporting, and unequal digital visibility. Strong IT research aligns questions, design, analysis, ethics, and governance while reporting uncertainty and limitations honestly.

20. Key Terms

1. Research design: The overall plan linking questions, evidence, methods, and inference.

2. Exploratory design: An approach used to investigate poorly understood phenomena and generate concepts or hypotheses.

3. Descriptive design: A design that characterizes distributions, frequencies, or conditions.

4. Explanatory design: A design aimed at understanding relationships, causes, or mechanisms.

5. Predictive design: A design focused on estimating unknown or future outcomes.

6. Evaluative design: A design that assesses the merit, effects, or value of an intervention.

7. Cross-sectional design: A design collecting observations at one point or short period.

8. Longitudinal design: A design examining units or populations over time.

9. Experimental design: A design in which the researcher assigns conditions, ideally randomly.

10. Quasi-experimental design: A design estimating effects without random assignment by using a credible comparison structure.

11. Construct validity: The extent to which a measure represents the intended construct.

12. Internal validity: The credibility that an observed effect is caused by the proposed factor.

13. External validity: The extent to which findings generalize or transfer beyond the study.

14. Confounder: A variable related to both a presumed cause and outcome that can distort their association.

15. Counterfactual: The outcome that would have occurred for a unit under an alternative condition.

16. Unit of analysis: The entity about which conclusions are made.

17. Measurement bias: Systematic distortion arising from how a variable is defined or measured.

18. Difference-in-differences: A quasi-experimental estimator comparing changes across treatment and comparison groups.

19. Data leakage: Inadvertent use of information in model development that would not be available at prediction time.

20. Triangulation: Combining methods, data sources, researchers, or theories to strengthen interpretation.

21. Review Questions

Short Answer Questions

1. How does an explanatory question differ from a predictive question?

2. Define internal validity and identify one threat to it.

3. Why is the unit of analysis important in IT research?

4. What is the purpose of a counterfactual?

5. Distinguish a cross-sectional study from a longitudinal study.

Long Answer Questions

1. Explain how a doctoral researcher should align a research question with design, sampling, measurement, and analysis.

2. Compare experimental and quasi-experimental designs, including their assumptions and practical limitations.

3. Discuss construct, internal, external, and statistical conclusion validity using IT examples.

4. Explain how causal diagrams can improve the analysis of an organizational technology intervention.

5. Develop a research design for evaluating a machine-learning cybersecurity system, including outcomes, comparison strategy, ethics, and validity threats.

Analytical / Application Questions

1. An organization reports that incidents fell after deploying a security platform, but it also changed reporting procedures during the same period. Identify the threats to inference and propose a stronger evaluation design.

2. A predictive model performs well overall but poorly for users with accessibility needs. Explain how the finding should affect validation, governance, and deployment decisions.

3. A university introduces an AI tutoring system in selected departments and plans to compare examination scores after one semester. Design a study that addresses selection bias, spillovers, baseline differences, and ethical concerns.

22. Multiple Choice Questions with Answers

Question 1

Which design is most appropriate for identifying previously unknown barriers to employees’ adoption of a secure coding platform?

A. Exploratory qualitative design

B. Cross-sectional experiment

C. Regression-discontinuity design

D. Predictive validation design

Correct Answer: A. Exploratory qualitative design

Question 2

What is the principal purpose of random assignment in an experiment?

A. To increase the number of measured variables

B. To balance potential confounders across conditions in expectation

C. To guarantee external validity

D. To eliminate measurement error

Correct Answer: B. To balance potential confounders across conditions in expectation

Question 3

A study measures security awareness and insecure behavior at the same time. Which limitation is most direct?

A. It cannot establish temporal precedence confidently

B. It necessarily has low statistical power

C. It cannot use descriptive statistics

D. It must be an experimental study

Correct Answer: A. It cannot establish temporal precedence confidently

Question 4

In a difference-in-differences study, which assumption is especially important?

A. The outcome must be normally distributed

B. The treatment group must be twice as large as the control group

C. The groups would have followed parallel trends without treatment

D. All participants must receive the intervention simultaneously

Correct Answer: C. The groups would have followed parallel trends without treatment

Question 5

Which example best illustrates data leakage in predictive IT research?

A. Removing duplicate records before model training

B. Using a future fraud outcome indirectly as a feature during training

C. Reporting performance separately by department

D. Using a validation set for model selection

Correct Answer: B. Using a future fraud outcome indirectly as a feature during training

Question 6

A measure of ‘digital trust’ correlates strongly with several other trust instruments but omits privacy concerns central to the concept. Which issue is most relevant?

A. Construct validity

B. Temporal precedence

C. Statistical power

D. Cluster randomization

Correct Answer: A. Construct validity

Question 7

Which design is most suitable for evaluating a policy assigned to organizations above an eligibility score threshold?

A. Regression discontinuity

B. Simple case study

C. Convenience survey

D. Unstructured observation

Correct Answer: A. Regression discontinuity

Question 8

Why should a predictive model be assessed on data not used for tuning?

A. To guarantee causal inference

B. To estimate performance on unseen data more credibly

C. To remove the need for feature definitions

D. To ensure every subgroup has identical outcomes

Correct Answer: B. To estimate performance on unseen data more credibly

Question 9

In the fictional authentication case, why was the initial pre-post comparison insufficient?

A. It used too many outcome measures

B. It could not separate the intervention from coinciding historical changes

C. It automatically created random assignment

D. It measured customer outcomes

Correct Answer: B. It could not separate the intervention from coinciding historical changes

Question 10

Which practice most directly supports transparent and reproducible research?

A. Changing primary outcomes after seeing results without documentation

B. Reporting only statistically significant findings

C. Documenting data provenance, analysis decisions, and protocol deviations

D. Keeping all computational procedures undocumented

Correct Answer: C. Documenting data provenance, analysis decisions, and protocol deviations

Advanced Research Methods in IT
Chapter 5 of 15