14. Doctoral Research Methods in Big-Data Analytics

Introduction: From Large Datasets to Defensible Knowledge

Big-data analytics has become an important site of doctoral research because it connects computational infrastructures, statistical reasoning, organizational decision-making, and social consequences. Yet access to large volumes of data does not by itself produce rigorous scholarship. A doctoral contribution depends on the quality of the research question, the suitability of the method, the validity of the evidence, the transparency of the analytical process, and the significance of the resulting explanation or artifact.

Research in this field is methodologically plural. A scholar may design a new streaming architecture, test whether an algorithm improves operational outcomes, study how people behave on a digital platform, examine a public-sector data initiative, or combine statistical analysis with interviews. These activities require different assumptions about reality, evidence, causation, and generalization. The central challenge is therefore not simply to select a sophisticated tool, but to align the research paradigm, design, data, analytical method, and claims.

This chapter presents a doctoral framework for conducting rigorous research in big-data analytics. It examines design science, experiments, computational social science, case study research, mixed methods, reproducible analytics, theory construction, causal inference, and research ethics. The discussion treats technical performance and scholarly validity as related but distinct concerns. A model may be accurate but unfair; a causal claim may be statistically significant but substantively weak; and a computational result may be impressive but impossible to reproduce.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Distinguish major research paradigms and select methods appropriate to different big-data questions.

2. Formulate research problems, constructs, research questions, hypotheses, and contribution claims.

3. Design and evaluate artifacts using design-science principles.

4. Construct experiments and assess causal claims using appropriate controls, estimands, and validity criteria.

5. Apply computational social-science methods critically, including attention to measurement and representativeness.

6. Conduct rigorous case studies and integrate qualitative and quantitative evidence through mixed methods.

7. Build reproducible analytical workflows and document data, code, models, and environments.

8. Identify ethical, legal, governance, privacy, and fairness issues in big-data research.

9. Evaluate limitations, threats to validity, and the boundaries of generalization.

Research Questions, Contributions, and Philosophical Positioning

A doctoral project begins with a problem rather than a dataset. A useful problem identifies a meaningful gap between what is needed and what is currently possible, understood, or explained. In big-data analytics, the gap may be technological, methodological, theoretical, empirical, or socio-organizational. For example, an organization may need low-latency anomaly detection under changing data distributions; researchers may lack a theory explaining adoption of algorithmic recommendations; or an apparently successful predictive system may produce unequal error rates across groups.

Research questions should be sufficiently precise to guide design and sufficiently important to justify investigation. Questions beginning with “how” often support process, design, or explanatory inquiry; “what is the effect of” suggests causal estimation; “under what conditions” invites contingency analysis; and “to what extent” requires operationalized measurement. A question such as “Can machine learning improve fraud detection?” is incomplete because it does not specify the intervention, comparison, outcome, population, time horizon, or costs of errors. A stronger version might ask: “Among verified online transactions, how does a cost-sensitive gradient-boosting model affect the rate of undetected fraud and false payment blocks compared with the incumbent rules engine during a six-month prospective evaluation?”

A contribution is not synonymous with novelty. A contribution may extend theory, refine a construct, introduce a validated artifact, establish a causal relationship, provide a new dataset or benchmark, reveal a mechanism, or demonstrate the boundary conditions of an existing claim. Doctoral work should state its contribution explicitly and distinguish between contribution to knowledge and contribution to practice.

Philosophical assumptions influence method. Positivist and post-positivist studies generally emphasize measurable constructs, hypothesis testing, probabilistic explanation, and threats to validity. Interpretive research examines how actors construct meaning within social and organizational contexts. Pragmatist research selects methods according to the problem and values useful, warranted knowledge rather than allegiance to one philosophical camp. Critical approaches ask how data systems reproduce power, exclusion, or unequal visibility. These positions are not merely labels: they shape what counts as evidence, how the researcher relates to participants, and what kind of claim can legitimately be made.

A Research Architecture for Big-Data Studies

A coherent study can be represented as an alignment among five elements: the research problem, the knowledge claim, the evidence required, the method used to obtain that evidence, and the evaluation criteria. Misalignment is common. Researchers sometimes use available platform data to answer a question about a population the data do not represent, or use predictive accuracy to support a causal conclusion. A research architecture makes such errors visible before data collection begins.

Important concepts include constructs, variables, units of analysis, population, sample, intervention, outcome, mechanism, context, and boundary condition. The unit of analysis may be a transaction, user, organization, geographic area, text document, network tie, or time interval. Confusing the unit of observation with the unit of analysis can produce ecological or atomistic fallacies. For example, a relationship observed across regions does not necessarily hold for individuals within those regions.

Operationalization translates an abstract construct into observable indicators. “Trust in automated advice” might be measured through validated survey items, acceptance behavior, override frequency, or interview accounts. Each indicator captures only part of the construct. Researchers should document the rationale for measurement, assess reliability and validity, and consider whether the measurement process itself changes behavior.

A useful protocol normally specifies the research questions, conceptual model, hypotheses where appropriate, data sources, inclusion and exclusion rules, preprocessing, analysis plan, evaluation metrics, ethical safeguards, and dissemination strategy. Pre-registration can distinguish confirmatory analysis from exploratory discovery. It does not prohibit exploration; rather, it prevents exploratory results from being presented retrospectively as if they were predicted in advance.

Design Science: Building and Knowing Through Artifacts

Design science is appropriate when the research objective includes creating and evaluating an artifact intended to solve a relevant problem. Artifacts may be algorithms, software systems, data models, methods, architectures, workflows, or governance mechanisms. The artifact is not the contribution by itself. The contribution lies in justified design knowledge: principles, constructs, models, methods, or evidence explaining why and under what conditions the artifact works.

A design-science cycle commonly includes problem identification, objective definition, design and development, demonstration, evaluation, and communication. Relevance comes from the problem environment, while rigor comes from existing knowledge and appropriate evaluation. These two dimensions should interact. A technically elegant solution that addresses no consequential problem has limited relevance; a practically important system unsupported by prior knowledge or systematic evaluation has weak rigor.

Evaluation must be matched to the artifact and claim. A predictive model may be evaluated using discrimination, calibration, robustness, computational cost, and fairness. A data platform may require tests of throughput, latency, fault tolerance, scalability, maintainability, and security. A decision-support artifact may also require user studies, field deployment, workflow analysis, and assessment of unintended consequences. Benchmarking against a weak baseline is inadequate. Baselines should include the current practice, credible simple methods, and competitive state-of-the-art alternatives where feasible.

Design research often benefits from iterative cycles. Early prototypes expose requirements and failure modes; later versions support controlled evaluation. However, iteration must not become undocumented trial and error. Researchers should preserve version histories, record design decisions, identify changes in requirements, and explain how evidence influenced subsequent designs. Generalizable design principles should be abstracted from the specific implementation without claiming universal validity.

Experiments and the Logic of Causal Inference

A causal claim states that an intervention changes an outcome relative to a defined counterfactual. The counterfactual is what would have happened to the same unit under an alternative condition. Randomized experiments estimate causal effects by assigning units to treatment and control conditions through a random mechanism, thereby balancing observed and, in expectation, unobserved confounders.

In big-data settings, experiments may be online controlled trials, laboratory studies, field experiments, simulation experiments, or system benchmarks. A well-designed online experiment requires a clearly defined treatment, stable assignment, an appropriate unit of randomization, sufficient duration, pre-specified outcomes, and monitoring for interference and novelty effects. If users influence one another, individual randomization may violate the assumption that one unit’s treatment does not affect another unit’s outcome. Cluster or network-based designs may be more suitable.

Researchers should distinguish the average treatment effect from effects for particular subgroups. Subgroup analysis can reveal heterogeneity, but searching across many subgroups increases the likelihood of false discoveries. Hypotheses, multiplicity adjustments, confidence intervals, and sample-size reasoning should be reported. Statistical significance does not establish practical importance; effect size, uncertainty, cost, and operational consequences are also necessary.

When randomization is impossible, quasi-experimental approaches may be used. Difference-in-differences compares changes over time between treated and comparison groups, relying on a credible parallel-trends assumption. Regression discontinuity uses a threshold-based assignment rule and estimates effects near the cutoff. Instrumental-variable methods require an instrument that affects treatment, affects the outcome only through treatment, and is sufficiently relevant. Matching and weighting can improve comparability but cannot automatically remove unmeasured confounding.

Causal diagrams, such as directed acyclic graphs, help identify confounders, mediators, colliders, and inappropriate adjustment variables. Controlling for a mediator may remove part of the effect of interest; conditioning on a collider can create bias. Doctoral researchers should specify the estimand before selecting an estimator. They should also conduct sensitivity analysis to assess how strong unmeasured confounding would need to be to change the conclusion.

Computational Social Science and Digital Trace Data

Computational social science uses computational methods to study social behavior, institutions, communication, and collective processes. Digital trace data include platform interactions, search activity, mobility records, transaction histories, sensor observations, and online text. These sources enable analysis at scale and temporal resolution, but they are not transparent windows into society.

The first issue is construct validity. A “like” may indicate approval, politeness, strategic visibility, or accidental interaction. A geolocation record may reflect device behavior rather than a person’s actual activity. Automated sentiment scores can be influenced by dialect, context, irony, or domain-specific language. Researchers should validate measures against human judgments, established instruments, or behavioral criteria and should report known error patterns.

The second issue is selection. Platform users are not necessarily representative of the broader population, and observed activity is shaped by platform design, moderation, access, incentives, and algorithms. Large samples can therefore produce precise estimates of a biased population. Weighting, sensitivity analysis, comparison with external benchmarks, and explicit population statements are essential.

Network analysis introduces additional methodological concerns. Centrality measures describe structural position, not necessarily influence or importance. Homophily can be mistaken for social influence, and common exposure can be mistaken for contagion. Temporal data, longitudinal designs, and appropriate null models can help distinguish competing explanations, although observational network data rarely eliminate all ambiguity.

Natural-language processing can support large-scale analysis of discourse, but models inherit biases from training data and annotation schemes. Researchers should report corpus construction, annotation procedures, inter-rater agreement, model versions, performance across relevant groups, and the consequences of classification errors. De-identification is difficult for text because rare combinations of words, events, and locations may re-identify individuals.

Case Studies and Mixed-Methods Inquiry

Case study research is valuable when a phenomenon must be understood within its real-world context, especially when boundaries between technology, organization, and environment are unclear. A case may be a firm, public agency, platform, project, implementation episode, or ecosystem. A case study is not simply a descriptive report. It requires a research design, a logic of case selection, theoretically informed data collection, triangulation, and transparent analysis.

Single-case designs are appropriate for critical, revelatory, longitudinal, or theoretically significant cases. Multiple-case designs support replication logic: cases may be selected because similar conditions are expected to produce similar outcomes, or because contrasting conditions are expected to produce different outcomes. Interviews, documents, system logs, observations, and quantitative indicators can be combined. Researchers should maintain a chain of evidence linking raw material to codes, categories, patterns, and conclusions.

Mixed methods integrates qualitative and quantitative evidence in a single inquiry. In an explanatory sequential design, quantitative results are followed by qualitative investigation of unexpected patterns or mechanisms. In an exploratory sequential design, qualitative insights guide instrument or model development, followed by quantitative testing. Convergent designs collect both forms of evidence and compare or integrate them. Integration should occur at the level of research questions, sampling, analysis, or interpretation; merely placing interviews beside a regression does not constitute mixed methods.

Qualitative rigor involves credibility, dependability, confirmability, reflexivity, and a sufficiently documented analytic process. Coding should be conceptually justified, and rival explanations should be considered. Quantitative rigor involves measurement quality, identification assumptions, uncertainty, robustness, and appropriate generalization. Mixed-methods conclusions should state where findings converge, complement one another, or remain inconsistent.

Reproducible and Open Analytical Practice

Reproducibility is a property of a research process that allows another investigator to obtain the same or substantively equivalent result using the documented materials and procedures. Replicability may refer to obtaining consistent findings through a new study, while reproducibility concerns the original computational path. Both are important, but neither is guaranteed by placing code in a public repository.

A reproducible workflow separates raw, intermediate, and final data; records transformations; fixes or documents software versions; and automates analysis where possible. Version control should cover code, configuration, documentation, and, when appropriate, model artifacts. Containers and environment specifications reduce dependency problems. Data provenance records where data came from, when they were acquired, how they were altered, and who authorized access.

A strong research repository commonly includes a readme file, data dictionary, workflow instructions, licenses, ethical restrictions, computational requirements, tests, and a statement of limitations. Sensitive data should not be released merely in the name of openness. Controlled access, synthetic data, secure enclaves, statistical disclosure controls, and reproducible summaries may provide safer alternatives.

Reproducibility also requires analytical discipline. Researchers should avoid silent deletion of observations, undocumented feature selection, leakage from future information, and repeated tuning against the test set. Train, validation, and test data must be separated according to the deployment setting. In time-dependent systems, temporal splitting is often more realistic than random splitting. A model card or dataset statement can document intended use, performance conditions, limitations, and risks.

Theory Building, Validation, and Generalization

Theory in big-data analytics can explain relationships, mechanisms, processes, or design principles. Inductive theory building begins with observations and develops concepts and relationships; deductive research derives hypotheses from theory and tests them; abductive reasoning moves between empirical surprises and plausible explanations. Computational scale does not replace theoretical reasoning. It can reveal patterns, but patterns require interpretation and boundary conditions.

Construct validity asks whether the study measures or manipulates what it claims to measure. Internal validity concerns whether the observed relationship supports the proposed explanation rather than an alternative. External validity concerns transfer to other populations, settings, or times. Statistical conclusion validity concerns the appropriateness and precision of inference. In design science, additional questions include utility, efficacy, efficiency, robustness, and ethical acceptability.

Generalization may be statistical, analytical, theoretical, or transfer-oriented. A random sample can support statistical generalization under appropriate assumptions. A carefully selected case can support analytical generalization to a theoretical proposition. A design principle may transfer when its underlying mechanism and boundary conditions are present, even if the original context differs. Doctoral writing should state which form of generalization is intended.

Ethics, Governance, and Responsible Research

Big-data research creates ethical obligations because data can be persistent, linkable, inferential, and consequential. Ethical review should consider informed consent, privacy expectations, data minimization, security, secondary use, vulnerable groups, re-identification, fairness, and the possibility of harm from publication. Public availability does not automatically mean ethical unrestricted use.

Consent may be impracticable in some large-scale studies, but alternatives require justification, safeguards, and proportionality. Researchers should assess whether individuals could reasonably expect the data to be used for the proposed purpose. Institutional review, data-protection law, contractual restrictions, and sector-specific regulation may impose different requirements. Legal compliance is necessary but does not exhaust ethical responsibility.

Algorithmic fairness cannot be reduced to one universal metric. Equal error rates, demographic parity, calibration, and individual fairness may be incompatible under some conditions. Researchers must identify the relevant harm, affected groups, decision context, and governance objective. Fairness evaluation should include data collection, labeling, model development, deployment, monitoring, and avenues for contesting decisions.

Responsible governance includes role-based access, audit logs, retention schedules, incident response, model monitoring, documentation, human oversight, and clear accountability. Researchers should disclose conflicts of interest, funding constraints, limitations, and negative results. When publication could enable misuse, a proportionate risk assessment and responsible disclosure process are warranted.

Illustrative Case: Predictive Maintenance in a Public Transit System

Background

A metropolitan transit authority operates electric buses and collects vehicle telemetry, maintenance records, route information, weather observations, and driver reports. It commissions a doctoral project to determine whether predictive analytics can reduce service interruptions without increasing unnecessary part replacement.

Situation and Research Problem

The authority’s existing rules identify failures only after threshold violations. The proposed project aims to design a predictive-maintenance artifact and estimate its operational effect. A central risk is that historical maintenance records reflect the old policy: components not inspected or replaced were less likely to be recorded as defective. A model trained on these records may learn administrative practices rather than mechanical condition.

Research Design

The researcher adopts a sequential mixed-methods design. First, interviews with mechanics, planners, and drivers identify failure mechanisms, workflow constraints, and meaningful outcomes. Second, a design-science cycle produces a model using time-windowed telemetry and maintenance history. The model is compared with the existing rules and a simple statistical baseline. Evaluation includes precision at the maintenance capacity available, recall of failures within a defined horizon, calibration, latency, and subgroup performance across vehicle models and depots.

For causal evaluation, the authority randomizes eligible vehicles by depot or matched vehicle blocks to receive model-assisted maintenance recommendations or continue with existing practice. The primary outcome is service-disruption hours per vehicle-month; secondary outcomes include emergency repairs, planned maintenance hours, replacement cost, and false alarms. Assignment at the vehicle level is rejected because mechanics at the same depot may apply recommendations across treatment conditions, creating contamination.

Analysis and Governance

The researcher uses temporal validation to prevent future telemetry from entering historical predictions. Missingness is analyzed because a missing sensor reading may indicate a technical problem rather than random absence. Interviews are revisited after the trial to explain why recommendations were accepted or overridden. The analysis plan defines the estimand, treatment period, exclusions, and handling of vehicle downtime before results are inspected.

Because telemetry can indirectly reveal driver behavior, access is restricted and individual performance is not reported. The system is positioned as decision support rather than automatic disciplinary surveillance. Model documentation describes known failure modes, confidence limits, and escalation procedures.

Outcome and Learning Points

Suppose the evaluation finds a moderate reduction in service disruptions but no reduction in total maintenance cost because the model generates too many low-value alerts for one vehicle class. The appropriate conclusion is not that predictive maintenance universally succeeds or fails. Rather, the evidence supports a conditional claim: the artifact improves reliability under specified data quality and maintenance-capacity conditions, while its economic value depends on alert precision and workflow integration.

The case demonstrates that technical performance, causal impact, organizational adoption, and ethical acceptability must be evaluated together. It also shows why a doctoral contribution may include a validated design principle—such as combining temporal validation with capacity-constrained evaluation—rather than merely a model with a high predictive score.

Common Failure Modes and Best Practices

Several errors recur in doctoral big-data projects. Researchers may begin with a convenient dataset and retrofit a question; confuse correlation with causation; report only aggregate accuracy; use leaked information; ignore class imbalance; treat a platform sample as representative; overfit through repeated testing; or present a prototype demonstration as evidence of real-world impact. Other failures are organizational: inadequate stakeholder engagement, unclear ownership of data, weak documentation, and no plan for deployment or withdrawal.

Best practice is to establish a research protocol early, conduct a literature-based conceptual analysis, define the unit of analysis and estimand, justify data inclusion rules, maintain a decision log, compare against credible baselines, and evaluate robustness under plausible changes in data and context. Researchers should separate development from final evaluation, report uncertainty and negative findings, involve domain experts without allowing them to obscure methodological weaknesses, and treat ethics as part of design rather than an approval form completed at the end.

Emerging research directions include foundation models for scientific analysis, privacy-preserving computation, federated learning, causal machine learning, synthetic data, automated experiment design, data-centric AI, and continuous monitoring of model drift. These developments create opportunities but also intensify questions about provenance, energy use, intellectual property, auditability, and concentration of computational resources. Future doctoral work will increasingly need to combine statistical and computational competence with institutional, legal, and ethical analysis.

Chapter Summary

Rigorous doctoral research in big-data analytics requires alignment among research questions, philosophical assumptions, evidence, methods, and claims. Design science supports the creation and evaluation of useful artifacts, but its contribution depends on generalizable design knowledge and credible evaluation. Experiments and quasi-experiments support causal reasoning when interventions, counterfactuals, assumptions, and uncertainty are clearly specified. Computational social science offers powerful analysis of digital traces while demanding careful attention to construct validity, selection, network dependence, and algorithmic bias.

Case studies explain technology in context, and mixed methods integrate numerical patterns with mechanisms, meanings, and organizational processes. Reproducible analytics depends on provenance, versioning, environment control, transparent workflows, and responsible data access. Theory building requires explicit constructs, mechanisms, boundary conditions, and a defensible form of generalization. Across all approaches, ethical governance, privacy, fairness, accountability, and disclosure of limitations are central to scholarly quality rather than peripheral concerns.

Key Terms

Construct: An abstract concept that a study seeks to define and measure.

Operationalization: The process of translating a construct into observable indicators or procedures.

Unit of analysis: The entity about which a research claim is made.

Estimand: The precisely defined causal or descriptive quantity a study seeks to estimate.

Design science: Research that creates and evaluates artifacts to produce actionable and generalizable design knowledge.

Artifact: A designed method, model, system, architecture, or process created to address a problem.

Counterfactual: The outcome that would have occurred for the same unit under an alternative condition.

Confounder: A variable associated with both an exposure and an outcome that can distort their relationship.

Internal validity: The credibility of a study’s explanation for the observed result.

External validity: The extent to which findings transfer beyond the studied setting.

Construct validity: The degree to which an operational measure represents the intended construct.

Computational social science: The use of computational data and methods to study social behavior and institutions.

Digital trace data: Records generated through online, mobile, transactional, or sensor-mediated activity.

Mixed methods: An integrated research design combining qualitative and quantitative approaches.

Reproducibility: The ability to obtain the same computational result using documented materials and procedures.

Data provenance: Documentation of a dataset’s origin, transformations, access, and history.

Causal inference: Reasoning about the effect of an intervention relative to a counterfactual.

Algorithmic fairness: The evaluation and governance of unequal impacts or error patterns across groups.

Boundary condition: A circumstance that specifies when a theory, relationship, or artifact is expected to apply.

Triangulation: The deliberate comparison of evidence from different sources, methods, or investigators.

Review Questions

Short Answer Questions

1. Why is a large dataset insufficient evidence of research quality?

2. Define construct validity and provide one big-data example.

3. What distinguishes an artifact from a design-science contribution?

4. Explain the difference between reproducibility and replicability.

5. Why can public digital trace data still raise ethical concerns?

Long Answer Questions

1. Explain how a doctoral researcher can align a research question, epistemological position, method, and contribution claim.

2. Discuss the major stages of a design-science research cycle and the evaluation criteria appropriate for analytical artifacts.

3. Compare randomized experiments, difference-in-differences, and regression discontinuity as approaches to causal inference.

4. Explain the methodological and ethical challenges of using platform data in computational social science.

5. Discuss how mixed methods and case study research can reveal organizational mechanisms that model performance metrics cannot capture.

Analytical / Application Questions

1. A fraud-detection model has high accuracy but performs poorly for a small customer group. Design an evaluation that addresses predictive validity, fairness, calibration, and operational consequences.

2. A university wants to test whether an AI writing assistant improves student learning, but students in the same class share materials. Propose an experimental design and identify threats to causal validity.

3. Develop a reproducible research plan for a streaming anomaly-detection study using sensitive industrial data. Include provenance, validation, access control, and dissemination safeguards.

22. Multiple Choice Questions with Answers

Question 1

A doctoral researcher claims that a new predictive model reduces equipment failures because it has higher accuracy than the existing model. What is the principal methodological problem?

A. Accuracy can never be used to evaluate predictive models.

B. Predictive performance alone does not establish a causal effect on equipment failures.

C. Causal claims require qualitative interviews only.

D. Existing models should not be used as baselines.

Correct Answer: B. Predictive performance alone does not establish a causal effect on equipment failures.

Question 2

Which design-science activity most directly supports the claim that an artifact works in its intended organizational setting?

A. Changing the model repeatedly without recording the changes

B. Demonstrating the artifact on an artificial dataset only

C. Evaluating the artifact against credible baselines using context-relevant outcomes

D. Reporting the complexity of the algorithm without testing it

Correct Answer: C. Evaluating the artifact against credible baselines using context-relevant outcomes

Question 3

A study uses social-media users to estimate the political opinions of an entire country. Which threat is most directly relevant?

A. Selection and representativeness bias

B. Hardware latency

C. Container incompatibility

D. Randomization contamination

Correct Answer: A. Selection and representativeness bias

Question 4

In an online experiment, users influence one another across treatment conditions. Which assumption is threatened?

A. Construct reliability

B. No interference between units

C. External validity of a case study

D. Software version control

Correct Answer: B. No interference between units

Question 5

Why is temporal splitting often preferable to random splitting when evaluating a system deployed over time?

A. It guarantees that the model will be fair.

B. It removes the need for a validation set.

C. It better represents prediction using past data to forecast future cases.

D. It always produces higher accuracy.

Correct Answer: C. It better represents prediction using past data to forecast future cases.

Question 6

Which statement best describes mixed-methods integration?

A. Adding an interview appendix after completing a quantitative study

B. Using qualitative and quantitative evidence to address connected research questions

C. Replacing statistical analysis with stakeholder opinions

D. Using two software packages on the same dataset

Correct Answer: B. Using qualitative and quantitative evidence to address connected research questions

Question 7

A researcher adjusts for a variable that lies on the pathway between an intervention and its outcome, although the goal is to estimate the total effect. What is the likely consequence?

A. Part of the total effect may be removed.

B. Randomization is automatically strengthened.

C. The outcome becomes perfectly measured.

D. External validity is guaranteed.

Correct Answer: A. Part of the total effect may be removed.

Question 8

Which practice most strongly improves computational reproducibility?

A. Keeping preprocessing decisions in memory rather than documenting them

B. Reporting only the final accuracy score

C. Versioning code and specifying the computational environment

D. Deleting intermediate data and logs

Correct Answer: C. Versioning code and specifying the computational environment

Question 9

A digital-text dataset is technically de-identified, but rare combinations of words and events can identify individuals. What issue does this illustrate?

A. Re-identification risk

B. Parallel-trends validity

C. Treatment randomization

D. Model calibration

Correct Answer: A. Re-identification risk

Question 10

In the transit predictive-maintenance case, why was randomization by depot or matched vehicle block preferred to individual vehicle randomization?

A. It guarantees that all vehicles have identical failure rates.

B. It reduces the risk that mechanics apply recommendations across treatment conditions.

C. It eliminates the need to measure outcomes.

D. It converts a predictive study into a case study.

Correct Answer: B. It reduces the risk that mechanics apply recommendations across treatment conditions.