Chapter 14: 14. Epistemology and Theory Development in IT Research

1. Introduction

Information technology research is not only concerned with building systems or measuring outcomes. It also asks deeper questions: What counts as knowledge? How can claims about technologies be justified? When does an observed relationship represent causation rather than coincidence? How should researchers study phenomena that are simultaneously technical, organizational, social, and ethical?

These questions are epistemological. Epistemology concerns the nature, sources, limits, and justification of knowledge. In information technology (IT), epistemological choices influence the research problem, the formulation of concepts, the selection of methods, the interpretation of evidence, and the type of contribution claimed. A researcher investigating algorithmic decision-making may conduct an experiment to estimate causal effects, interpret users’ experiences through interviews, critique structural inequalities reproduced by technology, or design and evaluate a new artifact. Each approach can be rigorous, but each rests on different assumptions about reality and knowledge.

This chapter examines five influential research paradigms: positivism, interpretivism, critical theory, pragmatism, and design science. It then explains theory development, conceptual frameworks, causality, validity, measurement, reproducibility, mixed-method evidence, and defensible doctoral contributions. The central argument is that methodological quality depends on coherence: the research question, philosophical assumptions, theory, evidence, analysis, and claims must fit together.

2. Learning Objectives

By the end of this chapter, doctoral researchers should be able to:

1. Explain epistemology and distinguish major research paradigms in IT.

2. Compare positivist, interpretive, critical, pragmatic, and design-science approaches.

3. Develop constructs, propositions, hypotheses, theories, and conceptual frameworks.

4. Assess causal claims, measurement quality, validity, reliability, and reproducibility.

5. Integrate quantitative and qualitative evidence through defensible mixed-method designs.

6. Identify and justify theoretical, empirical, methodological, design, and practical contributions.

7. Evaluate ethical, political, and governance implications of IT knowledge production.

3. Meaning and Concept

3.1 Epistemology and ontology

Ontology concerns what exists and what the nature of reality is. An ontological position may treat organizational performance as an objectively measurable condition, or it may view performance as a socially negotiated interpretation shaped by stakeholders. Epistemology concerns how researchers can know about that reality. A positivist epistemology generally emphasizes observation, measurement, regularity, and testing. An interpretive epistemology emphasizes meaning, context, language, and situated understanding.

Methodology is the logic connecting philosophical assumptions to research procedures. Methods are the specific techniques used to collect and analyze evidence, such as surveys, experiments, interviews, ethnography, archival analysis, simulation, or artifact evaluation. A method should not be selected merely because it is familiar. It should be selected because it can answer the research question under the chosen epistemological assumptions.

3.2 Theory and conceptual framework

A theory is a systematic set of concepts and relationships that explains, predicts, interprets, or guides action regarding a phenomenon. A theory is more than a list of variables. It specifies how and why concepts are connected, under what conditions relationships hold, and what evidence would support or challenge the explanation.

A conceptual framework is an organized representation of the key concepts and presumed relationships in a study. It may be derived from existing theories, prior empirical findings, professional knowledge, or a combination of sources. A framework guides inquiry but is not automatically a theory. A framework becomes theoretically stronger when its constructs are clearly defined, relationships are justified, boundary conditions are identified, and explanatory mechanisms are articulated.

3.3 Research contribution

A doctoral contribution is a defensible addition to knowledge. It may refine an existing theory, explain a previously unclear mechanism, identify a boundary condition, develop a measurement instrument, create and evaluate an artifact, integrate fragmented evidence, or expose an important assumption or inequality. Novelty alone is insufficient. A contribution must also be significant, credible, transparent, and relevant to an identifiable scholarly or practical problem.

4. Fundamental Principles

4.1 Paradigmatic coherence

Research should demonstrate coherence among ontology, epistemology, research questions, methods, analysis, and claims. For example, a study that seeks statistical estimates of the average causal effect of a software intervention requires different evidence from a study seeking to understand how employees interpret that intervention. Neither design is universally superior.

4.2 Reflexivity

Researchers are part of the knowledge-production process. Their professional background, theoretical commitments, institutional interests, and relationship with participants can influence the study. Reflexivity requires explicit examination of these influences rather than pretending that they do not exist. In qualitative work, reflexivity may be recorded through field notes, positionality statements, and analytic memos. In quantitative work, it may involve preregistration, disclosure of analytical decisions, and sensitivity analysis.

4.3 Evidence must fit the claim

Evidence supports particular kinds of claims. A randomized experiment may provide strong evidence about an intervention under specified conditions, but it may provide limited insight into users’ interpretations or long-term institutional consequences. A rich case study may explain processes and meanings but cannot automatically establish population-level frequency. Researchers should avoid extending claims beyond the design’s evidential capacity.

4.4 Concepts require disciplined definition

Constructs such as trust, usability, resilience, digital maturity, fairness, and innovation are abstract. A construct definition states what the concept means and distinguishes it from related concepts. Operationalization specifies how it will be observed or measured. Conceptual ambiguity produces invalid comparisons and weak theory.

4.5 Transparency supports cumulative knowledge

A credible study reports decisions about sampling, instruments, coding, exclusions, missing data, model specifications, and analytic changes. Transparency allows readers to assess the reasoning and enables replication, reproduction, extension, or informed disagreement.

5. Major Components / Elements

5.1 Positivist paradigm

Positivism assumes that systematic observation and formal analysis can reveal regularities in an external reality. Common methods include experiments, surveys, econometric models, log analysis, and statistical hypothesis testing. Typical questions ask whether one factor affects another, how frequently a phenomenon occurs, or whether a model predicts an outcome.

Strengths include explicit measurement, comparability, estimation of uncertainty, and the ability to test rival explanations. Risks include reductionism, mistaking measurement for the phenomenon itself, neglecting context, and treating statistically significant associations as causal explanations.

5.2 Interpretive paradigm

Interpretivism views social and organizational reality as meaning-laden and contextually produced. Technologies acquire significance through human practices, institutional norms, language, and history. Methods include interviews, participant observation, ethnography, document analysis, narrative inquiry, and qualitative case studies.

Interpretive rigor depends on credible engagement with the field, rich contextual description, transparent interpretation, attention to disconfirming evidence, and a clear connection between data and claims. The objective is generally not universal prediction but understanding how actors construct and negotiate reality.

5.3 Critical paradigm

Critical research examines power, inequality, ideology, domination, and the distribution of benefits and harms. It asks who designs technology, whose interests are served, who is excluded, and how apparently neutral systems may reproduce social structures. Critical inquiry may draw on political economy, critical race theory, feminist theory, postcolonial theory, or labor studies.

Critical research is not simply research that criticizes a system. It links critique to historically and institutionally grounded analysis and often seeks emancipation or transformation. Researchers must nevertheless support normative claims with rigorous evidence and avoid replacing analysis with advocacy unsupported by data.

5.4 Pragmatic paradigm

Pragmatism begins with problems and consequences rather than a rigid commitment to one method or metaphysical position. It permits researchers to combine methods when doing so improves understanding or decision quality. Pragmatic research asks what works, for whom, in what circumstances, and with what consequences.

Pragmatism is especially useful for complex IT problems requiring both outcome measurement and contextual explanation. However, methodological flexibility is not an excuse for arbitrary method selection. The researcher must explain how different forms of evidence answer complementary questions and how conflicts between findings are handled.

5.5 Design-science paradigm

Design science treats the purposeful creation and evaluation of artifacts as a central means of producing knowledge. Artifacts may include algorithms, architectures, methods, governance models, processes, or decision-support systems. Design-science research differs from routine engineering because it abstracts design knowledge into principles, constructs, models, methods, or validated technological rules.

A design-science study identifies a relevant problem, defines objectives, develops an artifact, evaluates its utility and limitations, and communicates both the artifact and the generalizable knowledge derived from the process. Evaluation may address functionality, performance, usability, security, fairness, scalability, or organizational fit.

5.6 Causality

Causal inference concerns whether changing one condition would change an outcome. A temporal association is not enough. Causal reasoning requires attention to confounding, selection, reverse causality, mediation, moderation, and interference among units. Randomization can balance observed and unobserved confounders in expectation, while observational studies require design and analytical strategies such as matching, fixed effects, instrumental variables, regression discontinuity, difference-in-differences, or causal graphs.

A causal claim should identify the treatment, outcome, population, time frame, intervention mechanism, and assumptions. “The system improved performance” is weaker than “Among comparable service teams, deployment of the decision-support tool increased completed cases per employee over six months, conditional on training and staffing assumptions.”

5.7 Validity and reliability

Internal validity concerns whether the study supports its proposed explanation. External validity concerns the transferability or generalizability of findings beyond the study setting. Construct validity concerns whether operational measures represent the intended concepts. Statistical conclusion validity concerns whether the analysis supports the stated relationship, considering power, uncertainty, model assumptions, and multiplicity.

Reliability concerns consistency of measurement or procedure. A reliable measure can still be invalid if it consistently measures the wrong construct. Qualitative quality is often discussed through credibility, dependability, confirmability, and transferability rather than reliability alone.

5.8 Reproducibility and replicability

Reproducibility generally refers to obtaining consistent results using the same data, code, and procedures. Replicability refers to obtaining consistent findings in a new study or setting. Computational notebooks, version control, documented data pipelines, preregistration, open materials, and careful metadata improve reproducibility. Privacy, security, licensing, and commercial confidentiality may limit full openness, but limitations should be documented and alternatives, such as synthetic data or controlled access, should be considered.

6. Process / Framework / Methodology

A defensible theory-development process can be organized into the following stages.

Stage 1: Define the phenomenon and problem

Specify the phenomenon, affected stakeholders, context, and practical or theoretical deficiency. A broad topic such as “artificial intelligence in healthcare” must be narrowed to a phenomenon such as clinicians’ calibration of trust in diagnostic recommendations.

Stage 2: Review and map existing knowledge

A systematic or scoping review identifies established concepts, competing explanations, methods, contradictions, and neglected populations. Literature review should be analytical rather than merely chronological. A literature matrix can map constructs, mechanisms, contexts, measures, findings, and limitations.

Stage 3: Select a paradigm and formulate questions

Questions should reflect the intended contribution. Examples include: What is the average effect of a transparency intervention? How do employees interpret automated monitoring? How does algorithmic allocation reproduce disadvantage? What design principles improve secure collaboration? Paradigm choice should follow the problem, while acknowledging that researchers may use multiple paradigms in a carefully integrated program.

Stage 4: Develop concepts and relationships

Define constructs, identify antecedents and outcomes, propose mechanisms, and state boundary conditions. A conceptual framework may use boxes and arrows, but the written rationale is more important than the diagram. Each arrow should represent a theoretically justified relationship rather than a convenient visual connection.

Stage 5: Choose evidence and design

Decide whether the study requires experimental, observational, interpretive, critical, design-oriented, or mixed evidence. Establish the unit of analysis, sampling logic, data sources, time horizon, ethical safeguards, and analysis plan.

Stage 6: Operationalize and pilot

Develop instruments, coding schemes, intervention protocols, or artifact specifications. Pilot testing can reveal ambiguous survey items, unreliable sensors, unrealistic workflows, or unanticipated ethical risks. Revise transparently and distinguish exploratory from confirmatory analysis.

Stage 7: Analyze and integrate

Use analysis appropriate to the data and claim. Quantitative analysis may include estimation, model diagnostics, uncertainty intervals, robustness checks, and sensitivity analysis. Qualitative analysis may include iterative coding, comparison across cases, memoing, negative-case analysis, and theory refinement. Mixed-method integration can occur through connecting, building, merging, or embedding designs.

Stage 8: Evaluate rival explanations and limitations

Ask what else could explain the findings. Consider selection effects, measurement error, history, maturation, common-method bias, researcher influence, implementation fidelity, and contextual change. Limitations should specify how they affect interpretation rather than appearing as a generic list.

Stage 9: State the contribution and boundary conditions

Explain what is new, how it was established, why it matters, and where it may not apply. A contribution should be linked directly to evidence and should not exceed the scope of the design.

7. Important Theories and Models

7.1 Theory-building components

Theory development commonly uses constructs, definitions, propositions, hypotheses, mechanisms, and boundary conditions. A proposition is a theoretically reasoned statement about a relationship. A hypothesis is a testable statement, often specifying direction and measurable variables. A mechanism explains how or why a relationship occurs. A boundary condition identifies when the relationship changes or fails.

7.2 Technology acceptance and use models

Technology acceptance models explain intentions or behaviors concerning technology use through perceptions such as usefulness, ease of use, social influence, facilitating conditions, and habit. Their value lies not in mechanically applying a questionnaire but in determining whether the constructs fit the technology, population, and research question. A doctoral study might extend such a model by examining institutional trust or perceived professional identity as a mechanism.

7.3 Sociotechnical perspectives

Sociotechnical theory treats technical systems and social arrangements as interdependent. Outcomes arise from the configuration of technology, tasks, roles, skills, incentives, norms, and institutional structures. This perspective helps explain why technically successful implementations may fail in practice when workflows, authority, or accountability are misaligned.

7.4 Structuration perspectives

Structuration theory examines how human action and institutional structures recursively shape one another. Technologies embody rules and resources, but users may appropriate, modify, or resist them. The perspective is useful for studying digital platforms, enterprise systems, and organizational change over time.

7.5 Institutional and affordance perspectives

Institutional theory explains how regulatory, normative, and cultural pressures influence organizational technology choices. Affordance theory focuses on action possibilities arising from the relationship between an artifact and a user in a context. An affordance is not simply a feature; it depends on capabilities, goals, interpretation, and conditions of use.

7.6 Design theories and design principles

Design theories connect problem conditions, artifact features, mechanisms, and expected outcomes. A design principle might state that an explanation interface should expose uncertainty and relevant evidence to support calibrated human judgment, rather than merely increasing user confidence. Such principles require evaluation and should specify context and trade-offs.

8. Practical Applications

Epistemological awareness improves research on cybersecurity, digital transformation, health information systems, artificial intelligence, data governance, software engineering, and platform work. A cybersecurity experiment may test whether training reduces phishing susceptibility, while interviews may explain why employees ignore warnings. A critical study may investigate which groups bear the costs of surveillance. A design-science project may create a usable and privacy-preserving authentication method. Together, these approaches can produce more complete knowledge than any one method alone.

For managers, the practical implication is that evidence should be matched to decisions. A dashboard showing correlation cannot by itself justify a costly intervention. A successful pilot in one department may not transfer to another. Employee resistance may reflect legitimate concerns about autonomy rather than a simple training deficit. Theory helps managers understand why an intervention works, fails, or produces unintended consequences.

9. Examples

Consider a university introducing an AI advising tool. A positivist study could compare retention rates before and after implementation, preferably with a suitable comparison group. An interpretive study could examine how students and advisers understand machine-generated recommendations. A critical study could investigate whether the tool disadvantages first-generation or multilingual students. A pragmatic mixed-method study could combine outcome data with interviews to determine both effectiveness and implementation conditions. A design-science study could develop an explanation and escalation mechanism, then evaluate its accuracy, usability, fairness, and institutional fit.

These studies do not ask identical questions. Treating them as competitors obscures the possibility that a complex IT problem has multiple dimensions requiring different forms of evidence.

10. Technology and Digital Applications

Digital research environments affect both the object of study and the research process. Platform logs provide fine-grained behavioral traces but may omit motives and offline activity. Machine-learning models can support prediction, coding, and simulation, but they introduce risks of opacity, data leakage, automation bias, and unstable performance across populations. Digital twins and simulation environments enable scenario analysis, yet their conclusions depend on assumptions about system boundaries and data-generating processes.

Software repositories, version-control systems, reproducible workflows, and automated testing can strengthen methodological transparency. However, tools do not guarantee rigor. A sophisticated model with biased data or poorly defined outcomes remains epistemically weak. Researchers should document data provenance, preprocessing, model versions, hyperparameters, evaluation metrics, and known failure modes.

11. Strategic / Managerial Importance

Organizations increasingly make strategic decisions on the basis of analytics, benchmarks, and algorithmic recommendations. Weak research can therefore create financial, operational, legal, and reputational risk. Managers need to distinguish descriptive evidence from causal evidence, short-term gains from sustainable outcomes, and user acceptance from ethical legitimacy.

A theoretically informed evidence strategy supports investment prioritization, implementation planning, risk management, and responsible innovation. It also encourages managers to involve affected stakeholders, test assumptions, monitor distributional effects, and revise systems when evidence changes. Doctoral research can contribute by providing validated models, implementation principles, measurement instruments, governance mechanisms, or explanations of organizational conditions.

12. Detailed Case Study

The following is a fictional illustrative case.

Background

MetroCare, a regional healthcare network, introduced a machine-learning system to prioritize patients for follow-up support. The system used clinical and administrative data to estimate risk of readmission. Executives expected improved resource allocation and lower readmission rates.

Problem

After six months, overall readmission rates had not changed. Clinicians reported that they often ignored the rankings, while community advocates alleged that the system undervalued patients with irregular access to care. Management initially proposed additional user training.

Analysis

A purely positivist analysis of system accuracy would not explain adoption or equity concerns. The research team therefore used a pragmatic mixed-method design. Quantitative analysis compared outcomes across clinics, examined calibration by demographic and access-related characteristics, and assessed whether use of the tool was associated with improved follow-up. Interviews and observations examined how clinicians interpreted risk scores, fitted them into workflows, and negotiated accountability.

The analysis identified three mechanisms. First, the model performed less well for patients whose records contained fragmented information. Second, the interface displayed rankings without uncertainty or reasons, reducing clinician trust. Third, clinics differed in staffing and referral capacity, so a high-risk score did not imply that action was feasible. The problem was therefore not simply user resistance or model accuracy; it was a sociotechnical misalignment.

Strategy / Solution

MetroCare revised the data pipeline, established subgroup performance monitoring, displayed uncertainty and relevant explanatory factors, and introduced a clinician review and override process. The organization also changed the objective from maximizing predicted risk to prioritizing feasible and equitable support. A phased evaluation combined controlled implementation across clinics with interviews and workflow observation.

Outcome

The revised system improved follow-up completion in clinics with adequate staffing and reduced performance disparities for several groups. It had little effect where community referral capacity was limited. Clinician trust improved when explanations and escalation procedures were introduced, but some staff continued to reject automated prioritization for complex cases.

Learning Points

The case demonstrates that predictive performance is not equivalent to organizational value. Causal effects depend on implementation conditions. Interpretive evidence can reveal meanings and workflow constraints that logs cannot show. Critical analysis is necessary when benefits and errors are unevenly distributed. A defensible contribution would not claim that the system universally improves care; it would specify the mechanisms and boundary conditions under which the intervention produces benefit.

13. Advantages / Benefits

A pluralistic approach to IT research offers several benefits. It improves problem definition by recognizing technical, human, institutional, and political dimensions. It supports stronger explanations by combining measurement with contextual understanding. It encourages responsible innovation by making power, fairness, and unintended consequences visible. Design science connects knowledge to usable artifacts, while causal methods improve intervention decisions. Transparent and reproducible practices strengthen trust and allow cumulative scholarship.

14. Limitations / Challenges

Paradigms can become rigid intellectual communities that dismiss alternative evidence. Mixed methods may produce superficial integration if researchers merely place quantitative and qualitative results in separate sections. Theories may be imported without conceptual fit, and measurement instruments may be reused without validating them in new contexts. Causal inference is difficult when interventions evolve, units interact, or outcomes are delayed. Reproducibility may be constrained by proprietary systems, privacy requirements, changing platforms, and inaccessible data. Design-science projects may overemphasize artifact novelty and underdevelop theoretical abstraction.

Doctoral researchers also face practical constraints: limited time, restricted access, complex stakeholder relationships, and pressure to produce positive results. These constraints make scope control, preregistration where appropriate, transparent reporting, and early methodological consultation essential.

15. Ethical and Governance Considerations

Research involving digital data must address informed consent, privacy, confidentiality, data minimization, security, and lawful processing. Researchers should consider whether participants understand secondary uses of their data and whether vulnerable groups face disproportionate risks. Ethical review is necessary but not sufficient; responsible judgment continues throughout the project.

Algorithmic research requires attention to fairness, explainability, contestability, accountability, and the possibility of automation bias. Researchers should not assume that removing sensitive attributes eliminates discrimination, because proxy variables and institutional processes may reproduce inequality. Governance should define responsibility for data quality, model updates, overrides, incident response, and redress.

Critical and participatory research should avoid extracting knowledge from communities without reciprocal benefit. Researchers should negotiate access, protect dissenting participants, and report findings without exposing individuals or groups to foreseeable harm. Open science must be balanced with privacy, security, intellectual property, and indigenous or community data sovereignty.

16. Current and Emerging Trends

Several trends are reshaping theory development in IT. Generative AI is being used for coding, literature discovery, simulation, and artifact construction, creating new questions about authorship, verification, data provenance, and hallucinated evidence. Causal machine learning seeks heterogeneous treatment effects and policy-relevant predictions, but it does not remove the need for causal assumptions. Digital trace research is expanding rapidly, while concerns about representativeness and platform dependence are increasing.

Responsible AI research increasingly combines technical evaluation with social impact assessment. Reproducible computational research is becoming more systematic through containers, workflow automation, data versioning, and registered reports. Living theories and continuously updated evidence syntheses may become important in rapidly changing technological environments. Participatory and co-design approaches are also gaining influence as researchers seek to include affected communities in problem definition and artifact evaluation.

17. Best Practices

1. Start with a precise phenomenon and problem rather than a preferred method.

2. State ontological and epistemological assumptions in accessible language.

3. Define constructs and distinguish them from indicators.

4. Use theory to explain mechanisms, not merely to decorate a framework.

5. Match causal claims to design strength and state assumptions explicitly.

6. Pilot instruments, interventions, and artifact evaluations.

7. Separate exploratory discovery from confirmatory testing.

8. Use triangulation to compare complementary or conflicting evidence.

9. Report negative cases, null findings, uncertainty, and implementation failures.

10. Preserve analysis code, materials, decisions, and version histories where ethical and lawful.

11. Evaluate artifacts for utility, safety, fairness, usability, and contextual fit.

12. State boundary conditions and avoid universal claims from local evidence.

13. Engage affected stakeholders in ethically appropriate ways.

14. Link the claimed contribution directly to the evidence produced.

18. Future Scope

Future IT research will increasingly require theories capable of explaining adaptive, data-intensive, and networked systems. Researchers will need to study interactions among humans, autonomous agents, institutions, and infrastructures rather than treating technology as an isolated variable. Longitudinal designs will be especially important because digital systems evolve through feedback, learning, regulation, and user appropriation.

There is also scope for stronger theories of algorithmic accountability, human-AI collaboration, platform governance, digital inequality, sustainable computing, and resilience in critical infrastructures. Doctoral researchers can contribute by combining causal inference with interpretive and critical explanation, creating evaluation standards for emerging artifacts, developing context-sensitive measures, and producing design principles that address both performance and public values.

19. Chapter Summary

Epistemology shapes what IT researchers regard as knowledge and how they justify claims. Positivism emphasizes measurement and regularities; interpretivism emphasizes situated meaning; critical research examines power and inequality; pragmatism selects methods according to consequential problems; and design science develops and evaluates artifacts as vehicles for knowledge. These paradigms are distinct, but carefully justified research programs may combine their insights.

Strong theory development requires clear constructs, mechanisms, propositions, hypotheses, boundary conditions, and conceptual coherence. Causal claims require more than association, while validity, reliability, transparency, and reproducibility determine the credibility of evidence. Mixed methods are valuable when integration is purposeful. A doctoral contribution must be novel, significant, credible, and appropriately bounded. Ultimately, rigorous IT research connects philosophical clarity with methodological discipline and ethical responsibility.

20. Key Terms

1. Epistemology — The study of the nature, sources, limits, and justification of knowledge.

2. Ontology — Assumptions about what exists and the nature of reality.

3. Paradigm — A broad set of philosophical assumptions guiding research.

4. Methodology — The logic connecting research assumptions, questions, and methods.

5. Construct — An abstract concept used in theorizing, such as trust or resilience.

6. Operationalization — The process of specifying how a construct will be observed or measured.

7. Theory — An organized explanation of concepts, relationships, mechanisms, and conditions.

8. Conceptual framework — A structured representation of concepts and presumed relationships in a study.

9. Proposition — A theoretically reasoned statement about a relationship.

10. Hypothesis — A testable statement concerning expected empirical relationships.

11. Mechanism — The process explaining how or why a relationship occurs.

12. Boundary condition — A circumstance limiting when a theoretical relationship applies.

13. Causality — A relationship in which changing one condition produces a change in an outcome under specified assumptions.

14. Construct validity — The degree to which a measure represents the intended concept.

15. Internal validity — The credibility of an explanation within a study.

16. External validity — The extent to which findings apply beyond the study setting.

17. Reproducibility — The ability to obtain consistent results using the same data and procedures.

18. Replicability — The ability to obtain consistent findings in a new study.

19. Triangulation — The use of multiple sources, methods, theories, or investigators to examine a phenomenon.

20. Design science — Research that creates and evaluates artifacts to produce generalizable design knowledge.

21. Review Questions

Short Answer Questions

1. What is the difference between ontology, epistemology, methodology, and method?

2. How does interpretivism differ from positivism in its treatment of social reality?

3. Why is a conceptual framework not automatically a theory?

4. Distinguish reproducibility from replicability.

5. What is a boundary condition, and why is it important in theory development?

Long Answer Questions

1. Compare positivist, interpretive, critical, pragmatic, and design-science paradigms in terms of assumptions, typical questions, methods, strengths, and limitations.

2. Explain how a doctoral researcher can move from a practical IT problem to a theoretically defensible contribution.

3. Discuss the relationship among measurement, construct validity, reliability, and causal inference.

4. Explain how mixed-method research can integrate quantitative and qualitative evidence without becoming methodologically incoherent.

5. Evaluate the ethical and governance challenges involved in researching algorithmic decision-support systems.

Analytical / Application Questions

1. A company reports that introducing an analytics dashboard increased employee productivity by 15 percent. Design a study to assess whether the dashboard caused the increase, and identify at least three rival explanations.

2. An automated recruitment system has high average accuracy but produces lower selection rates for a minority group. Analyze the issue using positivist, interpretive, and critical perspectives, and propose an appropriate research design.

3. Develop a conceptual framework for adoption of a cybersecurity behavior in a large organization. Define the constructs, propose mechanisms and boundary conditions, and explain how you would evaluate the framework.

22. Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes a conceptual framework from a fully developed theory?

A. A conceptual framework can organize concepts and relationships, whereas a theory also explains mechanisms and conditions.

B. A conceptual framework always uses statistical variables, whereas a theory uses only qualitative concepts.

C. A conceptual framework is empirical, whereas a theory is never tested.

D. There is no meaningful distinction between them.

Correct Answer: A. A conceptual framework can organize concepts and relationships, whereas a theory also explains mechanisms and conditions.

Question 2

A researcher wants to understand how clinicians interpret an AI recommendation system within their daily workflow. Which paradigm is most directly aligned with this purpose?

A. Positivism

B. Interpretivism

C. Critical realism only

D. Pure experimentalism

Correct Answer: B. Interpretivism

Question 3

Why is a statistically significant association not necessarily evidence of causality?

A. Statistical tests cannot use numerical data.

B. Associations may result from confounding, reverse causality, or selection effects.

C. Causality can only be studied through interviews.

D. Significance testing proves that all relationships are accidental.

Correct Answer: B. Associations may result from confounding, reverse causality, or selection effects.

Question 4

Which activity most directly improves computational reproducibility?

A. Removing all documentation to simplify the project

B. Changing the analysis after seeing results without recording the change

C. Preserving versioned code, data-processing steps, and analysis environments

D. Reporting only the findings that support the hypothesis

Correct Answer: C. Preserving versioned code, data-processing steps, and analysis environments

Question 5

What is the main purpose of a boundary condition in a theory?

A. To identify when or where a proposed relationship may not apply

B. To eliminate the need for empirical evidence

C. To guarantee that a hypothesis is true

D. To replace construct definitions

Correct Answer: A. To identify when or where a proposed relationship may not apply

Question 6

Which feature is central to design-science research?

A. Avoiding the creation of practical artifacts

B. Creating and evaluating an artifact while generating generalizable design knowledge

C. Using only historical documents

D. Treating user experience as irrelevant

Correct Answer: B. Creating and evaluating an artifact while generating generalizable design knowledge

Question 7

A reliable instrument may still be invalid when it:

A. Produces consistent measurements of the wrong construct

B. Has clearly defined items

C. Is tested in a pilot study

D. Reports measurement uncertainty

Correct Answer: A. Produces consistent measurements of the wrong construct

Question 8

What is a major benefit of pragmatic mixed-method research?

A. It permits methods to be combined when they answer complementary aspects of a problem.

B. It removes the need to justify methodological choices.

C. It guarantees that all findings will agree.

D. It restricts research to managerial opinions.

Correct Answer: A. It permits methods to be combined when they answer complementary aspects of a problem.

Question 9

Which research perspective most explicitly examines how technology may reproduce power differences and inequality?

A. Critical theory

B. Positivism

C. Descriptive statistics

D. Instrumental measurement theory

Correct Answer: A. Critical theory

Question 10

Which is the strongest example of a defensible doctoral contribution?

A. Claiming that a new system is innovative because it uses current technology

B. Repeating an existing survey without contextual justification

C. Identifying and validating a mechanism and boundary condition that refine an existing IT theory

D. Reporting a positive result while omitting limitations

Correct Answer: C. Identifying and validating a mechanism and boundary condition that refine an existing IT theory