Chapter 10: Design Science, Action Research, and Evaluation

1. Introduction

Information technology research is not limited to describing existing phenomena. It also seeks to create improved systems, methods, models, processes, and interventions that address important organizational and societal problems. Design science research and action research are two major approaches for achieving this purpose. Both are especially valuable in doctoral research because they connect theoretical knowledge with practical change.

Design science research focuses on the disciplined creation and evaluation of artifacts. An artifact may be a software platform, algorithm, reference architecture, process model, decision framework, method, or design theory. Action research focuses on solving a real problem through collaboration between researchers and practitioners while generating knowledge from the intervention. Participatory approaches extend this collaboration by involving users and affected communities in problem definition, design, implementation, and evaluation.

These approaches require more than building a technically functional product. A doctoral researcher must demonstrate that the problem is significant, the artifact or intervention is grounded in existing knowledge, the research process is rigorous, and the resulting contribution is useful and generalizable. Evaluation is therefore central. It establishes whether an artifact works, for whom it works, under which conditions it works, and what theoretical knowledge can be derived from its development and use.

2. Learning Objectives

By the end of this chapter, students should be able to:

1. Explain the meaning, purposes, and philosophical foundations of design science and action research.

2. Distinguish artifacts, interventions, design principles, reference models, and design theories.

3. Apply build–evaluate cycles to the development of IT solutions.

4. Design rigorous utility, quality, and impact evaluations.

5. Integrate participatory methods into artifact and intervention research.

6. Assess ethical, governance, and organizational issues in field-based research.

7. Develop a defensible doctoral research contribution from a practical IT problem.

3. Meaning and Concept

3.1 Design science research

Design science research (DSR) is a research paradigm concerned with the creation and systematic evaluation of artifacts intended to solve identified problems. Its central premise is that knowledge can be generated through purposeful design. Design is not treated as ad hoc engineering; it is a form of inquiry in which the researcher investigates how an artifact can produce desired outcomes in a particular context.

Common artifact types include constructs, models, methods, instantiations, architectures, algorithms, software systems, and organizational interventions. A construct provides concepts for describing a problem. A model represents relationships among concepts. A method specifies procedures for performing a task. An instantiation demonstrates that a model or method can be implemented in a working system. A design theory explains how and why a class of artifacts produces outcomes under particular conditions.

3.2 Action research

Action research is a collaborative and intervention-oriented methodology in which researchers and practitioners jointly diagnose a problem, plan action, implement change, observe consequences, and reflect on learning. The process is normally cyclical. Each cycle can improve the practical situation while refining theoretical understanding.

Action research differs from consultancy because it has an explicit research purpose, systematic data collection, transparent analysis, and a contribution to knowledge. It differs from conventional observational research because the researcher deliberately participates in changing the situation. Its validity depends on both practical improvement and credible explanation of how knowledge was produced.

3.3 Participatory research

Participatory research involves stakeholders in decisions about the research problem, requirements, design, implementation, and assessment. It is particularly important where technology affects vulnerable groups, public services, professional identity, or community resources. Participation may involve workshops, co-design, design games, interviews, prototypes, citizen panels, and collaborative interpretation of findings.

Participation is not merely asking users to test a finished product. It gives stakeholders meaningful influence over what is designed and how success is defined. Researchers must nevertheless manage unequal power, conflicting interests, and the possibility that participation becomes symbolic rather than substantive.

3.4 Evaluation

Evaluation is the systematic assessment of an artifact, intervention, or design proposition against explicit criteria. It can examine technical performance, usability, usefulness, organizational fit, adoption, safety, cost, equity, and long-term impact. A strong evaluation aligns its method with the research claim. For example, a claim about algorithmic accuracy requires performance testing, while a claim about organizational adoption may require longitudinal field research.

4. Fundamental Principles

Several principles guide rigorous design-oriented research.

Problem relevance. Research should address a real and significant problem. Significance may arise from economic loss, safety risks, social exclusion, regulatory obligations, environmental impact, or a substantial gap in scholarly knowledge.

Artifact-centered contribution. The artifact must embody knowledge. Simply producing software is insufficient unless the research explains its design rationale, novelty, operation, and contribution.

Rigor through existing knowledge. Design decisions should be informed by theories, prior studies, standards, domain knowledge, and established engineering practices. The researcher should show what is known, what is inadequate, and how the new design extends knowledge.

Iterative development. Requirements and solutions are refined through repeated cycles of design, implementation, testing, and reflection. Iteration should be documented rather than presented as an unexplained sequence of changes.

Demonstrable utility. The artifact must be shown to solve, reduce, or illuminate the target problem. Utility can be direct, such as improved response time, or indirect, such as better decision quality or increased transparency.

Context sensitivity. An artifact is rarely universally effective. Its outcomes depend on users, infrastructure, culture, regulations, resources, and task conditions. Researchers should specify boundary conditions rather than overgeneralize.

Communication. Results must be communicated to both academic and practitioner audiences. Technical design details, theoretical implications, implementation guidance, and limitations should be presented appropriately for each audience.

Reflexivity. Researchers should examine how their assumptions, relationships, authority, and participation influence the research process and findings.

5. Major Components / Elements

A design-oriented doctoral study normally contains the following elements.

Problem and context

The problem statement identifies the current condition, affected stakeholders, consequences, and evidence that the problem matters. Context includes organizational structure, technical environment, workflows, institutional rules, and relevant social conditions.

Objectives and requirements

Objectives state desired outcomes. Requirements translate those outcomes into functional, quality, ethical, and organizational expectations. Requirements may be elicited through interviews, observations, document analysis, process mining, surveys, and participatory workshops.

Knowledge base

The knowledge base includes theories, methods, standards, design patterns, empirical findings, and comparable artifacts. A systematic literature review is often used to establish the research gap and identify candidate design principles.

Artifact or intervention

The artifact is the designed solution or knowledge representation. It should have an explicit architecture, components, assumptions, inputs, outputs, and intended mechanism of action. In action research, the intervention may include new technology, revised procedures, training, governance arrangements, or a combination of these.

Design rationale

Design rationale explains why decisions were made, what alternatives were considered, and how decisions relate to requirements and theory. It is essential for research transparency and for converting project experience into transferable knowledge.

Evaluation strategy

The strategy defines evaluation questions, criteria, measures, participants, comparison conditions, data sources, and analysis methods. It should be planned early, because evaluation requirements often influence the artifact itself.

Contribution

Contributions may include a new artifact, design principles, a reference model, a method, empirical evidence, a design theory, or an explanation of how an intervention works in context. A contribution should be distinguished from implementation effort.

6. Process / Framework / Methodology

6.1 A design science process

A practical design science process consists of the following connected activities:

1. Problem identification and motivation. Establish the problem, affected stakeholders, consequences, and research opportunity.

2. Definition of objectives. Specify what a better solution should accomplish and identify measurable success criteria.

3. Design and development. Create the artifact using appropriate methods, technologies, and design principles.

4. Demonstration. Show that the artifact can be used to address one or more instances of the problem. Demonstration may involve a prototype, simulation, scenario, or operational deployment.

5. Evaluation. Assess quality, effectiveness, utility, and limitations using suitable empirical methods.

6. Communication. Report the artifact, process, findings, theoretical contribution, practical implications, and limitations.

These activities are not strictly linear. Evaluation may reveal that the problem was misunderstood, requiring a new design cycle. Demonstration may expose integration barriers, while stakeholder feedback may alter objectives.

6.2 Build–evaluate cycles

A build–evaluate cycle begins with a provisional design. The researcher develops a prototype or design representation, tests it against selected criteria, interprets the findings, and modifies the design. Early evaluations often focus on feasibility and technical correctness. Later evaluations examine usability, comparative performance, organizational fit, adoption, and impact.

A useful research record maintains a version history, evaluation log, decision register, and traceability matrix linking problems, requirements, design features, measures, and findings. This evidence allows examiners to see how knowledge emerged rather than only viewing the final artifact.

6.3 Action research cycles

A typical action research cycle includes:

• Diagnosing: jointly understand the current problem and its causes.
• Planning action: agree on an intervention, responsibilities, resources, and indicators.
• Taking action: implement the agreed change.
• Evaluating action: collect evidence about effects, experiences, and unintended consequences.
• Reflecting: interpret results with participants and determine the next cycle or theoretical insight.

Action research may include several cycles across different units or time periods. The researcher should distinguish between action outcomes, research data, and personal impressions.

6.4 Evaluation designs

Ex ante evaluation occurs before implementation. Expert review, scenario analysis, simulation, and requirements validation can identify weaknesses early.

Artificial evaluation tests an artifact in a controlled environment. Laboratory experiments, benchmark datasets, usability tests, and performance tests allow stronger control over variables.

Naturalistic evaluation examines use in a realistic organizational or social setting. Field studies, case studies, longitudinal observation, and action research provide contextual evidence but offer less experimental control.

Formative evaluation supports improvement during development. Summative evaluation assesses the completed artifact or intervention against predefined criteria. A mature study often uses multiple forms of evaluation.

6.5 Utility assessment

Utility is multidimensional. It may include effectiveness, efficiency, usability, learnability, reliability, security, maintainability, adaptability, acceptability, affordability, equity, and sustainability. Measures should combine objective indicators with stakeholder perceptions where appropriate.

For example, a clinical decision-support tool may be assessed through diagnostic accuracy, alert response time, clinician workload, false-alert rates, patient safety, and trust. No single metric adequately represents utility.

7. Important Theories and Models

Design science research framework

A widely used framework connects the environment, the knowledge base, and the research process. The environment supplies problems, stakeholders, organizational capabilities, and technology. The knowledge base supplies theories and methods. Research activities create and evaluate artifacts, producing contributions that can inform both practice and scholarship.

Design principles

A design principle is a generalized prescription linking a problem context, a design action, and an expected effect. For example: “To improve transparency in automated eligibility decisions, expose the principal decision factors in language appropriate to the affected user.” Principles should be supported by design experience and evaluation evidence.

Reference models

A reference model provides an abstract structure for a class of systems or processes. It identifies common components and relationships while allowing implementation-specific variation. Reference models support comparison, standardization, reuse, and communication across organizations.

Design theories

A design theory explains the purpose, scope, constructs, principles, artifact structure, implementation procedures, and expected outcomes of a class of designs. It is more general than a single prototype and more prescriptive than a descriptive theory. Developing one requires abstraction across design episodes and evidence about mechanisms and boundary conditions.

Socio-technical systems theory

Socio-technical theory emphasizes that technical performance depends on interactions among technology, people, tasks, structures, incentives, and institutions. It discourages solutions that optimize software while ignoring work practices and organizational power.

Technology acceptance and use models

Models such as perceived usefulness, perceived ease of use, social influence, and facilitating conditions help explain adoption. They should not be treated as universal explanations; organizational context, risk, professional norms, and mandatory use can alter relationships.

Realist evaluation

Realist evaluation asks, “What works, for whom, in what circumstances, and through what mechanisms?” It is useful for complex interventions whose outcomes vary across settings. Its central logic is context–mechanism–outcome configuration.

8. Practical Applications

Design science is appropriate for developing cybersecurity controls, decision-support systems, data governance frameworks, privacy-enhancing technologies, digital health platforms, educational analytics, sustainability dashboards, and enterprise architectures. Action research is appropriate when an organization must change its practices while researchers study the change, such as implementing agile governance, improving incident response, or introducing data stewardship.

A university may use participatory design to create an accessibility service. Students with disabilities, instructors, administrators, and technology staff jointly identify barriers, prototype solutions, and define success. A logistics company may develop a routing algorithm through successive cycles of simulation, pilot deployment, driver feedback, and operational measurement. A public agency may use action research to introduce a responsible artificial intelligence policy, learning from each implementation cycle.

9. Examples

A doctoral researcher might develop a machine-learning method for detecting fraudulent transactions. The artifact is not only the trained model; it includes feature engineering, threshold rules, explanation mechanisms, deployment procedures, and governance controls. Evaluation should compare detection performance with existing methods and examine false positives, fairness, computational cost, and analyst usability.

Another researcher might create a maturity model for cloud governance. The model can be evaluated through expert review, application in multiple organizations, inter-rater reliability, and assessment of whether maturity scores support better investment decisions. The contribution may be the constructs, assessment method, and evidence-based progression logic.

An action researcher working with a hospital may introduce a digital handover system. Several cycles can examine workflow redesign, training, implementation, observed communication quality, and staff acceptance. The study should report both improvements and resistance, including conditions that limited success.

10. Technology and Digital Applications

Digital technologies expand the possibilities for design-oriented research. Cloud platforms enable rapid prototyping and scalable experimentation. Application programming interfaces allow artifacts to integrate with enterprise systems. Process-mining tools reveal actual workflows and identify deviations from prescribed procedures. Digital twins and simulation environments permit evaluation before expensive or risky deployment.

Artificial intelligence can support adaptive artifacts, recommendation systems, anomaly detection, and automated decision-making. However, AI artifacts require evaluation of robustness, explainability, bias, privacy, security, human oversight, and performance drift. Blockchain may support distributed records and provenance, but its use must be justified against simpler alternatives. Internet-of-Things systems enable real-time interventions but introduce challenges involving sensor reliability, interoperability, and surveillance.

Low-code platforms, collaborative repositories, automated testing, and telemetry facilitate iterative build–evaluate cycles. Digital experimentation should nevertheless remain theoretically and ethically governed; large volumes of usage data do not automatically constitute valid evidence.

11. Strategic / Managerial Importance

Design-oriented research helps managers convert uncertain technology opportunities into evidence-based decisions. It can reduce implementation risk by exposing weaknesses through prototypes and staged evaluation. It also supports capability development because employees participate in defining problems, interpreting evidence, and institutionalizing new practices.

Managers should distinguish between a successful pilot and a sustainable organizational capability. Scaling requires attention to architecture, data quality, cybersecurity, procurement, skills, support, regulation, and ownership. A design may be technically effective but strategically inappropriate if it creates vendor dependence, conflicts with organizational values, or cannot be maintained.

Action research is valuable for transformation because it combines change management with learning. It makes assumptions visible and gives stakeholders a structured way to negotiate competing priorities. Senior leaders should protect time for participation and reflection rather than demanding immediate results from every cycle.

12. Detailed Case Study

Illustrative fictional case: A predictive maintenance platform for municipal water services

#### Background

A municipal water authority operates pumps and treatment equipment across several remote facilities. Maintenance is primarily calendar-based or reactive. The authority has sensor data but uses it inconsistently. The research team collaborates with engineers, operations managers, and field technicians.

#### Problem

Unexpected pump failures interrupt water supply and create emergency repair costs. Previous analytics projects failed because predictions were difficult to interpret, sensor data were unreliable, and technicians did not trust recommendations. The research problem is how to design a predictive maintenance artifact that improves maintenance decisions while fitting field workflows.

#### Analysis

Initial interviews and observations show that technicians value practical explanations more than abstract probability scores. Data analysis reveals missing readings, inconsistent asset identifiers, and seasonal patterns. The team defines requirements: early warning, understandable causes, integration with work orders, low false-alert rates, and support for technician override. The research gap concerns the combination of predictive analytics with explainable, workflow-integrated maintenance decisions in low-resource municipal settings.

#### Strategy / Solution

The team develops a platform containing a data-quality module, a prediction model, an explanation interface, and a work-order integration service. A reference model describes data, analytics, human decision, intervention, and feedback components. The first cycle uses historical data and simulation. The second deploys the platform at two facilities. Technicians review alerts weekly, record reasons for accepting or rejecting them, and suggest interface changes. A third cycle introduces revised thresholds and an asset-data governance procedure.

Evaluation combines precision and recall, warning time, emergency failures, maintenance cost, technician trust, and workload. A comparison is made with similar facilities using conventional maintenance. Qualitative interviews explain why some alerts are acted upon and others are ignored.

#### Outcome

The illustrative evaluation finds fewer emergency failures and improved prioritization, but only after data-quality controls and workflow integration are introduced. Technicians reject several technically accurate alerts because they are not actionable. The final artifact therefore includes recommended inspections, confidence ranges, explanation of influential sensor patterns, and an override log. The research contribution is a set of design principles for human-centered predictive maintenance, not merely the software prototype.

#### Learning Points

The case demonstrates that technical accuracy is only one dimension of utility. Participatory involvement reveals requirements that data scientists might overlook. Iterative evaluation identifies the mechanisms producing value: trustworthy data, understandable explanations, workflow compatibility, and institutionalized feedback. The case also shows why boundary conditions matter; results from a municipal authority may not transfer directly to a highly automated industrial plant.

13. Advantages / Benefits

Design science produces tangible and reusable outputs while addressing practical problems. It encourages innovation, integrates theory with engineering, and makes design decisions explicit. Iterative evaluation can reduce failure risk and improve fit with users. Action research creates organizational learning, supports change, and provides access to rich contextual data. Participatory approaches improve relevance, legitimacy, usability, and equity. Reference models and design principles enable reuse beyond the original case.

14. Limitations / Challenges

Design research can become overly solution-oriented and insufficiently theoretical. Researchers may mistake a working prototype for a research contribution. Evaluation is often difficult because real organizations change during the study, comparison groups may be unavailable, and long-term impacts take time to appear. Action research raises concerns about researcher influence, role conflict, confidentiality, and limited generalizability. Participation can be expensive and may privilege articulate or powerful stakeholders. Rapid technological change can make artifacts obsolete before evaluation is complete.

Other challenges include technical debt, poor data quality, integration constraints, resistance to change, competing organizational objectives, and pressure to report only positive outcomes. Doctoral researchers should document negative findings and distinguish limitations from failures of the research process.

15. Ethical and Governance Considerations

Researchers must obtain informed consent where required, protect personal and organizational data, and minimize risks arising from experimentation. Participants should understand whether they are research subjects, co-designers, employees affected by change, or all three. Power relationships are especially important in workplaces where refusal may be difficult.

AI and analytics artifacts require attention to discrimination, explainability, contestability, privacy, security, and human accountability. Researchers should define who can override a system, who is responsible for errors, and how decisions are audited. Data governance should address ownership, retention, access, provenance, quality, and secondary use.

Participatory research must represent marginalized groups and ensure that their contributions are not extracted without benefit or recognition. Governance arrangements should specify decision rights, intellectual property, publication permissions, incident management, and responsibilities after the research ends.

16. Current and Emerging Trends

Important trends include human-centered and responsible AI, privacy-preserving computation, digital twins, autonomous systems, continuous experimentation, and sustainable computing. There is increasing interest in evaluating not only immediate performance but also environmental cost, energy consumption, social impact, and resilience.

Open science practices are expanding through open artifacts, reproducible code, preregistered evaluation plans, shared datasets where lawful, and detailed design histories. Researchers are also combining design science with causal inference, process mining, computational social science, and living-lab approaches. Future studies will increasingly need to evaluate artifacts operating in dynamic ecosystems rather than isolated organizational settings.

17. Best Practices

1. Begin with a demonstrably significant problem rather than a preferred technology.

2. Define the intended contribution before building the artifact.

3. Engage relevant stakeholders early and maintain participation throughout the cycles.

4. Ground design decisions in literature, theory, standards, and domain evidence.

5. Maintain traceability from problem to requirement, feature, measure, and finding.

6. Use multiple evaluation methods suited to different claims.

7. Assess unintended effects, equity, security, sustainability, and maintainability.

8. Record rejected alternatives, negative results, and changes in assumptions.

9. Separate evidence from researcher interpretation and disclose positionality.

10. State context, boundary conditions, and transfer requirements clearly.

11. Plan for deployment, governance, ownership, and post-study maintenance.

12. Communicate both the artifact and the knowledge generated through its design.

18. Future Scope

Future doctoral research can develop stronger theories of artifact adoption, adaptation, and evolution. There is scope for methods that combine participatory design with automated experimentation, formal verification, and real-world monitoring. More research is needed on evaluating AI systems over time as data distributions, user behavior, and regulations change.

Other promising areas include climate-conscious digital design, inclusive technologies for underserved communities, trustworthy autonomous systems, cross-organizational reference architectures, and methods for measuring long-term public value. Researchers will also need approaches for governing artifacts that are co-produced by humans, algorithms, platforms, and communities.

19. Chapter Summary

Design science research creates and evaluates artifacts to address important problems and generate transferable knowledge. Action research combines intervention with collaborative inquiry, while participatory approaches give stakeholders meaningful influence over research and design. Rigorous studies connect problem relevance, theoretical grounding, iterative development, utility assessment, and explicit contribution.

Evaluation may be technical, human, organizational, economic, ethical, or societal. Build–evaluate cycles, reference models, design principles, and design theories help researchers move from a single solution toward reusable knowledge. Successful doctoral research must report not only what was built, but why it was designed, how it was evaluated, what mechanisms produced outcomes, where it applies, and what limitations remain.

20. Key Terms

1. Action research: Collaborative inquiry involving diagnosis, intervention, observation, and reflection.

2. Artifact: A designed construct, model, method, system, architecture, or process.

3. Build–evaluate cycle: Iterative sequence of creating, testing, interpreting, and refining an artifact.

4. Construct: Conceptual vocabulary used to describe a phenomenon or problem.

5. Design principle: Generalized prescription connecting a design action with an expected effect.

6. Design science research: Research focused on creating and evaluating artifacts for important problems.

7. Design theory: Explanatory and prescriptive knowledge about how a class of artifacts works.

8. Demonstration: Showing that an artifact can address an instance of a problem.

9. Evaluation: Systematic assessment against explicit criteria.

10. Formative evaluation: Evaluation conducted to improve a developing artifact.

11. Instantiated artifact: A concrete implementation of a model, method, or design.

12. Naturalistic evaluation: Assessment in a realistic operational context.

13. Participatory design: Design in which affected stakeholders actively shape the solution.

14. Reference model: Abstract representation of common structures in a class of systems.

15. Research contribution: New knowledge, artifact, method, principle, evidence, or theory produced by research.

16. Socio-technical system: A system formed through interaction between technology, people, tasks, and institutions.

17. Summative evaluation: Assessment of a completed artifact or intervention.

18. Utility: The extent to which an artifact provides valuable outcomes for intended users and stakeholders.

21. Review Questions

Short Answer Questions

1. What distinguishes design science research from conventional software development?

2. What are the principal stages of an action research cycle?

3. Why is evaluation central to design-oriented research?

4. What is the purpose of a reference model?

5. Name four dimensions that may be included in a utility assessment.

Long Answer Questions

1. Explain how design science research can generate both practical and theoretical contributions.

2. Compare artificial, naturalistic, formative, and summative evaluation.

3. Discuss the role of stakeholder participation in designing responsible IT artifacts.

4. Explain how socio-technical systems theory improves the analysis of IT interventions.

5. Describe how a doctoral researcher can develop design principles from several build–evaluate cycles.

Analytical / Application Questions

1. An organization has a highly accurate fraud-detection model, but analysts rarely use it. Design an evaluation plan that identifies the reasons for low utility and proposes an iterative improvement strategy.

2. A public agency intends to deploy facial-recognition technology in transport stations. Develop a participatory and ethical research design that addresses effectiveness, privacy, bias, accountability, and public legitimacy.

3. A university wants to introduce an AI advising system but has limited historical data. Recommend an appropriate design science and action research strategy, including artifact development, stakeholder roles, evaluation measures, and boundary conditions.

22. Multiple Choice Questions with Answers

Question 1

Which statement best describes design science research?

A. It only observes technology use without changing the environment.

B. It creates and evaluates artifacts to address significant problems.

C. It focuses exclusively on statistical theory testing.

D. It replaces evaluation with expert opinion.

Correct Answer: B. It creates and evaluates artifacts to address significant problems.

Question 2

What is the primary purpose of a build–evaluate cycle?

A. To finalize requirements before any stakeholder involvement

B. To produce software without theoretical justification

C. To iteratively improve an artifact using evidence

D. To avoid testing in realistic environments

Correct Answer: C. To iteratively improve an artifact using evidence

Question 3

Which activity is characteristic of action research?

A. Collaborative diagnosis followed by intervention and reflection

B. Permanent separation of researchers from practitioners

C. Evaluation only after organizational change is complete

D. Use of laboratory data exclusively

Correct Answer: A. Collaborative diagnosis followed by intervention and reflection

Question 4

A reference model is best understood as:

A. A detailed implementation that cannot be adapted

B. An abstract structure for a class of systems or processes

C. A survey instrument for measuring user satisfaction

D. A legal contract governing software procurement

Correct Answer: B. An abstract structure for a class of systems or processes

Question 5

Which evaluation question is most appropriate for a utility assessment?

A. Does the artifact provide valuable outcomes for intended users?

B. Was the artifact developed using the newest programming language?

C. Does the artifact eliminate the need for organizational governance?

D. Can the artifact be described without reference to context?

Correct Answer: A. Does the artifact provide valuable outcomes for intended users?

Question 6

Why should AI artifacts be evaluated beyond predictive accuracy?

A. Accuracy is unrelated to technical performance.

B. AI systems never require human oversight.

C. Issues such as fairness, explainability, privacy, and drift affect responsible use.

D. Stakeholders cannot understand numerical performance measures.

Correct Answer: C. Issues such as fairness, explainability, privacy, and drift affect responsible use.

Question 7

Which approach gives stakeholders meaningful influence over problem definition and design?

A. Participatory design

B. Black-box benchmarking

C. Uncontrolled experimentation

D. Purely deductive modeling

Correct Answer: A. Participatory design

Question 8

What is a design principle?

A. A generalized prescription linking a design action to an expected effect

B. A list of programming errors in a prototype

C. A financial forecast for technology investment

D. A record of participant identities

Correct Answer: A. A generalized prescription linking a design action to an expected effect

Question 9

A naturalistic evaluation primarily examines an artifact:

A. In a realistic operational setting

B. Only through mathematical proof

C. Without users or organizational context

D. Before a problem has been identified

Correct Answer: A. In a realistic operational setting

Question 10

Which practice most strengthens the credibility of a doctoral design science study?

A. Reporting only successful tests

B. Hiding changes made during development

C. Linking problems, requirements, design features, measures, and findings

D. Treating a working prototype as sufficient evidence

Correct Answer: C. Linking problems, requirements, design features, measures, and findings