Chapter 11: Computational, Big Data, and AI Research Methods

1. Introduction

Information technology research increasingly examines phenomena that are too large, fast, complex, or dynamic for conventional data-collection methods alone. Digital platforms generate event logs, transactions, text, images, sensor readings, software repositories, location traces, and network interactions. Machine learning, natural language processing, network analysis, process mining, simulation, and artificial intelligence (AI) provide methods for analysing these data and for developing computational explanations of technological and organisational behaviour.

Computational research does not simply mean applying sophisticated software to a large dataset. It is a research strategy requiring an explicit connection between theory, data-generating processes, computational procedures, validation, interpretation, and ethical governance. Large datasets may contain systematic bias, duplicated records, missing populations, measurement error, privacy risks, and hidden dependencies. A highly accurate model may still be scientifically weak if it predicts a proxy rather than the construct of interest, leaks information from the future, or cannot be reproduced.

This chapter presents a rigorous framework for computational and AI-assisted research in information technology. It addresses methodological foundations, major analytical approaches, practical applications, governance requirements, and emerging directions. The central argument is that computational power must be subordinated to research validity: the purpose of a method is not merely to produce predictions or patterns, but to support credible, transparent, useful, and ethically defensible knowledge.

2. Learning Objectives

By the end of this chapter, doctoral researchers should be able to:

1. Explain the meaning, scope, and assumptions of computational and big data research.

2. Select machine learning, NLP, network, process-mining, simulation, and digital-trace methods appropriately.

3. Design data pipelines that reduce bias, leakage, privacy exposure, and reproducibility failure.

4. Distinguish prediction, explanation, description, causal inference, and simulation.

5. Evaluate computational models using suitable technical and substantive criteria.

6. Integrate AI-assisted tools into research while retaining human accountability.

7. Develop governance procedures for ethical, transparent, and reproducible research.

3. Meaning and Concept

Computational research uses formal algorithms, programmable procedures, and computational representations to collect, process, analyse, model, or simulate phenomena. Big data generally refers to data characterised by substantial volume, velocity, variety, and often complexity or uncertainty. These characteristics are not sufficient to make a study scientifically valuable. A small, carefully sampled dataset may provide stronger evidence than a massive, poorly defined digital trace.

AI-assisted research includes the use of machine learning systems, generative models, automated coding tools, intelligent search, and decision-support systems during research. AI may assist with transcription, classification, literature discovery, code generation, anomaly detection, or hypothesis generation. It should not be treated as an autonomous epistemic authority. Researchers remain responsible for defining constructs, checking outputs, documenting procedures, and identifying errors.

Several research purposes must be distinguished. Descriptive analysis characterises what occurred; predictive analysis estimates what is likely to occur; explanatory research investigates why a phenomenon occurs; causal research estimates the effect of an intervention; and simulation explores how a system may behave under specified assumptions. One method can support several purposes, but evidence appropriate for prediction does not automatically establish causation.

4. Fundamental Principles

4.1 Alignment between question and method

A research question should determine the data and method, rather than the availability of an attractive dataset. Questions about classification, temporal forecasting, interaction structure, organisational workflow, or policy effects require different designs.

4.2 Construct validity

Digital variables are often proxies. Clicks may represent attention, convenience, or automated activity; sentiment scores may not represent genuine attitudes; repository commits may not represent productivity. Researchers must define constructs, justify operationalisations, and test whether measures behave as theory predicts.

4.3 Data-generating-process awareness

A dataset is produced by software settings, institutional policies, user choices, business incentives, and historical conditions. Missing records may reflect exclusion rather than random absence. Researchers should document who or what generated each observation and what is systematically absent.

4.4 Separation of training and evaluation

A model must be evaluated on data that were not used to fit or tune it. Temporal, individual-level, organisational, or site-level separation may be necessary where random splitting would permit dependence or leakage.

4.5 Transparency and reproducibility

Computational claims should be supported by versioned code, data dictionaries, parameter settings, environment specifications, model cards, and, where lawful, accessible data or synthetic substitutes. Reproducibility includes both rerunning the analysis and obtaining substantively consistent results under reasonable variations.

4.6 Fairness and inclusiveness

Performance should be examined across relevant groups and contexts. Aggregate accuracy can conceal serious disparities. Fairness is not a single universal metric; it involves normative choices about errors, benefits, burdens, and legitimate uses.

4.7 Proportionality and accountability

Data collection and automation should be proportionate to the research purpose. High-impact decisions require stronger validation, human oversight, contestability, and monitoring than low-risk exploratory analysis.

5. Major Components / Elements

A computational study normally contains the following elements:

1. Research question and theory: the phenomenon, mechanism, or decision of interest.

2. Data source: administrative data, digital traces, sensors, experiments, public repositories, surveys, or generated data.

3. Data architecture: storage, schemas, identifiers, metadata, access controls, and provenance.

4. Pre-processing: cleaning, deduplication, entity resolution, missing-data treatment, tokenisation, scaling, and feature construction.

5. Analytical method: statistical model, machine-learning algorithm, NLP pipeline, graph method, process-mining technique, or simulation.

6. Validation design: holdout data, cross-validation, temporal testing, external validation, robustness checks, and sensitivity analysis.

7. Interpretation: substantive meaning, uncertainty, limitations, and relationship to theory.

8. Governance: consent, privacy, security, fairness, intellectual property, accountability, and retention.

9. Communication: clear reporting for technical and non-technical audiences.

6. Process / Framework / Methodology

A defensible workflow begins with problem formulation. The researcher specifies the unit of analysis, target population, outcome, time horizon, intervention if relevant, and acceptable error. A data-management plan then records sources, permissions, retention rules, transformations, and risks.

Data preparation should be performed through a documented pipeline rather than undocumented manual edits. Quality checks should identify missingness, duplicates, impossible values, outliers, class imbalance, temporal inconsistencies, and changes in collection procedures. Sampling decisions should be justified. If the dataset is a convenience sample of platform users, conclusions should not be presented as population estimates without qualification.

Feature engineering translates raw records into variables. It should be performed using information available at the intended prediction or decision point. For example, predicting customer churn using a cancellation complaint filed after the prediction date is leakage, even if the model performs well. Labels should be defined independently where possible, and annotation guidelines should be tested for consistency.

Model selection should consider interpretability, computational cost, expected error, deployment context, and research purpose. Baseline models are essential because complex models do not automatically provide better evidence. Training, tuning, and final testing should be separated. For longitudinal data, time-based validation is usually more realistic than random splitting. For network or organisational data, entities or sites may need to be held out.

Evaluation should combine technical and substantive criteria. Classification measures may include precision, recall, F1 score, calibration, and area under a curve; regression may use mean absolute error or root mean squared error. Metrics must reflect the consequences of false positives and false negatives. Researchers should report confidence or uncertainty intervals where appropriate and conduct subgroup, sensitivity, and ablation analyses.

Interpretation requires caution. Feature importance is not necessarily causation, and an interpretable explanation generated after prediction may be an approximation rather than the model’s actual reasoning. Qualitative inspection of errors, counterexamples, and representative cases can reveal failures hidden by summary metrics.

Finally, the study should be audited and documented. A computational audit records data versions, code versions, random seeds, dependencies, model parameters, exclusions, and deviations from the preregistered or planned protocol. Post-deployment systems require monitoring for drift, changing populations, adversarial behaviour, and unexpected social effects.

7. Important Theories and Models

7.1 The data-information-knowledge distinction

Raw events become information when structured and contextualised, and knowledge when interpreted in relation to concepts, mechanisms, and evidence. This model reminds researchers that computation does not remove the need for theoretical reasoning.

7.2 The CRISP-DM model

CRISP-DM describes business understanding, data understanding, data preparation, modelling, evaluation, and deployment. Its iterative structure is useful for research, although doctoral studies must add stronger theory, ethics, and reproducibility requirements.

7.3 The knowledge discovery process

Knowledge discovery treats pattern extraction as a sequence involving selection, cleaning, transformation, data mining, and interpretation. The final stage is critical: a statistically visible pattern may be spurious, trivial, or socially harmful.

7.4 The socio-technical systems perspective

Algorithms operate within institutions, interfaces, policies, users, and power relationships. Outcomes cannot always be attributed to the model alone. A hiring classifier, for example, may reproduce inequality through historical data, job definitions, recruiter behaviour, and feedback from earlier decisions.

7.5 The network perspective

Network analysis represents entities as nodes and relationships as edges. Degree, centrality, density, communities, paths, and structural position can reveal diffusion and dependence. Network observations violate ordinary independence assumptions, so inference must account for relational structure.

7.6 Process and event-log models

Process mining uses event logs to discover, compare, and monitor workflows. Process models represent activities, ordering, concurrency, rework, and bottlenecks. Conformance checking compares observed behaviour with a reference process, while performance analysis examines time and resource use.

7.7 Agent-based and system-dynamics models

Agent-based models represent heterogeneous actors and local rules; system-dynamics models represent stocks, flows, feedback loops, and delays. Both are useful for exploring scenarios, but their conclusions depend on assumptions and calibration. Simulation is not evidence that the real system will behave identically.

8. Practical Applications

Machine learning supports intrusion detection, demand forecasting, software-defect prediction, recommender systems, fraud detection, health-risk estimation, and resource allocation. Supervised learning uses labelled outcomes; unsupervised learning identifies structure without labels; semi-supervised and self-supervised methods exploit partially labelled data; reinforcement learning optimises actions through feedback.

NLP enables document classification, topic discovery, information extraction, summarisation, question answering, and sentiment or stance analysis. Researchers should examine domain language, sarcasm, multilingual variation, annotation bias, and the difference between linguistic form and social meaning.

Network analysis is useful for studying collaboration, cybersecurity events, communication, supply chains, citation structures, and platform diffusion. Process mining can analyse service delivery, software development, incident response, procurement, and healthcare workflows. Digital trace analysis can study user journeys, online communities, system use, and technology adoption, subject to privacy and representativeness constraints.

Simulation supports capacity planning, cybersecurity exercises, platform governance, autonomous-system testing, and policy comparison. It is especially valuable when experimentation in the real world is costly or dangerous.

9. Examples

A university may predict students at risk of withdrawing from a course using learning-management-system activity. A responsible design defines the intervention point, excludes post-withdrawal records, evaluates performance across demographic groups, and ensures that predictions trigger supportive services rather than punitive labelling.

A software engineering study may use repository data to examine review delays. The researcher should distinguish author identity, project size, language, release cycle, and organisational policy. A simple correlation between review delay and defect rate does not establish that delay causes defects.

A cybersecurity study may use event logs and graph methods to identify unusual lateral movement. Because attackers adapt and normal behaviour changes, the model requires temporal validation, adversarial testing, and an escalation process that avoids treating every anomaly as an incident.

10. Technology and Digital Applications

Relevant technologies include cloud data warehouses and object stores, distributed processing frameworks, notebook environments, workflow orchestrators, containerisation, version-control systems, experiment-tracking tools, graph databases, process-mining platforms, and secure research environments. NLP systems may use tokenisation, embeddings, transformer architectures, retrieval-augmented generation, and human annotation interfaces.

Technology choices should follow research requirements. Cloud deployment may improve scalability but introduces jurisdiction, access-control, cost, and vendor-dependence issues. Generative AI may accelerate coding or drafting, but generated code can contain security defects, fabricated citations, licensing problems, or undocumented assumptions. Every AI-assisted contribution should be reviewed and recorded.

11. Strategic / Managerial Importance

Computational research helps organisations detect operational inefficiency, anticipate demand, allocate resources, manage risk, and evaluate digital transformation. It can connect strategic objectives with observable indicators and provide early warnings from real-time data. However, managerial value depends on decision integration. A dashboard that produces predictions without ownership, intervention protocols, or evaluation may create the appearance of intelligence without improving outcomes.

Leaders should establish data ownership, model-risk governance, escalation rules, audit rights, procurement standards, and responsibilities for monitoring. They should also support interdisciplinary teams combining domain expertise, methodological competence, security, legal knowledge, and affected-user perspectives.

12. Detailed Case Study

Background

The following is a fictional illustrative case. MetroCare, a regional healthcare network, introduced an AI system to predict which patients might miss outpatient appointments. The organisation hoped to reduce waiting-list delays and improve resource utilisation.

Problem

The initial model used appointment history, communication records, travel distance, demographic variables, and prior service use. It achieved strong overall accuracy in a retrospective evaluation. However, clinic managers observed that predictions were less reliable for migrants, older patients, and people using telephone rather than online services. A review also discovered that some variables were recorded after staff had already begun reminder interventions.

Analysis

The research team reconstructed the data-generating process and found three problems. First, digital communication variables were proxies for access to smartphones and stable internet. Second, the random train-test split placed records from the same patients in both sets, producing overly optimistic performance. Third, post-intervention records created temporal leakage. The team also found that the cost of a missed appointment differed across clinics, so a single threshold was not operationally appropriate.

Strategy / Solution

The team redesigned the study using a time-based split and patient-level separation. It removed post-decision variables, documented missingness, and included a telephone-based data pathway. Performance was reported by clinic, age group, language-support category, and communication channel. Rather than automatically cancelling or deprioritising appointments, the model generated a review list for trained coordinators. Coordinators could record reasons for overriding predictions, and a quarterly audit examined calibration, false negatives, complaints, and access outcomes.

Outcome

The revised model had lower headline accuracy but produced more reliable operational performance and fewer disparities. Reminder resources were directed toward patients who voluntarily selected additional support. The organisation also discovered that transport barriers, rather than appointment forgetfulness, explained many cases. It therefore expanded transport assistance instead of relying exclusively on algorithmic reminders.

Learning Points

The case illustrates that leakage can inflate apparent performance, digital traces can encode structural inequality, and fairness requires analysis of context and consequences. It also demonstrates that a less complex decision-support system, combined with human review and service redesign, may be more valuable than a highly automated system.

13. Advantages / Benefits

Computational methods can analyse scale and speed beyond manual approaches, identify subtle patterns, integrate heterogeneous data, support continuous monitoring, test complex scenarios, and improve reproducibility through executable workflows. They can also make previously inaccessible organisational processes visible and enable more targeted interventions. These benefits are greatest when computational evidence is combined with domain knowledge and complementary qualitative or experimental methods.

14. Limitations / Challenges

Important limitations include biased sampling, missing data, unstable labels, concept drift, spurious correlations, class imbalance, weak construct validity, dependence among observations, privacy risks, adversarial manipulation, high infrastructure costs, and limited interpretability. Large language models add risks of hallucination, prompt sensitivity, training-data contamination, and inconsistent outputs. Computational studies may also encourage data-driven hypotheses that lack theoretical importance.

External validity is particularly difficult when a model is trained in one institution, country, platform, or historical period. Reproducibility may be limited by proprietary data, changing APIs, unavailable hardware, or software dependencies. Researchers should report these constraints rather than implying universal applicability.

15. Ethical and Governance Considerations

Researchers must establish a lawful and ethically defensible basis for collecting and linking data. Consent may be difficult for public digital traces, but public availability does not eliminate privacy expectations. Data minimisation, pseudonymisation, encryption, access control, retention limits, and disclosure-risk assessment are essential.

Governance should address purpose limitation, secondary use, sensitive attributes, re-identification, intellectual property, cross-border transfer, and the rights of affected individuals. High-impact applications require impact assessments, human oversight, explanations appropriate to the audience, avenues for correction and appeal, and clear accountability when systems fail.

Ethical review should consider not only direct participants but also bystanders, groups represented in historical data, and communities affected by deployment. Researchers should avoid claiming neutrality where design choices embed values. AI-generated text, code, images, or analyses should be disclosed according to institutional policy, and confidential research materials should not be entered into uncontrolled external systems.

16. Current and Emerging Trends

Important developments include foundation models, multimodal learning, retrieval-augmented generation, federated learning, privacy-preserving computation, causal machine learning, synthetic data, graph neural networks, digital twins, automated machine learning, and responsible MLOps. These technologies may reduce data movement, improve representation learning, or support complex decision environments.

At the same time, regulation and professional standards increasingly emphasise risk classification, documentation, transparency, security, and accountability. Emerging research is likely to focus less on benchmark performance alone and more on robustness, calibration, fairness, energy use, human-AI collaboration, and real-world impact.

17. Best Practices

1. Start with a theoretically meaningful question and define the intended use.

2. Map the data-generating process and document inclusion and exclusion.

3. Establish a data dictionary, provenance record, and secure access plan.

4. Use leakage-resistant splitting based on time, person, site, or project where necessary.

5. Compare complex methods with transparent baselines.

6. Evaluate calibration, subgroup performance, robustness, and operational consequences.

7. Separate exploratory analysis from confirmatory claims.

8. Preserve code, environments, parameters, seeds, and data-processing decisions.

9. Use human review for high-impact decisions and monitor post-deployment drift.

10. Report negative results, uncertainty, limitations, and deviations from the plan.

11. Involve domain experts and affected stakeholders.

12. Treat AI tools as assistants requiring verification, not as authors or authorities.

18. Future Scope

Future computational research will increasingly combine causal inference with machine learning, sensor and edge data with privacy protection, and large language models with verified retrieval and structured reasoning. Digital twins may enable operational experimentation, while federated and confidential-computing approaches may allow collaboration without centralising sensitive data.

The field will also need stronger methods for evaluating human-AI teams, measuring long-term social effects, detecting model-induced behavioural change, and preserving scientific records when commercial platforms alter their systems. Doctoral researchers can contribute by developing transparent benchmarks, interdisciplinary theories, governance frameworks, and methods that connect computational performance with human and organisational outcomes.

19. Chapter Summary

Computational, big data, and AI research methods expand the ability of IT scholars to study complex digital systems. Their value depends on alignment between question and method, valid constructs, awareness of data-generating processes, leakage-resistant evaluation, fairness, transparency, and ethical governance. Machine learning supports prediction, NLP analyses language, network analysis examines relationships, process mining reconstructs workflows, and simulation explores possible system behaviour. None of these methods replaces theory or critical interpretation. Reliable computational research is reproducible, proportionate, context-sensitive, and accountable to the people and organisations affected by its conclusions.

20. Key Terms

1. Algorithmic bias: Systematic and consequential distortion in an algorithm’s outputs.

2. Big data: Data characterised by scale, speed, diversity, and complexity.

3. Calibration: Agreement between predicted probabilities and observed frequencies.

4. Concept drift: Change over time in the relationship between inputs and outcomes.

5. Construct validity: The extent to which a measure represents the intended concept.

6. Digital trace data: Records generated through digital activities and systems.

7. Entity resolution: Linking records that refer to the same entity.

8. Feature engineering: Creating analytical variables from raw data.

9. Data leakage: Unintended use of information unavailable at the intended prediction time.

10. Machine learning: Algorithms that learn patterns from data for specified tasks.

11. Natural language processing: Computational analysis and generation of human language.

12. Network analysis: Study of entities and their relationships as graphs.

13. Process mining: Discovery and analysis of processes from event logs.

14. Reproducibility: Ability to recreate an analysis using documented materials and procedures.

15. Simulation: Computational representation used to explore system behaviour under assumptions.

16. Synthetic data: Artificially generated data designed to resemble properties of real data.

17. Temporal validation: Evaluation using later observations than those used for training.

18. Explainability: The degree to which model behaviour can be understood or communicated.

21. Review Questions

Short Answer Questions

1. Why does a large dataset not automatically produce valid research findings?

2. Distinguish prediction from causal explanation.

3. What is data leakage, and why is it dangerous?

4. State two applications of process mining in IT.

5. Why should AI-assisted research outputs be independently verified?

Long Answer Questions

1. Explain a complete workflow for designing and validating a computational IT research study.

2. Compare supervised learning, network analysis, process mining, and simulation in terms of questions, data, and limitations.

3. Discuss how construct validity and the data-generating process affect digital-trace research.

4. Develop an ethical and governance framework for an AI system used in a high-impact organisational decision.

5. Explain why reproducibility requires more than publishing model accuracy results.

Analytical / Application Questions

1. A university develops a model to identify students likely to fail. Design a leakage-resistant evaluation and identify at least four fairness or governance risks.

2. An organisation’s process-mining analysis reveals that one department has unusually long approval times. Explain how you would distinguish process inefficiency from differences in case complexity.

3. A research team uses a large language model to code interview transcripts. Propose a validation, privacy, and reproducibility protocol for the study.

22. Multiple Choice Questions with Answers

Question 1

Which practice most directly prevents temporal data leakage in a predictive study?

A. Increasing the number of model layers

B. Using only variables available at the prediction time

C. Removing all categorical variables

D. Reporting only aggregate accuracy

Correct Answer: B. Using only variables available at the prediction time

Question 2

What is the primary purpose of a baseline model?

A. To guarantee fairness

B. To replace theoretical reasoning

C. To provide a transparent reference for judging added complexity

D. To eliminate the need for testing

Correct Answer: C. To provide a transparent reference for judging added complexity

Question 3

Which method is specifically designed to analyse event logs and organisational workflows?

A. Process mining

B. Sentiment analysis

C. Image segmentation

D. Cluster sampling

Correct Answer: A. Process mining

Question 4

Why can aggregate accuracy be misleading in an AI system?

A. Accuracy is never mathematically defined

B. It may conceal substantially different error rates across groups

C. It always measures causation

D. It can only be used for regression

Correct Answer: B. It may conceal substantially different error rates across groups

Question 5

In network analysis, what does a node generally represent?

A. A relationship or connection

B. An entity such as a person, system, or organisation

C. A model parameter

D. A missing value

Correct Answer: B. An entity such as a person, system, or organisation

Question 6

Which statement best describes simulation research?

A. It proves that real systems will behave exactly as simulated

B. It explores system behaviour under specified assumptions

C. It requires no validation or calibration

D. It is limited to text data

Correct Answer: B. It explores system behaviour under specified assumptions

Question 7

Which action best supports computational reproducibility?

A. Publishing only the final accuracy score

B. Changing preprocessing steps without recording them

C. Versioning code, data-processing decisions, parameters, and environments

D. Using proprietary software without documentation

Correct Answer: C. Versioning code, data-processing decisions, parameters, and environments

Question 8

A model uses a complaint recorded after a customer has already cancelled to predict cancellation. What problem is present?

A. Underfitting

B. Temporal leakage

C. Network sparsity

D. Randomisation

Correct Answer: B. Temporal leakage

Question 9

What is a central ethical concern when analysing public digital traces?

A. Public availability always removes privacy obligations

B. Individuals and bystanders may still face re-identification or unexpected use

C. Digital traces cannot contain sensitive information

D. Ethical review is unnecessary for computational studies

Correct Answer: B. Individuals and bystanders may still face re-identification or unexpected use

Question 10

Why should a complex machine-learning model be compared with a transparent model?

A. To determine whether complexity provides meaningful improvement

B. To ensure the complex model has more parameters

C. To avoid collecting validation data

D. To make all predictions causal

Correct Answer: A. To determine whether complexity provides meaningful improvement