Computational Research, Big Data and AI

Introduction

Business research increasingly examines digital traces: transactions, web clicks, customer reviews, sensor readings, search queries, electronic records, and social-media interactions. These sources can provide evidence at a scale, speed, and temporal resolution that traditional surveys and interviews rarely achieve. Machine learning can identify patterns, classify documents, forecast demand, and support decisions. Text analytics can turn unstructured language into analyzable evidence. Generative artificial intelligence can assist with coding, synthesis, and communication.

However, computational scale does not automatically produce valid knowledge. Large datasets may be biased, dependent, incomplete, poorly measured, or generated by systems that change over time. A highly accurate prediction may reveal little about why an outcome occurs. An impressive model may fail when deployed in a different population. Responsible computational research therefore requires methodological judgement as well as technical skill.

This chapter presents a research-oriented framework for using large-scale digital data, machine learning, text analytics, social-media data, and generative AI. It emphasizes the distinction between prediction and explanation, sound validation, data-leakage prevention, fairness, interpretability, and transparent use of AI.

Learning objectives

By the end of this chapter, you should be able to:

1. Explain the nature and research value of big data and computational methods.
2. Distinguish prediction, description, explanation, and causal inference.
3. Select suitable machine-learning and text-analytics approaches for research questions.
4. Design defensible training, validation, and test procedures.
5. Identify data leakage, sampling problems, drift, and algorithmic bias.
6. Evaluate model performance using appropriate metrics and substantive judgement.
7. Use explainability methods without confusing them with causal explanation.
8. Apply ethical, legal, and governance principles to digital research.
9. Use generative AI transparently, critically, and with human verification.

1. From conventional data to computational research

Traditional research often begins with a clearly defined population, a designed sample, and variables collected for a stated purpose. Computational research frequently begins with data produced incidentally by digital systems. A retailer may possess millions of transaction records; a platform may record searches, clicks, and recommendations; a bank may store time-stamped payment events.

The term big data commonly refers to datasets characterized by several dimensions:

These dimensions are not a definition based on a fixed numerical threshold. A dataset is “big” relative to the computational, organizational, and methodological resources needed to use it.

A crucial distinction is between data exhaust and research data. Data exhaust is produced as a by-product of activity, whereas research data are deliberately designed or selected to answer a question. Digital traces may be behaviorally rich but conceptually ambiguous. A click can indicate interest, accidental selection, automated activity, or interface confusion. Consequently, researchers must establish what an observation represents before modeling it.

Computational research usually involves a pipeline:

1. Define the research problem and target population.
2. Identify the data-generating process and obtain lawful access.
3. Collect, document, clean, and link data.
4. Construct variables or representations.
5. Train and validate models.
6. Evaluate statistical, substantive, ethical, and operational performance.
7. Report limitations and preserve reproducibility.

The pipeline is iterative. Decisions made during collection and preprocessing can affect conclusions as strongly as the choice of algorithm.

2. Four purposes of computational analysis

A research project should state whether it is primarily descriptive, predictive, explanatory, or causal. These purposes overlap, but they are not interchangeable.

Description

Description summarizes what has occurred. Examples include the distribution of customer complaints by category, the frequency of topics in annual reports, or the network structure of partnerships. Description may be exploratory or confirmatory, but it does not by itself explain causes or forecast future outcomes.

Prediction

Prediction estimates an unknown or future outcome from available information. For an outcome Y and features X, a prediction system estimates Ŷ = f(X). The function f may be a regression, decision tree, neural network, or another model. Predictive usefulness is assessed by performance on data not used to fit the model.

Prediction can be valuable even when the relationships are not causal. For example, the timing of a customer’s support calls may help predict cancellation without causing cancellation.

Explanation

Explanation seeks to clarify how or why variables are related, often through a theory or mechanism. A transparent statistical model may support explanation, but interpretability alone is insufficient. A model can be easy to read and still reflect confounding, reverse causality, or measurement error.

Causal inference

Causal inference asks what would happen under an intervention, such as changing a price or sending a message. Causal claims require assumptions about the research design, treatment assignment, confounding, interference, and measurement. A predictive feature is not automatically a cause. Causal inference may use experiments, natural experiments, matching, instrumental variables, regression discontinuity, or carefully justified longitudinal designs.

Prediction and explanation compared

A useful research report states explicitly which goal is being pursued. Confusing these goals is one of the most common errors in computational research.

3. Machine learning for research

Machine learning (ML) refers to computational procedures that learn patterns from data for prediction, classification, ranking, generation, or representation. It is not a single method.

Supervised learning

In supervised learning, examples include an outcome label. Regression predicts a numerical outcome such as sales; classification predicts a category such as fraud or sentiment. Common methods include linear and logistic regression, decision trees, random forests, gradient boosting, support-vector machines, and neural networks.

For binary classification, a confusion matrix distinguishes true positives, false positives, true negatives, and false negatives. Important metrics include:

Here, TP is the number of correctly identified positive cases, TN correctly identified negative cases, FP negative cases incorrectly labeled positive, and FN positive cases incorrectly labeled negative. Accuracy is often misleading when classes are imbalanced. Precision matters when false positives are costly; recall matters when missing positives is costly. The appropriate metric follows the decision context.

For a numerical outcome, mean absolute error is:

where yᵢ is the observed value, ŷᵢ is the prediction, and n is the number of evaluated cases. MAE has the same units as the outcome and treats errors linearly. Root mean squared error is:

RMSE gives relatively greater influence to large errors and is therefore sensitive to outliers.

Unsupervised learning

Unsupervised learning has no supplied outcome label. Clustering groups observations according to a distance or similarity measure; dimensionality-reduction methods summarize many variables; topic models identify recurring patterns in text. These methods are exploratory and require substantive interpretation. A cluster is not automatically a naturally existing segment, and a topic label is an interpretation supplied by the researcher.

Training, validation, and test data

A model fitted to all available data may appear successful simply because it has memorized idiosyncrasies. Data are therefore separated into:

• Training data: used to estimate model parameters.
• Validation data: used to select models, tune hyperparameters, or choose thresholds.
• Test data: held back until final evaluation.

For independent observations, a random split may be suitable. For time-dependent research, chronological validation is generally more credible: earlier observations train the model and later observations test it. For grouped data, such as multiple transactions from the same customer, splitting individual rows can cause information from one customer to appear in both training and test sets. Group-based splitting is safer.

Cross-validation repeatedly divides the training data into folds to estimate expected performance and tune models. It does not replace a genuinely untouched final test set. If many models or specifications are tried and only the best result is reported, the test set has effectively been used for selection and the estimate becomes optimistic.

Data leakage

Data leakage occurs when information unavailable at the intended prediction time, or information influenced by the outcome, enters model development. Examples include:

• using a post-cancellation customer-service code to predict cancellation;
• normalizing the entire dataset before splitting it;
• including future sales in a historical forecast;
• allowing duplicate customers into both training and test sets;
• selecting variables after inspecting test performance.

Leakage can produce excellent apparent accuracy and poor real-world performance. The remedy is to define the prediction moment, construct a realistic information boundary, split data before learned preprocessing, and audit every feature for temporal and institutional plausibility.

4. Text analytics and social-media research

Text analytics transforms language into structured representations. A basic document-term matrix records the number of occurrences of terms in documents. Term frequency–inverse document frequency (TF–IDF) gives greater weight to terms frequent in a document but uncommon across the collection:

TF(t,d) is the frequency of term t in document d, N is the number of documents, and DF(t) is the number of documents containing the term. Exact variants differ in normalization and smoothing. TF–IDF is useful for search and classification but does not understand meaning, irony, or context.

More advanced approaches use word embeddings, document embeddings, transformer models, and large language models. These represent semantic or contextual relationships, but they may encode social stereotypes, reproduce training-data biases, and be difficult to audit. A model’s output should therefore be treated as a measurement with error, not as an objective reading of meaning.

A sound text-analysis study includes:

1. A clear unit of analysis, such as a post, sentence, review, or document.
2. A sampling strategy and inclusion rule.
3. Cleaning decisions, including treatment of emojis, spelling, links, and multilingual text.
4. A coding scheme and, where possible, human-coded reference data.
5. Reliability checks between human coders or between human labels and model labels.
6. Error analysis by language, topic, author group, and document length.
7. Documentation of model version, prompts, preprocessing, and thresholds.

Social-media data create additional concerns. Users are not necessarily representative of a target population; highly active users may be overrepresented; bots and coordinated campaigns can distort patterns; deleted or private content may be inaccessible; and public availability does not eliminate privacy expectations. Researchers should minimize collection, avoid unnecessary personal identifiers, respect platform terms and institutional review requirements, and report the time and platform context of collection.

Sentiment analysis illustrates the difference between measurement and interpretation. A positive sentiment score may reflect sarcasm, a quoted statement, or a product feature rather than satisfaction. Validation against a domain-specific sample is more credible than assuming a general-purpose lexicon is accurate.

5. Model validation, fairness, and explainability

Validity beyond accuracy

A model should be assessed across several dimensions:

• Discrimination: Can it distinguish cases with different outcomes?
• Calibration: Do predicted probabilities correspond to observed frequencies?
• Robustness: Does performance persist under reasonable changes in data or assumptions?
• Fairness: Are errors and benefits distributed acceptably across relevant groups?
• Operational usefulness: Does it improve a decision relative to a credible baseline?
• Reproducibility: Can another researcher reconstruct the analysis?

A simple baseline, such as predicting the majority class or the previous period’s value, is essential. A complex model is not useful merely because it outperforms an inappropriate baseline.

Fairness and bias

Bias can enter through problem definition, sampling, labels, features, measurement, missingness, modeling, deployment, and feedback. Historical outcomes may encode unequal treatment. A model trained on those outcomes can reproduce or intensify inequality.

Fairness has multiple, sometimes incompatible, definitions. For example, demographic parity compares the proportion receiving a positive decision across groups. Equal opportunity compares true-positive rates. Calibration compares the meaning of predicted risk across groups. Because base rates and error trade-offs differ, satisfying one criterion may prevent satisfying another. Researchers should not select a metric mechanically; they should identify affected parties, legal requirements, harms, and the decision’s purpose.

Bias auditing may involve subgroup performance tables, intersectional analysis, missing-data analysis, qualitative review, and monitoring after deployment. Removing a sensitive attribute does not necessarily remove its influence because proxy variables may remain. Conversely, retaining a sensitive attribute can sometimes improve auditing or fairness correction. The decision requires governance and justification, not a simplistic rule.

Explainability

Interpretability means that humans can understand some aspect of a model or its output. A linear model may provide global coefficients; a decision tree may show rules; feature-importance methods rank influential variables; local explanation methods approximate why a particular prediction was produced. Counterfactual explanations describe changes that might alter an output.

Explanations have limitations. Feature importance is not causality. A local approximation may be unstable. A plausible narrative can be post hoc and misleading. Explanations should therefore be evaluated for fidelity, stability, comprehensibility, and usefulness to the intended audience. For high-impact decisions, explanation should be complemented by documentation, human review, contestability, and the ability to correct data.

6. Generative AI in research

Generative AI systems produce text, code, images, summaries, or other outputs from learned patterns. They can support brainstorming, transcription review, code explanation, synthetic data exploration, translation, and first-pass document classification. They can also hallucinate references, invent quotations, misstate results, expose confidential information, and obscure authorship or accountability.

Responsible use requires a documented protocol:

1. Define the permitted task and prohibit uses that exceed the system’s reliability.
2. Do not submit confidential, personal, proprietary, or restricted research data unless approved safeguards exist.
3. Record the tool, model or service, date, relevant settings, and prompts where reproducibility matters.
4. Verify every factual claim, citation, calculation, code segment, and generated classification.
5. Preserve the researcher’s intellectual responsibility for design, analysis, and conclusions.
6. Disclose material AI assistance in a methods or acknowledgements statement.
7. Check for bias, privacy risks, copyright concerns, and fabricated sources.

AI-generated text is not evidence. It is an instrument whose output must be evaluated against primary sources, validated data, and the research question. When AI is used to label data, a sample should be independently coded by humans and disagreements examined. When AI writes code, the researcher must run tests, inspect data transformations, and verify outputs.

7. An integrated workflow for defensible computational research

A practical workflow can be organized around seven questions:

What is the claim? Define whether the project describes, predicts, explains, or estimates a causal effect.

What is the population? Specify the target population, observation period, unit of analysis, and inclusion criteria.

How were the data generated? Document platform incentives, missingness, selection, measurement, and changes in the data-generating process.

What information was available at the decision time? Establish a temporal boundary to prevent leakage.

How will performance be evaluated? Select metrics, baselines, subgroup analyses, and validation design before examining final results.

What could go wrong? Conduct sensitivity analysis, error analysis, bias assessment, and privacy review.

How will the work be reproduced and governed? Preserve code, data dictionaries, version information, decision logs, and access controls; explain what cannot be shared.

A reproducible project does not necessarily publish sensitive data. It can provide synthetic examples, aggregation, executable code without restricted inputs, detailed schemas, and an audit trail.

Applied case: predicting subscription cancellation

Background

The illustrative case of Northstar Learning, a fictional online education company, concerns voluntary subscription cancellation. The company wants to identify subscribers who may cancel within the next 30 days so that it can offer support.

Problem

The analytics team has transaction history, login events, course progress, support contacts, plan type, and cancellation records. It proposes a gradient-boosting classifier and reports 94% accuracy. Management considers sending retention offers automatically to all customers classified as high risk.

Analysis

The research team first defines the prediction moment: the end of each week. Only information recorded by that time may be used. It discovers that a “closed account reason” field is created after cancellation and had been included in the proposed features. This is leakage and is removed.

The team also finds that rows, rather than subscribers, were randomly split. A subscriber with many weekly records could therefore appear in both training and test sets. The data are restructured into subscriber-week observations, with subscriber-level grouping and chronological validation. The final test period is held out until the model and threshold are fixed.

Because cancellation is uncommon, accuracy is replaced by precision, recall, calibration, and expected intervention cost. A simple baseline based on recent inactivity is compared with logistic regression and gradient boosting. Results are reported separately by plan type, region, language, and length of subscription. The team finds lower recall for a small-language group because support interactions are under-recorded, not because the group is less likely to cancel.

Approach and solution

The company uses the model only to prioritize voluntary, human-reviewed support. It does not automatically deny benefits or apply punitive treatment. Every intervention is logged, customers can decline contact, and outcomes are monitored. A randomized evaluation compares appropriately targeted support with usual service, because predictive accuracy alone cannot establish that an offer reduces cancellation.

Outcome

The model performs better than the inactivity baseline on the held-out period, but its estimated business value is modest after contact costs. The evaluation also shows that a supportive onboarding intervention for new subscribers is more effective than a broad discount. The company therefore deploys a limited pilot, monitors subgroup performance and drift, and schedules a review after one quarter.

Learning points

The case demonstrates that a high headline accuracy can conceal leakage, dependence, class imbalance, and unequal performance. It also shows why prediction should be separated from causal evaluation: identifying likely cancellers does not prove that an intervention will retain them.

Challenges, governance, and contemporary developments

Computational research faces continuing technical and institutional challenges. Data and models drift as customer behavior, interfaces, policies, and economic conditions change. Distributed data may be difficult to link without increasing privacy risk. Synthetic data can assist development but may fail to preserve rare cases or may reproduce source-data privacy risks. Federated learning and privacy-enhancing technologies can reduce centralized collection, but they introduce communication, security, and evaluation challenges.

Regulatory expectations increasingly emphasize lawful processing, purpose limitation, data minimization, security, transparency, risk management, and accountability. Requirements differ across jurisdictions and applications; researchers should consult applicable privacy, consumer-protection, employment, sectoral, and AI regulations rather than assume that public data are unrestricted. Institutional review may be appropriate even for secondary digital data, especially where individuals or vulnerable groups may be affected.

Best practice is proportionate governance. A low-risk exploratory analysis may require documentation and privacy safeguards; a system influencing credit, employment, education, health, or access to essential services requires stronger validation, human oversight, impact assessment, monitoring, and avenues for appeal. Governance is not an administrative addition to methodology: it determines whether the evidence can be responsibly used.

Chapter Summary

Computational research uses digital traces, machine learning, text analytics, social-media data, and AI to study business and social phenomena at scale. Big data offer volume and variety but may suffer from selection, measurement, privacy, and quality problems. Researchers must distinguish description, prediction, explanation, and causal inference: predictive association does not establish mechanism or causality.

Machine-learning studies require a defensible separation of training, validation, and test data, with temporal or group-based splitting where appropriate. Data leakage produces falsely optimistic results. Metrics must match the decision context, and performance should be compared with simple baselines and examined across subgroups. Bias can arise throughout the data and modeling pipeline; fairness criteria involve trade-offs and require contextual judgement. Explainability can improve accountability but is not equivalent to causal explanation.

Text and social-media analysis require clear units of analysis, validated coding, attention to bots and representativeness, and respect for privacy. Generative AI may assist research but must be used transparently, with human verification, secure data practices, and disclosure. A defensible computational project links technical validation to substantive theory, ethical governance, reproducibility, monitoring, and responsible decision-making.

Key Terms

Big data: Digital or other datasets whose scale, speed, variety, or complexity creates methodological and computational challenges.

Data leakage: Use of information in model development that would not be available at the intended prediction time.

Data drift: Change over time in the distribution of inputs, outcomes, or relationships used by a model.

Explainability: The extent to which a model or output can be understood by relevant people.

Generative AI: AI that produces new content such as text, code, images, or audio.

Machine learning: Methods that learn patterns from data for prediction, classification, ranking, or representation.

Out-of-sample performance: Model performance on observations not used to fit or select the model.

Precision: The proportion of predicted positive cases that are actually positive.

Recall: The proportion of actual positive cases correctly identified.

Calibration: Agreement between predicted probabilities and observed frequencies.

Text analytics: Computational analysis that converts language data into structured evidence.

TF–IDF: A term-weighting method combining within-document frequency with rarity across documents.

Algorithmic bias: Systematic, consequential error or unequal performance associated with data, models, or deployment.

Fairness: A family of principles concerning equitable treatment, errors, benefits, and burdens across groups.

Prediction: Estimation of an unknown or future outcome.

Causal inference: Estimation of effects under interventions or counterfactual conditions.

Training data: Data used to estimate model parameters.

Validation data: Data used to tune models or select among candidate specifications.

Test data: Data held back for final performance evaluation.

Data-generating process: The institutional, behavioral, and technical process that produces observed data.

Review Questions

Short Answer Questions

1. Why can a large digital dataset still provide weak evidence for a research question?
2. Distinguish prediction from causal inference.
3. What is data leakage? Give one example.
4. Why can accuracy be misleading for an imbalanced classification problem?
5. State two safeguards for responsible use of generative AI in research.

Long Answer Questions

1. Explain how training, validation, and test data should be designed for a time-dependent business prediction problem.
2. Discuss the main sources of bias in a social-media text-analysis study and propose controls.
3. Compare description, prediction, explanation, and causal inference as research purposes.
4. Explain why explainability does not by itself establish causality or fairness.
5. Design a governance framework for an AI system used to prioritize customer interventions.

Analytical / Application Questions

1. A fraud model identifies 90% of fraudulent transactions but also flags many legitimate customers. Which metrics and organizational costs should be examined before deployment? Justify your answer.
2. A researcher predicts employee turnover using a “termination interview outcome” variable recorded after employees leave. Diagnose the methodological problem and redesign the feature-construction and validation process.
3. A sentiment model performs well overall but poorly for one language group. Develop an investigation and remediation plan that addresses data, measurement, fairness, and deployment monitoring.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes prediction from explanation?

A. Prediction estimates outcomes, whereas explanation seeks mechanisms or reasons.

B. Prediction always establishes causation, whereas explanation never uses data.

C. Prediction applies only to numerical outcomes, whereas explanation applies only to text.

D. Prediction requires experiments, whereas explanation requires no validation.

Correct Answer: A. Prediction estimates outcomes, whereas explanation seeks mechanisms or reasons.

Question 2

Which practice most directly prevents temporal data leakage?

A. Increasing the number of model features

B. Using only information available at the prediction time

C. Removing all sensitive variables

D. Reporting accuracy instead of recall

Correct Answer: B. Using only information available at the prediction time

Question 3

A classifier has TP = 80 and FP = 20. What is its precision?

A. 60%

B. 75%

C. 80%

D. That value cannot be calculated without FN

Correct Answer: C. 80%

Question 4

Why may accuracy be inappropriate for a rare-event classification problem?

A. It cannot be computed for binary outcomes

B. A majority-class model may achieve high accuracy while missing most rare cases

C. It always gives lower values than recall

D. It measures only the cost of false positives

Correct Answer: B. A majority-class model may achieve high accuracy while missing most rare cases

Question 5

Which dataset is normally held back for final evaluation after model selection?

A. Training data

B. Validation data

C. Test data

D. Imputation data

Correct Answer: C. Test data

Question 6

What does TF–IDF generally give greater weight to?

A. Terms common in every document

B. Terms frequent in a document but uncommon across the collection

C. Only terms appearing once

D. Words selected by a human coder only

Correct Answer: B. Terms frequent in a document but uncommon across the collection

Question 7

Which statement about feature importance is correct?

A. It automatically proves that a feature is causal

B. It can indicate model reliance but does not by itself establish causality

C. It eliminates the need for subgroup evaluation

D. It is identical across all algorithms

Correct Answer: B. It can indicate model reliance but does not by itself establish causality

Question 8

Why can removing a sensitive attribute fail to remove algorithmic bias?

A. Models cannot operate without sensitive attributes

B. Proxy variables may still encode information related to the attribute

C. Sensitive attributes determine the test-set size

D. Removing variables always creates data leakage

Correct Answer: B. Proxy variables may still encode information related to the attribute

Question 9

Which is an appropriate practice when using generative AI to draft research text?

A. Accepting citations without checking them

B. Uploading confidential participant data to any public tool

C. Verifying claims and disclosing material AI assistance

D. Treating generated text as primary evidence

Correct Answer: C. Verifying claims and disclosing material AI assistance

Question 10

Why should a predictive retention model not automatically be treated as evidence that an intervention works?

A. Prediction and causal intervention answer different questions

B. Retention models cannot use customer data

C. Interventions never affect customer behavior

D. Causal effects can be inferred from accuracy alone

Correct Answer: A. Prediction and causal intervention answer different questions