Bachelors Program in Computer Science Engineering
Data Mining
•
Bachelor
Emerging Directions in Data Mining
Introduction and Context
Data mining has evolved from discovering patterns in structured databases to learning representations, generating content, reasoning over heterogeneous evidence, and supporting decisions in dynamic environments. Classical methods such as decision trees, association-rule mining, clustering, and support-vector machines remain valuable, but contemporary systems increasingly combine them with deep learning, large pretrained models, distributed computation, privacy technologies, and human expertise.
The central change is methodological. Traditional data mining often assumes that useful features have been designed in advance. Emerging systems attempt to learn representations automatically, integrate text, images, audio, graphs, and sensor streams, and use external knowledge during inference. At the same time, these systems introduce serious concerns: hallucination, bias, privacy leakage, energy consumption, weak causal interpretation, unequal access, and unclear accountability.
This chapter examines major directions shaping data mining practice and research. The emphasis is not on treating every new model as automatically superior, but on understanding when a method is appropriate, how it should be evaluated, and what safeguards are required.
Learning Objectives
After studying this chapter, students should be able to:
Explain deep representation learning and distinguish supervised, self-supervised, and contrastive learning.
Describe generative AI, foundation models, and retrieval-augmented generation (RAG).
Distinguish prediction from causal discovery and identify important assumptions.
Explain federated learning, differential privacy, secure aggregation, and synthetic data generation.
Analyze multimodal, edge, and human-centered data-mining systems.
Select suitable evaluation metrics and identify risks involving bias, privacy, reliability, and governance.
Design a responsible data-mining workflow for a contemporary application.
1. From Hand-Crafted Features to Learned Representations
A representation is a transformation of raw observations into a form that makes a task easier. In classical mining, domain experts might convert a transaction into counts, averages, or indicator variables. Deep representation learning instead learns a function that maps an input x to an embedding z:
z = f_θ(x)
Here, x is the input, z is a learned vector representation, f is a parameterized model, and θ denotes its parameters. The representation is useful when similar or task-related observations occupy nearby regions of the representation space.
A supervised model learns representations by minimizing a task loss. For n labeled examples, a general empirical-risk objective is:
θ* = arg min_θ (1/n) ∑ᵢ₌₁ⁿ L(g_θ(xᵢ), yᵢ)
where yᵢ is the target, gθ is the complete predictor, and L is a loss function. The expression assumes that examples are sufficiently representative of the deployment population and that the loss reflects the application’s costs.
Deep networks learn successive abstractions. Convolutional networks are effective for spatial data; recurrent and transformer architectures model sequences; graph neural networks propagate information over relations. Their strengths include reduced manual feature engineering and transferability. Their weaknesses include high data and computational requirements, sensitivity to distribution shift, and limited interpretability.
Self-Supervised and Contrastive Learning
Self-supervised learning creates training signals from the data itself. Examples include predicting masked words, reconstructing a corrupted image, or determining whether two augmented views came from the same object. In contrastive learning, an anchor is encouraged to be close to a positive example and separated from negative examples. A common objective is the InfoNCE loss:
Here, zᵢ is an anchor representation, z⁺ᵢ is its positive counterpart, zⱼ are candidate representations, sim is a similarity function such as cosine similarity, and τ > 0 is a temperature parameter. The loss is applicable when positive and negative pairs are meaningfully defined. False negatives, inappropriate augmentations, and sampling bias can produce misleading representations.
Embeddings support clustering, nearest-neighbor search, anomaly detection, recommendation, and transfer learning. However, geometric proximity does not necessarily imply semantic equivalence or causal similarity. Embedding quality must therefore be tested using both intrinsic measures, such as retrieval similarity, and task-specific measures, such as precision, recall, calibration, and downstream utility.
2. Generative AI and Foundation Models
Discriminative models estimate or approximate a relationship from inputs to labels, while generative models learn a distribution from which new data can be produced. If X is a random variable representing observations, a generative model seeks to approximate p(X), or p(X,Y) when labels are included. Generation may use autoregressive models, variational autoencoders, generative adversarial networks, or diffusion models.
An autoregressive model factorizes a sequence as:
p(x₁,…,xT) = ∏ₜ₌₁ᵀ p(xₜ | x₁,…,xₜ₋₁)
The model predicts each element conditional on preceding elements. For language, elements are usually tokens; for other domains they may be image patches, events, or code units. This factorization does not prove that generated content is true; it describes statistical generation.
A foundation model is trained on broad data at substantial scale and adapted to many downstream tasks. Adaptation may involve prompting, fine-tuning, parameter-efficient tuning, or training a task-specific head. Large language models are foundation models for text and code, but the broader category includes vision, speech, multimodal, and scientific models.
Important capabilities include:
Transfer learning: knowledge acquired from pretraining is reused for another task.
In-context learning: examples supplied in a prompt influence the response without changing model parameters.
Embedding generation: inputs are mapped to vectors for search or classification.
Tool use: a model invokes databases, calculators, APIs, or programs.
Multimodal reasoning: information from different data types is processed jointly or through aligned encoders.
These capabilities do not eliminate the need for data mining. They alter the pipeline: data curation, schema design, retrieval, evaluation, provenance, and monitoring become more important.
Retrieval-Augmented Systems
A retrieval-augmented system separates evidence retrieval from response generation. A typical pipeline is:
Retrieval augmented data mining pipeline
Figure: Retrieval augmented data mining pipeline
Figure: A retrieval-augmented system combines search evidence with generative inference.
Documents are normally cleaned, divided into chunks, embedded, indexed, retrieved, reranked, and inserted into a prompt. RAG can reduce unsupported responses by grounding generation in current or organization-specific evidence. It does not guarantee correctness: retrieval may miss the relevant passage, documents may be wrong, and the model may misinterpret evidence. Evaluation should separately measure retrieval recall, citation correctness, answer faithfulness, latency, and user success.
Best practices include access-controlled indexes, source timestamps, document-level provenance, adversarial testing, refusal when evidence is insufficient, and human review for high-impact decisions.
3. Causal Discovery and Beyond-Correlation Mining
Prediction asks, “What is likely to occur?” Causal analysis asks, “What would change if an intervention occurred?” A correlation found by mining may be useful for prediction but can be unsafe for intervention.
A causal graph represents variables as nodes and proposed direct relationships as directed edges. Under assumptions such as causal sufficiency, acyclicity for a directed acyclic graph, and appropriate measurement, algorithms may use conditional independencies to infer candidate structures. Constraint-based methods test independence relationships; score-based methods compare graph structures using a scoring criterion; functional causal models impose equations and noise assumptions.
For treatment T, outcome Y, and covariates X, a simplified causal estimand is the average treatment effect:
ATE = E[Y(1) − Y(0)]
Y(1) and Y(0) are potential outcomes under treatment and control. The two potential outcomes for the same unit cannot both be observed, so identification requires assumptions. Under consistency, positivity, and conditional exchangeability given X, adjustment can be expressed as:
ATE = EX[E(Y | T=1,X) − E(Y | T=0,X)]
The formula applies only when X blocks relevant backdoor paths and is not itself affected by treatment. Unmeasured confounding, selection bias, measurement error, and time-varying effects can invalidate conclusions.
Causal discovery should therefore be treated as hypothesis generation, not automatic proof of causation. Domain knowledge, experiments, natural experiments, temporal information, and sensitivity analysis are essential. In practice, data-mining teams should distinguish clearly among association, prediction, causal estimation, and causal discovery.
4. Federated and Privacy-Preserving Data Mining
Centralizing data can improve modeling but may violate legal, ethical, or organizational constraints. Federated learning keeps training data at participating clients and exchanges model updates. In a basic round, client k computes an update using local data and the server aggregates updates, often by weighted averaging:
wt+1 = ∑ₖ₌₁ᴷ (nₖ / ∑ⱼ nⱼ) wₜ₊₁⁽ᵏ⁾
wₜ₊₁⁽ᵏ⁾ is the locally updated model, nₖ is the number of local training examples, and K is the number of participating clients. The method assumes that updates can be communicated reliably and that weighted averaging is appropriate. Non-independent, non-identically distributed client data can cause instability and unequal performance.
Privacy is not achieved merely because raw data remain local. Model updates may leak information, and malicious clients may poison training. Important safeguards include:
Secure aggregation: the server observes only an aggregate update, not each individual update.
Differential privacy: randomized noise limits the effect of any one record on released results.
Encryption and access control: protect data and communication channels.
Robust aggregation: reduce the influence of anomalous or malicious updates.
Auditability: record participation, versions, permissions, and incidents.
A mechanism M is ε-differentially private if, for neighboring datasets D and D′ differing in one individual, and every output set S:
Pr[M(D) ∈ S] ≤ e^ε Pr[M(D′) ∈ S] + δ
ε ≥ 0 is the privacy-loss parameter and δ ≥ 0 is a small failure probability. Smaller ε generally provides stronger privacy but may reduce accuracy. Differential privacy is a formal guarantee about outputs, not a promise that every participant is anonymous under every attack.
5. Synthetic Data and Data-Centric Mining
Synthetic data are generated records intended to preserve selected properties of real data without reproducing sensitive individuals. Methods include probabilistic sampling, generative adversarial networks, variational autoencoders, diffusion models, and rule-based simulation.
Synthetic data can support software testing, rare-event analysis, classroom experimentation, and preliminary model development. Their quality should be assessed on three separate dimensions:
Fidelity: similarity to the distribution and relationships in the source data.
Utility: performance of models or analyses using synthetic data.
Privacy: resistance to memorization, linkage, membership inference, and re-identification.
High fidelity is not automatically desirable. A generator that reproduces unique records may increase privacy risk. Conversely, excessive noise may destroy minority patterns and make the data scientifically useless. Synthetic data should be labeled, documented, and never treated as a substitute for validation on appropriately governed real-world data.
6. Multimodal Analytics
Organizations increasingly analyze combinations of text, tables, images, audio, video, graphs, and sensor streams. Multimodal systems may use early fusion, late fusion, intermediate fusion, or cross-attention. Early fusion combines features before prediction; late fusion combines modality-specific predictions; intermediate fusion learns interactions in a shared representation.
A useful design principle is to preserve modality provenance. For example, a medical prediction should record whether evidence came from an image, a clinical note, a laboratory result, or a device stream. Missing modalities and conflicting signals must be handled explicitly. Evaluation should include modality-specific ablations: if removing one modality produces no performance change, the system may not be using it meaningfully; if performance collapses, deployment must plan for its unavailability.
Multimodal mining is vulnerable to shortcut learning. A model may associate a hospital identifier, camera artifact, or documentation style with an outcome instead of learning the intended phenomenon. Stratified testing, external validation, counterfactual checks, and domain-expert review are necessary.
7. Edge Intelligence and Streaming Discovery
Edge intelligence places inference or limited learning near data sources such as cameras, vehicles, industrial machines, and mobile devices. Benefits include lower latency, reduced bandwidth use, local privacy, and resilience when connectivity is poor. Constraints include memory, energy, computation, model update logistics, and device heterogeneity.
A deployment decision should consider end-to-end latency rather than model execution time alone:
Each term is a time component, measured in compatible units such as milliseconds. Cloud processing may increase T_communicate; edge processing may increase T_infer if hardware is constrained. Compression, quantization, pruning, early-exit models, and hardware acceleration can reduce resource use but may alter accuracy.
Streaming mining also faces concept drift: the relationship between inputs and outcomes changes over time. Monitoring should track data distributions, prediction error, calibration, latency, and fairness. A drift alarm should trigger investigation rather than automatic retraining, since temporary events, sensor faults, and attacks can resemble genuine change.
8. Human-Centered Automated Discovery
Automated discovery systems propose clusters, rules, anomalies, causal hypotheses, or explanations. Human-centered design treats domain experts as participants in problem formulation, validation, and governance rather than as final approvers of opaque outputs.
A productive human–machine workflow contains:
Problem framing: define the decision, population, constraints, and harms.
Data interrogation: inspect provenance, missingness, representativeness, and labels.
Candidate discovery: generate patterns with uncertainty and supporting evidence.
Interactive validation: allow experts to reject, refine, or contextualize findings.
Controlled deployment: specify authority, escalation, logging, and rollback.
Post-deployment learning: monitor outcomes and revise the system.
Explanations should be appropriate to the user and purpose. Feature importance may describe model behavior but does not establish causality. A natural-language explanation may be persuasive yet factually wrong. Evidence-linked explanations, uncertainty estimates, examples, and counterexamples are generally more useful than confident prose alone.
9. Evaluation, Governance, and Responsible Practice
Contemporary mining systems require multidimensional evaluation. Accuracy alone is insufficient. A practical evaluation plan may include:
Dimension
Illustrative questions
Predictive performance
How well does the system generalize to unseen data?
Calibration
Do predicted probabilities correspond to observed frequencies?
Robustness
What happens under noise, shift, missing modalities, or attack?
Fairness
Are errors and benefits distributed acceptably across groups?
Privacy
Can records or sensitive attributes be inferred?
Efficiency
Are latency, memory, energy, and cost acceptable?
Human utility
Does the system improve decisions rather than merely scores?
Provenance
Can data, model versions, and evidence be traced?
For binary classification, precision and recall are:
Precision = TP/(TP+FP)
Recall = TP/(TP+FN)
TP, FP, and FN denote true positives, false positives, and false negatives. These metrics assume a defined positive class and should be interpreted with prevalence and application costs. A system for disease screening may prioritize recall, whereas an expensive intervention may require high precision.
Responsible practice includes data minimization, informed and lawful use, documentation, security testing, accessibility, contestability, and clear assignment of responsibility. Teams should maintain dataset documentation, model cards, risk registers, incident procedures, and change logs. Regulatory compliance varies by jurisdiction, but ethical obligations remain even where a specific rule is unclear.
Case Study: Predictive Maintenance for a Distributed Manufacturing Network
Background
Consider an illustrative company operating 2,000 industrial pumps across four factories. Each pump produces vibration, temperature, pressure, maintenance-log text, and occasional technician photographs. The business wants to predict failures early enough to schedule maintenance.
Problem
A centralized model initially performs well in the largest factory but poorly elsewhere. The factories cannot freely exchange raw operational records because of contractual and privacy restrictions. Failures are rare, sensors differ by supplier, and maintenance notes contain inconsistent terminology.
Analysis
The problem is multimodal, imbalanced, distributed, and subject to drift. A random record-level split would leak information because readings from the same pump are temporally related. Evaluation should instead use time-based and factory-based splits. The team must also distinguish predictive association from causal claims: a high temperature may predict failure without being the intervention that prevents it.
Approach and Solution
The team develops a shared representation using vibration windows and normalized sensor features, while a language encoder represents maintenance notes. A multimodal fusion layer combines the representations only when the relevant modality is available. Each factory trains locally using federated learning; secure aggregation prevents the server from inspecting individual updates. Differential privacy is considered for released analytics, with ε selected through a documented privacy–utility analysis rather than arbitrarily.
A retrieval component links each alert to maintenance manuals, prior verified incidents, and approved replacement procedures. It does not authorize repairs automatically. The dashboard displays predicted risk, evidence, sensor quality, uncertainty, and the option to reject the recommendation. Technicians record outcomes, creating feedback for monitoring and improvement.
Outcome
In this illustrative scenario, the system reduces missed high-risk events during controlled evaluation, but performance differs by factory. The team therefore deploys gradually, monitors subgroup and site-level performance, and retains a non-automated fallback. It discovers that one sensor vendor’s calibration artifact was a major predictive shortcut and removes the artifact before broader deployment.
Learning Points
The case demonstrates that emerging mining is a systems problem. Representation learning improves feature construction, federated learning addresses data locality, retrieval supplies operational evidence, and human review controls risk. None of these components removes the need for sound sampling, leakage prevention, monitoring, documentation, and accountability.
Best-Practice Design Checklist
Before deployment, a data-mining team should:
define the decision and harm model before selecting a model;
document data origin, consent, quality, missingness, and representativeness;
establish temporal, external, and subgroup validation sets;
compare against simple and interpretable baselines;
test for leakage, shortcut learning, privacy attacks, and adversarial inputs;
report uncertainty, calibration, costs, latency, and energy use;
preserve provenance for data, models, retrieved evidence, and outputs;
design human override, appeal, incident response, and rollback procedures;
monitor drift and fairness after deployment;
reassess whether automation is justified as conditions change.
Contemporary Research Directions
Research is moving toward smaller specialized models, efficient adaptation, multimodal foundation models, graph–language systems, privacy-preserving retrieval, continual learning, causal representation learning, and agents that use tools under explicit constraints. Important open problems include reliable uncertainty estimation for generative systems, evaluation under distribution shift, privacy guarantees for foundation models, energy-efficient training, causal reasoning with imperfect data, and mechanisms for meaningful human oversight.
A mature view of innovation asks not only whether a new method achieves a higher benchmark score, but also whether it is reproducible, secure, affordable, interpretable enough for its purpose, and beneficial to affected communities.
Chapter Summary
Emerging data mining combines learned representations with generative models, foundation models, retrieval, causal methods, privacy technologies, synthetic data, multimodal analytics, edge intelligence, and human-centered workflows. Deep representations reduce manual feature engineering but require careful validation. Generative and foundation models provide broad capabilities but can hallucinate, encode bias, and reproduce private information. Retrieval grounds outputs in external evidence but must be evaluated independently.
Causal discovery and causal estimation require assumptions that differ from predictive modeling. Federated learning keeps data local, while secure aggregation and differential privacy address additional risks. Synthetic data must be judged by fidelity, utility, and privacy. Multimodal and edge systems introduce integration, latency, drift, and resource challenges. Across all directions, responsible deployment depends on provenance, robust evaluation, fairness, security, human oversight, and governance.
Key Terms
Representation learning: Learning features or embeddings directly from data.
Embedding: A numerical vector representing an object, record, or modality.
Self-supervised learning: Learning from targets constructed from the data itself.
Contrastive learning: Learning representations by bringing positive pairs together and separating negatives.
Generative model: A model that approximates a data distribution and can produce new samples.
Foundation model: A broadly pretrained model adapted to many downstream tasks.
Prompting: Supplying instructions or examples to guide a pretrained model.
Retrieval-augmented generation: Generating outputs using evidence retrieved from an external store.
Hallucination: An unsupported or factually incorrect generated output.
Causal discovery: Inferring candidate causal structures from data and assumptions.
Average treatment effect: The expected difference between potential outcomes under treatment and control.
Federated learning: Collaborative model training in which raw client data remain local.
Secure aggregation: A protocol that reveals an aggregate update without exposing individual updates.
Differential privacy: A formal guarantee limiting the influence of one record on an output.
Synthetic data: Artificially generated data intended to preserve selected properties of real data.
Multimodal analytics: Mining information from multiple data types jointly.
Edge intelligence: Data processing or learning performed near the data source.
Concept drift: Change over time in data distributions or predictive relationships.
Provenance: Traceable information about data origin, processing, models, and evidence.
Human-centered automation: Automation designed around human goals, expertise, control, and accountability.
Review Questions
Short Answer Questions
What is the difference between supervised and self-supervised representation learning?
Why does retrieval augmentation not guarantee factual correctness?
State two assumptions needed for interpreting an adjustment formula as a causal effect.
What problem does secure aggregation address in federated learning?
Name the three principal dimensions used to evaluate synthetic data.
Long Answer Questions
Explain the architecture, benefits, and limitations of a retrieval-augmented data-mining system.
Compare centralized, federated, and privacy-preserving data mining.
Discuss why predictive accuracy is insufficient for evaluating a high-impact mining system.
Explain how multimodal analytics can suffer from shortcut learning and how it can be detected.
Describe a human-centered workflow for automated discovery from problem formulation through monitoring.
Analytical / Application Questions
Design an evaluation plan for a federated predictive-maintenance system deployed across factories with different sensors and failure rates.
A model predicts hospital readmission using clinical notes, images, and structured records. Identify privacy, fairness, causal, and missing-modality risks, and propose safeguards.
An organization wants to release synthetic customer data for research. Develop a testing framework covering fidelity, analytical utility, privacy, documentation, and release governance.
Multiple Choice Questions with Answers
Question 1
Which statement best describes a foundation model?
A. A model trained only on one small, labeled dataset
B. A broadly pretrained model adapted to multiple downstream tasks
C. A database containing only numerical features
D. A rule system that cannot be fine-tuned
Correct Answer: B. A broadly pretrained model adapted to multiple downstream tasks
Question 2
What is the primary purpose of retrieval-augmented generation?
A. To eliminate the need for any training data
B. To replace all databases with generated text
C. To provide external evidence to support model generation
D. To guarantee that every generated statement is true
Correct Answer: C. To provide external evidence to support model generation
Question 3
In the contrastive loss presented in the chapter, what does τ represent?
A. The number of training examples
B. A temperature parameter controlling similarity scaling
C. The target label
D. The model learning rate only
Correct Answer: B. A temperature parameter controlling similarity scaling
Question 4
Which statement correctly distinguishes prediction from causal analysis?
A. Prediction always identifies intervention effects
B. Causal analysis never uses observational data
C. Prediction concerns likely outcomes, whereas causal analysis concerns interventions
D. They are identical whenever a model has high accuracy