Unsupervised Learning and Representation Discovery

Introduction and Context

Unsupervised learning addresses problems in which observations are available without reliable target labels. Its purpose is not simply to “find groups,” but to discover useful structure: coherent segments, low-dimensional representations, latent factors, density variations, unusual observations, and relationships that can support subsequent prediction or decision-making. In advanced machine learning, unsupervised methods are best understood as modelling assumptions about how data were generated.

A dataset may contain observations \(x_1,\ldots,x_n\), where each \(x_i\in\mathbb{R}^p\) is described by \(p\) variables. Unlike supervised learning, there is no response variable \(y\) supplied for training. The analyst must therefore determine what constitutes meaningful structure and how its usefulness will be assessed. This creates both opportunities and risks: unsupervised methods can reveal previously unknown patterns, but they can also produce visually attractive or statistically coherent structures with little practical meaning.

This chapter examines clustering, dimensionality reduction, manifold learning, matrix factorization, anomaly detection, and density estimation. It emphasizes assumptions, mathematical objectives, validation, scalability, interpretation, and responsible application.

Learning Objectives

After studying this chapter, a student should be able to:

1. Explain the distinction between clustering, representation learning, density estimation, and anomaly detection.

2. Select and configure clustering algorithms according to data geometry, noise, scale, and operational requirements.

3. Derive and interpret the principal objectives of k-means, principal component analysis, and matrix factorization.

4. Compare linear dimensionality reduction with nonlinear manifold-learning methods.

5. Evaluate unsupervised results using internal, external, stability, and domain-based criteria.

6. Design an exploratory analytics workflow that addresses preprocessing, leakage, reproducibility, fairness, and governance.

1. A Framework for Unsupervised Learning

Unsupervised learning can be organized around four questions:

• Grouping: Which observations are similar enough to form segments?
• Representation: Can the data be expressed using fewer or more meaningful variables?
• Probability: Where is the data concentrated, and how can its distribution be estimated?
• Deviation: Which observations are inconsistent with the dominant structure?

These tasks overlap. A density estimator may identify low-density observations as anomalies; a low-dimensional representation may make clusters visible; and a factor model may provide features for clustering. Nevertheless, each method answers a different modelling question.

1.1 Similarity, distance, and scale

Most unsupervised algorithms rely on a similarity or distance measure. The Euclidean distance between vectors \(x\) and \(z\) is

\[

d(x,z)=\sqrt{\sum_{j=1}^{p}(x_j-z_j)^2}.

\]

It is appropriate when variables are numerically comparable and differences have meaningful magnitude. Squared Euclidean distance is often used computationally because it preserves the ordering of distances while avoiding square roots. Manhattan distance, \(\sum_j|x_j-z_j|\), can be more robust to coordinate-wise differences, while cosine similarity emphasizes direction rather than magnitude.

Feature scaling is essential when variables use different units. Standardization transforms variable \(j\) as

\[

z_{ij}=\frac{x_{ij}-\bar{x}_j}{s_j},

\]

where \(\bar{x}_j\) is the sample mean and \(s_j\) is the sample standard deviation. This is appropriate when equal standardized variation should receive comparable weight. It may be inappropriate when the original measurement scale is substantively meaningful or when extreme outliers make mean and standard deviation unstable. Robust scaling, log transformations, or domain-specific weighting may then be preferable.

Mixed data require suitable dissimilarities. For example, categorical variables should not automatically be encoded as arbitrary integers and treated as continuous measurements. Gower-type dissimilarities, one-hot representations with carefully chosen weights, or models designed for categorical data may be more defensible.

2. Clustering Methods

Clustering seeks a partition or hierarchy that makes observations within a group more similar than observations across groups. There is no universally correct clustering algorithm because “cluster” can mean a spherical concentration, a connected region, a density peak, or a latent probabilistic component.

2.1 k-means clustering

Given a chosen number of clusters \(K\), k-means estimates centroids \(\mu_1,\ldots,\mu_K\) by minimizing within-cluster squared error:

\[

\min_{C_1,\ldots,C_K,\mu_1,\ldots,\mu_K}

\sum_{k=1}^{K}\sum_{x_i\in C_k}\|x_i-\mu_k\|_2^2.

\]

For fixed assignments, the minimizing centroid is the arithmetic mean of the observations in that cluster. The standard algorithm alternates between assigning each observation to its nearest centroid and recomputing centroids. It terminates at a local optimum, so multiple initializations are important.

The objective assumes that Euclidean distance is meaningful and tends to favour compact, approximately spherical clusters with similar within-cluster scale. It is sensitive to outliers and to feature scaling. The selected \(K\) should not be treated as a discovered natural law. It may be chosen using domain requirements, stability, the elbow pattern in within-cluster sum of squares, silhouette analysis, or downstream usefulness.

For an observation \(x_i\), its silhouette coefficient is

\[

s_i=\frac{b_i-a_i}{\max(a_i,b_i)},

\]

where \(a_i\) is its mean distance to observations in its own cluster and \(b_i\) is the smallest mean distance to observations in another cluster. Values near 1 indicate separation, values near 0 indicate overlap, and negative values suggest possible misassignment. Silhouette values depend on the chosen distance and can favour particular cluster geometries.

2.2 Hierarchical clustering

Agglomerative hierarchical clustering begins with each observation as a separate cluster and repeatedly merges clusters. The result is represented by a dendrogram. A linkage rule defines distance between clusters:

• Single linkage: minimum pairwise distance; can detect elongated structures but is vulnerable to chaining.
• Complete linkage: maximum pairwise distance; favours compact clusters.
• Average linkage: average pairwise distance; provides an intermediate behaviour.
• Ward linkage: merges clusters producing the smallest increase in within-cluster squared error; it is principally associated with Euclidean numerical data.

A dendrogram permits different numbers of clusters to be obtained by cutting at different heights. However, the hierarchy is dependent on the distance, linkage, scaling, and noise. It should not be interpreted as a definitive taxonomy without stability and domain validation.

2.3 Density-based clustering

DBSCAN defines dense regions using a neighbourhood radius \(\varepsilon\) and a minimum number of points. A core point has at least the specified number of observations in its \(\varepsilon\)-neighbourhood. Density-connected observations form clusters, while some observations are labelled noise.

DBSCAN can discover non-spherical clusters and explicitly identify noise, but it struggles when clusters have very different densities or when the feature dimension is high. Its parameters are scale-dependent. The related HDBSCAN approach builds a density hierarchy and can be more flexible for variable-density data, although parameter interpretation and computational cost still require care.

2.4 Mixture models

A Gaussian mixture model assumes that the observed distribution is a weighted sum of \(K\) Gaussian components:

\[

p(x)=\sum_{k=1}^{K}\pi_k\,\mathcal{N}(x\mid\mu_k,\Sigma_k),

\]

where \(\pi_k\geq0\), \(\sum_k\pi_k=1\), \(\mu_k\) is a component mean, and \(\Sigma_k\) is a covariance matrix. Unlike hard clustering, it provides posterior membership probabilities:

\[

p(z_i=k\mid x_i)=

\frac{\pi_k\mathcal{N}(x_i\mid\mu_k,\Sigma_k)}

{\sum_{l=1}^{K}\pi_l\mathcal{N}(x_i\mid\mu_l,\Sigma_l)}.

\]

Parameters are commonly estimated using expectation-maximization (EM). The E-step computes membership responsibilities; the M-step updates mixture weights, means, and covariance matrices. Mixture models support elliptical clusters and uncertainty-aware assignments, but Gaussian assumptions may be inappropriate, covariance estimates can become singular, and increasing \(K\) can overfit. Regularized covariance estimation and model-selection criteria such as AIC or BIC may help, but information criteria do not replace substantive validation.

3. Dimensionality Reduction and Representation Discovery

High-dimensional data often contain redundant, noisy, or correlated variables. Dimensionality reduction seeks a mapping from \(\mathbb{R}^p\) to \(\mathbb{R}^q\), where \(q<p\), while preserving a selected notion of information.

3.1 Principal component analysis

For centered data matrix \(X\in\mathbb{R}^{n\times p}\), PCA finds orthogonal directions of maximum variance. The first loading vector solves

\[

\max_{\|w\|_2=1} \|Xw\|_2^2.

\]

Equivalently, if \(S=X^TX/(n-1)\) is the sample covariance matrix, \(w\) is an eigenvector of \(S\). Subsequent components maximize variance subject to orthogonality to earlier components. The score representation is \(Z=XW_q\), where \(W_q\) contains the first \(q\) loading vectors.

The variance explained by component \(j\) is its eigenvalue \(\lambda_j\), and the proportion explained by the first \(q\) components is

\[

\frac{\sum_{j=1}^{q}\lambda_j}{\sum_{j=1}^{p}\lambda_j}.

\]

PCA is linear and sensitive to scaling and outliers. Applying PCA to a covariance matrix gives high-variance variables greater influence; applying it to standardized variables gives each variable initially equal variance. The reconstruction error under rank-\(q\) orthogonal projection is minimized by the first \(q\) principal components, making PCA useful for compression and denoising under a squared-error criterion. High explained variance, however, does not guarantee preservation of predictive or causal information.

3.2 Singular value decomposition

The singular value decomposition (SVD) expresses a matrix as

\[

X=U\Sigma V^T,

\]

where \(U\) and \(V\) have orthonormal columns and \(\Sigma\) contains nonnegative singular values. A rank-\(q\) approximation is

\[

X_q=U_q\Sigma_qV_q^T.

\]

For the Frobenius norm, the Eckart–Young theorem states that this truncated SVD is an optimal rank-\(q\) approximation. In practice, randomized SVD is often used for large sparse matrices such as document-term matrices. SVD underlies latent semantic analysis, image compression, and many recommender-system pipelines.

3.3 Nonlinear manifold learning

A manifold is a lower-dimensional structure embedded in a higher-dimensional space. Manifold-learning methods attempt to preserve local or global geometry that linear projections cannot capture.

• Isomap constructs a neighbourhood graph, estimates geodesic distances along the graph, and applies classical multidimensional scaling. It can unfold curved manifolds but is sensitive to graph connectivity and shortcuts.
• Locally Linear Embedding (LLE) reconstructs each point from nearby points and seeks low-dimensional coordinates preserving those reconstruction weights.
• Laplacian Eigenmaps preserve local adjacency relationships through a graph-based objective.
• t-SNE constructs probability distributions representing neighbourhoods in high-dimensional and low-dimensional spaces and minimizes their divergence. It is useful for visualization but can distort global distances, cluster sizes, and apparent separation.
• UMAP uses a graph-based fuzzy-topological construction and optimized low-dimensional layout. It often provides useful visualization and exploratory embeddings, but its output remains parameter- and implementation-dependent.

Two-dimensional embeddings should not automatically be used as quantitative evidence of clusters. Random seeds, perplexity or neighbourhood parameters, preprocessing, and sample composition can materially change the display. For scientific inference, embeddings should be accompanied by sensitivity analysis and, where possible, analysis in the original feature space.

4. Matrix Factorization and Latent Factors

Matrix factorization represents a data matrix through lower-dimensional latent factors. A basic factorization seeks

\[

X\approx WH^T,

\]

where \(W\in\mathbb{R}^{n\times r}\), \(H\in\mathbb{R}^{p\times r}\), and \(r\ll\min(n,p)\). A common objective is

\[

\min_{W,H}\|X-WH^T\|_F^2.

\]

Because the factorization is not unique—transformations of the latent coordinates can produce the same product—interpretation requires constraints or domain structure.

Non-negative matrix factorization (NMF) imposes \(W\geq0\) and \(H\geq0\), often producing parts-based, additive representations. For documents, factors may resemble topics; for images, they may represent additive visual components. NMF is not automatically a probabilistic topic model, and the number of components must be selected and validated.

Probabilistic latent semantic analysis and latent Dirichlet allocation (LDA) model documents as mixtures of latent topics and topics as distributions over words. They are particularly suited to count data, but topic labels are analyst interpretations rather than observed truths. Topic coherence, held-out likelihood, stability, and expert review can be used together in evaluation.

In recommender systems, matrix factorization approximates a user-item rating matrix using user and item latent vectors. Because ratings are usually sparse and missingness may be informative, algorithms should optimize only observed entries and consider exposure bias. A high reconstruction score on observed ratings does not necessarily imply fair or effective recommendations.

5. Density Estimation and Anomaly Detection

5.1 Density estimation

Density estimation seeks an estimate \(\hat{p}(x)\) of the probability density. A histogram is simple but suffers from bin-width and dimensionality problems. A kernel density estimator is

\[

\hat{p}_h(x)=\frac{1}{nh^p}\sum_{i=1}^{n}K\left(\frac{x-x_i}{h}\right),

\]

where \(K\) is a kernel function, \(h>0\) is a bandwidth, and \(p\) is dimensionality. The factor \(h^p\) ensures correct density scaling in \(p\) dimensions. Larger bandwidths produce smoother estimates and lower variance but may conceal modes; smaller bandwidths retain detail but may overfit. KDE becomes difficult in high dimensions because observations become sparse relative to volume, a manifestation of the curse of dimensionality.

Parametric models, such as Gaussian mixtures, are more scalable in some settings but impose stronger distributional assumptions. Normalizing flows and autoregressive density models can represent complex distributions, but likelihood quality may not correspond directly to semantic typicality, especially in high-dimensional domains.

5.2 Anomaly detection

An anomaly is an observation that deviates from a reference notion of normality. The notion may be global, local, contextual, or collective:

• Point anomaly: one observation is unusual.
• Contextual anomaly: an observation is unusual under a particular time, location, or condition.
• Collective anomaly: a sequence or set is unusual although individual points may appear normal.

Statistical methods use tail probabilities or robust distances. The squared Mahalanobis distance is

\[

d_M^2(x)=(x-\mu)^T\Sigma^{-1}(x-\mu),

\]

assuming a meaningful mean vector \(\mu\) and nonsingular covariance matrix \(\Sigma\). Under a multivariate Gaussian model, this quantity has a chi-square distribution with \(p\) degrees of freedom in the population idealization, but practical thresholds require estimation uncertainty and robust treatment of contamination.

Isolation Forest detects observations that are isolated by random recursive partitions; anomalies tend to require fewer splits. One-class support vector machines estimate a boundary around normal data. Local Outlier Factor compares local density with the density of neighbouring observations. Autoencoders can identify high reconstruction error, but they may reconstruct anomalies well if the model is overexpressive or if anomalies are common in training data.

An anomaly score is not automatically a fraud probability or a security incident. Thresholds should reflect investigation capacity, false-positive costs, drift, and feedback from domain experts. Evaluation should use time-aware or otherwise leakage-resistant validation, especially when anomalies are rare.

6. Validation, Interpretation, and Reproducibility

Unsupervised evaluation is intrinsically more difficult because labels are absent or incomplete.

1. Internal criteria: within-cluster dispersion, silhouette, Calinski–Harabasz, or density-based measures assess geometry under particular assumptions.

2. External criteria: when trusted labels exist, adjusted Rand index, normalized mutual information, or cluster purity can compare discovered groups with reference categories. These labels should be treated as evaluation evidence, not necessarily as the correct target.

3. Stability: rerun methods under resampling, perturbation, alternative initializations, and nearby hyperparameters. A structure that disappears under small changes is weak evidence.

4. Predictive utility: test whether representations improve a pre-specified downstream task using a clean evaluation design.

5. Domain validity: consult subject-matter experts and assess whether clusters support plausible, actionable interpretations.

Cluster labels can encode socioeconomic or demographic characteristics even when protected variables are excluded. Analysts should inspect group composition, unequal error consequences, and whether a segmentation may enable discriminatory treatment. Sensitive data should be minimized, access-controlled, and governed according to applicable privacy and data-protection rules. Unsupervised systems used in employment, credit, healthcare, policing, or essential services require heightened review because exploratory groupings can become de facto decisions.

A reproducible workflow records preprocessing, feature definitions, distance metrics, random seeds, software versions, hyperparameters, excluded observations, and model-selection rationale. Pipelines should prevent information leakage: for example, scaling parameters and dimensionality-reduction transformations should be fitted on the training portion when the representation will be used for future decisions.

7. Integrated Workflow for Exploratory Analytics

A defensible workflow is as follows:

Define the decision or inquiry. Specify what “useful structure” would change. Avoid clustering merely because an algorithm is available.

Audit the data. Examine missingness, duplicates, measurement error, sampling bias, time dependence, categorical variables, and possible leakage.

Construct features deliberately. Use domain transformations, robust treatment of skew, appropriate encoding, and documented scaling. Remove identifiers that cause spurious similarity unless they are substantively relevant.

Establish baselines. Compare simple summaries, PCA, k-means, and robust distance methods before adopting complex models.

Fit multiple plausible methods. Compare algorithms whose assumptions match alternative hypotheses about geometry or density.

Validate stability and usefulness. Use resampling, sensitivity analysis, external evidence, expert review, and downstream evaluation.

Communicate uncertainty. Show overlap, membership probabilities, outliers, and sensitivity rather than presenting a single definitive map.

Monitor deployment. Track feature drift, cluster-size changes, anomaly-alert rates, and changes in operational outcomes. Unsupervised models can become obsolete as behaviour and measurement processes change.

8. Illustrative Case: Industrial Sensor Segmentation and Anomaly Detection

Background

Consider a manufacturing firm operating 2,400 industrial pumps. Each pump generates temperature, vibration, pressure, flow, and electrical measurements at five-minute intervals. The firm has few confirmed failure labels, but maintenance costs are increasing.

Problem

The firm wants to identify operating regimes, prioritize inspections, and detect early signs of failure without assuming that all pumps behave according to one normal pattern.

Analysis

The analytics team first aggregates measurements into physically meaningful hourly features: means, standard deviations, trends, maximum vibration, and pressure-flow relationships. Features are standardized using robust location and scale estimates because several sensors contain spikes. Missing intervals are retained as quality indicators rather than silently imputed. Data are split chronologically to prevent future information from entering historical analysis.

PCA reveals that load and flow explain much of the variation, while vibration-related features form a second major direction. However, a two-dimensional t-SNE plot shows apparent islands that are not stable across random seeds; therefore it is used only for visualization.

The team compares k-means, a Gaussian mixture model, and HDBSCAN. k-means produces five compact operating groups but merges a low-load regime with a high-vibration regime. The mixture model provides uncertain membership for transitional operating states. HDBSCAN identifies three dense regimes and a small set of noise points, but its results vary when the minimum cluster size changes.

Approach and solution

The firm adopts a hybrid design. A Gaussian mixture model identifies operating-regime probabilities, while a regime-specific anomaly score combines robust Mahalanobis distance and a local-density score. Alerts are not issued from a single unusual measurement; instead, an alert requires repeated high scores across two consecutive hours and is reviewed against maintenance logs.

The team evaluates the system using a temporal holdout. It measures alert precision at the available investigation capacity, time before confirmed failures, false-alert rates by pump type, and stability under sensor recalibration. Engineers review representative observations near cluster boundaries and inspect whether the segments correspond to known load and mechanical conditions.

Outcome

The resulting segments support different maintenance baselines: vibration that is normal under one load regime is anomalous under another. The system improves inspection prioritization in the pilot, but it does not replace engineering diagnosis. Several apparent anomalies are traced to a sensor firmware change, demonstrating that anomaly detection can reveal data-quality problems as well as equipment problems.

Learning points

The case illustrates that unsupervised learning is a process of measurement design, modelling, validation, and operational governance. Multiple methods can be complementary. Temporal validation is essential, and anomaly thresholds must be linked to action capacity. The most useful result is not necessarily the model with the best internal score, but the one that produces stable, interpretable, and decision-relevant evidence.

9. Contemporary Developments and Research Directions

Self-supervised learning increasingly creates representations from unlabelled data through tasks such as masked reconstruction, contrastive learning, and predictive coding. These methods can outperform classical representations in images, language, and multimodal data, but they require careful control of augmentation, negative sampling, computational cost, and representation bias.

Deep generative models, including variational autoencoders and diffusion models, provide flexible latent representations and data-generation mechanisms. Their samples and likelihoods must be assessed for memorization, demographic imbalance, privacy leakage, and mismatch between statistical plausibility and real-world usefulness.

Graph representation learning extends unsupervised discovery to networks, where similarity depends on links as well as attributes. Dynamic and streaming methods address changing distributions, while federated and privacy-preserving methods seek to reduce centralized data exposure. These developments increase capability but also make interpretability, reproducibility, and governance more important.

Challenges and Limitations

The principal limitations of unsupervised learning are not merely computational. They include non-identifiability of latent structures, sensitivity to representation and distance, dependence on hyperparameters, instability under sampling, high-dimensional sparsity, and the temptation to overinterpret visualizations. Clusters may reflect data collection processes rather than meaningful populations. Anomalies may represent measurement errors, rare but valid cases, or discrimination in the underlying process. No algorithm can resolve an ambiguous research question without assumptions and domain evidence.

Best Practices

• Define the intended use before choosing an algorithm.
• Inspect and document data-generating and measurement processes.
• Scale and encode variables according to their meaning.
• Compare methods with different geometric assumptions.
• Use robust preprocessing when contamination is plausible.
• Assess stability, not only a single internal score.
• Separate exploratory visualization from confirmatory claims.
• Validate downstream consequences and subgroup performance.
• Use temporal or grouped validation when observations are dependent.
• Preserve uncertainty, provenance, and human review in operational systems.

Chapter Summary

Unsupervised learning discovers structure without supplied target labels. Clustering methods differ in their assumptions: k-means favours compact Euclidean groups, hierarchical methods expose multiscale relationships, density-based methods identify irregular dense regions and noise, and mixture models provide probabilistic memberships. PCA and SVD create linear low-dimensional representations, while manifold methods attempt to preserve nonlinear neighbourhood or geodesic structure. Matrix factorization exposes latent components, with non-negativity and probabilistic assumptions influencing interpretation. Density estimation models concentration, and anomaly detection identifies observations that depart from a defined reference distribution or neighbourhood structure.

Reliable practice requires careful feature construction, scaling, distance selection, validation, stability analysis, domain interpretation, and governance. No unsupervised output is self-validating. In professional settings, the strongest solution combines statistical evidence with operational knowledge, explicit uncertainty, protection against leakage, monitoring for drift, and review of ethical consequences.

Key Terms

Clustering: Grouping observations according to a selected similarity or probability model.

Centroid: The mean vector representing a cluster in k-means or related methods.

Silhouette coefficient: An internal measure comparing within-cluster cohesion with nearest-cluster separation.

Hierarchical clustering: A method producing nested groups represented by a dendrogram.

Linkage: The rule used to define distance between clusters.

Density-based clustering: Clustering based on regions of high observation density.

Gaussian mixture model: A probabilistic model representing a distribution as weighted Gaussian components.

Expectation-maximization: An iterative procedure alternating latent-membership estimation and parameter updating.

Dimensionality reduction: Mapping data into fewer dimensions while preserving selected information.

Principal component analysis: Linear projection onto orthogonal directions of maximum variance.

Singular value decomposition: Matrix decomposition into orthogonal factors and singular values.

Manifold learning: Nonlinear representation learning based on presumed low-dimensional geometry.

Matrix factorization: Approximation of a matrix using lower-dimensional latent factors.

Non-negative matrix factorization: Factorization constrained to non-negative component and loading values.

Kernel density estimation: Nonparametric density estimation using weighted kernels around observations.

Anomaly: An observation, context, or collection that deviates from a reference pattern.

Mahalanobis distance: Covariance-adjusted distance measuring multivariate departure from a centre.

Concept drift: Change over time in the distribution or data-generating process.

Representation: A transformed set of variables used to express structure in data.

Review Questions

Short Answer Questions

1. Why is feature scaling important in distance-based clustering?

2. State the principal assumptions of k-means clustering.

3. What does the silhouette coefficient measure?

4. Distinguish PCA from t-SNE in terms of purpose and interpretation.

5. Define point, contextual, and collective anomalies.

Long Answer Questions

1. Compare k-means, hierarchical clustering, DBSCAN, and Gaussian mixture models in terms of assumptions, strengths, and limitations.

2. Derive the PCA optimization objective and explain its relationship to eigenvectors of the covariance matrix.

3. Explain how matrix factorization is used in text analytics and recommender systems, including the role of sparsity and missingness.

4. Discuss why unsupervised-model validation is difficult and propose a comprehensive validation framework.

5. Explain the ethical and governance risks of using unsupervised segmentation in high-impact decisions.

Analytical / Application Questions

1. A dataset contains customer income, age, transaction counts, and a categorical region variable. Design a preprocessing and clustering strategy, justifying scaling, encoding, distance, algorithm choice, and validation.

2. An anomaly detector produces many alerts immediately after a sensor firmware update. Analyse possible explanations and design an investigation that distinguishes equipment failures from data-quality and concept-drift problems.

3. Two-dimensional UMAP shows three visually separated groups, but cluster assignments change substantially across parameter settings. How should an analyst communicate and investigate this result before claiming that three real populations exist?

Multiple Choice Questions with Answers

Question 1

Which objective is minimized by standard k-means clustering?

A. Within-cluster sum of squared Euclidean distances

B. Total number of observations

C. Between-cluster covariance only

D. Kernel density estimation error

Correct Answer: A. Within-cluster sum of squared Euclidean distances

Question 2

What does a silhouette value near zero generally indicate?

A. A point is certainly an anomaly

B. A point lies near overlapping or weakly separated clusters

C. The clustering has perfect separation

D. The data have zero variance

Correct Answer: B. A point lies near overlapping or weakly separated clusters

Question 3

In a Gaussian mixture model, what does a posterior membership probability represent?

A. The number of features in a cluster

B. The probability that an observation belongs to a component given the observation

C. The Euclidean distance to the global mean

D. The bandwidth of a kernel

Correct Answer: B. The probability that an observation belongs to a component given the observation

Question 4

What is the main purpose of standardizing a numerical feature before Euclidean-distance clustering?

A. To make all observations identical

B. To prevent variables with larger units from dominating distance

C. To guarantee spherical clusters

D. To remove all outliers

Correct Answer: B. To prevent variables with larger units from dominating distance

Question 5

PCA chooses its first component to maximize which quantity subject to a unit-length loading vector?

A. Projected variance

B. Number of clusters

C. Anomaly count

D. Categorical entropy only

Correct Answer: A. Projected variance

Question 6

Which method explicitly uses a neighbourhood radius and a minimum number of points to identify dense regions and noise?

A. PCA

B. DBSCAN

C. SVD

D. Ward’s linkage only

Correct Answer: B. DBSCAN

Question 7

In kernel density estimation, increasing the bandwidth generally has what effect?

A. Produces a smoother estimate with potentially less detail

B. Always increases the number of modes

C. Makes density independent of sample size

D. Converts the model into k-means

Correct Answer: A. Produces a smoother estimate with potentially less detail

Question 8

Why can t-SNE plots not usually be interpreted as preserving global distances?

A. They are constructed to preserve every pairwise distance exactly

B. The optimization emphasizes local neighbourhood relationships and can distort global geometry

C. They require labelled target variables

D. They cannot represent more than one dimension

Correct Answer: B. The optimization emphasizes local neighbourhood relationships and can distort global geometry

Question 9

What is a key benefit of probabilistic clustering over hard assignment?

A. It eliminates the need for feature preprocessing

B. It provides uncertainty through membership probabilities

C. It guarantees the correct number of clusters

D. It is unaffected by model assumptions

Correct Answer: B. It provides uncertainty through membership probabilities

Question 10

Which validation practice is especially important when unsupervised models will be used on future time periods?

A. Randomly mixing all historical and future observations

B. Chronological or time-aware validation

C. Removing all temporal variables without review

D. Evaluating only the training reconstruction error

Correct Answer: B. Chronological or time-aware validation