Unsupervised Learning and Pattern Discovery

Introduction

Supervised learning uses labelled examples to predict a known outcome. Unsupervised learning begins with data for which the desired outcome is not provided. Its purpose is to discover structure: groups of similar observations, compact representations, unusual cases, frequent combinations, or meaningful relationships among variables.

This distinction is important in practical data science. A retailer may know each customer’s transaction history but not their segment. A hospital may have measurements from patients without predefined disease categories. A manufacturing system may record sensor readings without labels for every fault. In such settings, unsupervised methods can support exploration, segmentation, anomaly detection and feature engineering.

Unsupervised learning does not automatically reveal objective or naturally occurring categories. Algorithms impose definitions of similarity and structure through their choice of variables, distance measure and assumptions. Therefore, the data scientist must combine algorithmic output with domain knowledge, validation and ethical judgement.

Learning objectives

By the end of this chapter, you should be able to:

• distinguish supervised and unsupervised learning;
• prepare data for clustering and dimensionality reduction;
• explain and apply k-means clustering and hierarchical clustering;
• interpret principal component analysis (PCA) at an introductory level;
• select suitable evaluation and visualisation methods;
• recognise assumptions, limitations, ethical risks and practical uses;
• communicate discovered patterns without treating them as unquestionable facts.

1. The nature of pattern discovery

An observation is a row in a dataset and a feature is a measured attribute or variable. Let a dataset contain n observations and p features. It can be represented as a matrix X with n rows and p columns. In unsupervised learning, there is generally no target variable y supplied for prediction.

Pattern discovery may involve:

• Clustering: assigning observations to groups according to similarity.
• Dimensionality reduction: representing many variables using fewer constructed variables.
• Association discovery: identifying items or events that occur together.
• Anomaly detection: identifying observations that differ substantially from the majority.

A pattern is useful only when it is stable, interpretable and relevant to a decision. A visually attractive cluster can be an artefact of scale, missing values, sampling bias or the algorithm itself.

Similarity and distance

Many unsupervised methods require a measure of similarity. For two observations x and z with p numerical features, Euclidean distance is:

Here, xᵢ and zᵢ are the values of feature i. The distance has the same unit as the features only when the feature units are compatible; with mixed units, standardisation is usually required. Euclidean distance is most appropriate when differences are meaningfully measured on numerical, continuous scales and when straight-line separation is a reasonable model of similarity.

Other choices include Manhattan distance, cosine distance and distances designed for categorical variables. The distance measure is not a minor implementation detail: it defines what “similar” means.

2. Preparing data for unsupervised learning

Selecting variables

Variables should be selected because they represent the question being investigated, not simply because they are available. An identification number, for example, may be numerical but has no meaningful geometric interpretation. Including it can create artificial separation.

Highly duplicated variables can give one underlying concept excessive influence. Conversely, removing an important variable can hide a meaningful pattern. Subject-matter consultation is therefore valuable.

Missing values and outliers

Most clustering algorithms do not handle missing values automatically. Options include justified imputation, removal of unsuitable records, or use of methods that explicitly model missingness. The chosen method should be documented because imputation can affect cluster membership.

Outliers may represent data errors, rare but important cases, or a genuine small segment. Removing them without investigation can conceal precisely the patterns an organisation needs to understand.

Scaling

Suppose annual income is measured in thousands while the number of purchases is measured in units. Without scaling, income may dominate Euclidean distance simply because its numerical magnitude is larger. A common standardisation is the z-score:

where xᵢ is an observation’s value for feature i, μᵢ is the sample mean of feature i, and σᵢ is its sample standard deviation. The result is dimensionless, with sample mean approximately 0 and sample standard deviation approximately 1, provided σᵢ > 0. Standardisation is appropriate when relative deviations from each feature’s mean should be compared. It is not automatically appropriate when the original scale has substantive meaning or when extreme values make the mean and standard deviation unrepresentative.

Min–max scaling maps values to a selected interval, commonly [0, 1]:

It requires a nonconstant feature and is sensitive to extreme observations. Scaling parameters should be calculated from the relevant training or reference data and then applied consistently to later data.

Categorical data

Categorical variables should not ordinarily be encoded as arbitrary integers, because coding red, blue and green as 1, 2 and 3 falsely implies an order and equal spacing. One-hot encoding may be suitable for some distance-based workflows, although it can create high-dimensional sparse data. Alternative measures, such as Gower distance, can combine numerical, ordinal and categorical variables, but their interpretation must be understood.

3. k-means clustering

Principle

k-means partitions observations into k groups. Each group is represented by a centroid, which is the mean vector of the observations assigned to that group. The algorithm attempts to minimise within-cluster squared Euclidean distance:

Here, k is the number of clusters, Cⱼ is cluster j, x is an observation in that cluster, μⱼ is its centroid, and ||x − μⱼ||² is squared Euclidean distance. The objective is applicable to numerical features for which a mean centroid and squared-distance loss are meaningful.

Algorithm

A standard k-means procedure is:

1. Choose k initial centroids.
2. Assign each observation to the nearest centroid.
3. Recalculate each centroid as the mean of its assigned observations.
4. Repeat assignment and recalculation until assignments stop changing, centroid movement is small, or a maximum iteration limit is reached.

The method usually converges to a local optimum, not necessarily the global optimum. Different initial centroids can produce different results, so multiple initialisations are good practice. Empty clusters may also arise and require reinitialisation.

Worked example

Consider one-dimensional, already standardised observations: 1, 2, 3, 10 and 11, with k = 2. Suppose initial centroids are 2 and 10.

The first assignment gives C₁ = {1, 2, 3} and C₂ = {10, 11}. The updated centroids are:

The assignments do not change on the next iteration, so the algorithm has stabilised. The final clusters are the lower-valued observations and the higher-valued observations. The within-cluster objective is:

This result describes separation under the selected variable and distance measure; it does not prove that two real-world “types” exist.

Choosing k

There is no universally correct value of k. Two introductory tools are:

• Elbow method: calculate the within-cluster sum of squares for several values of k. Increasing k always cannot increase the optimised training objective, so the analyst looks for a point where additional clusters produce diminishing improvement.
• Silhouette coefficient: for observation i, let a(i) be its average distance to observations in its own cluster and b(i) the smallest average distance to another cluster. Then:

For observations in clusters containing more than one case, s(i) lies between −1 and 1. Larger values indicate that the observation is closer to its own cluster than to the nearest alternative. The coefficient is a diagnostic, not proof of meaningful segmentation.

The selected k should also be useful to stakeholders. A mathematically defensible solution with twelve indistinguishable segments may be less actionable than a stable solution with four interpretable segments.

Strengths and limitations of k-means

K-means is relatively fast, simple and easy to communicate. It works well when groups are compact, approximately spherical and reasonably similar in size. It is less suitable for elongated, nested or irregularly shaped groups, unequal density, substantial outliers, or categorical variables without an appropriate representation. Results are affected by scaling, k, initialisation and the selected features.

4. Hierarchical clustering

Hierarchical clustering creates a nested structure of groups displayed as a dendrogram. In agglomerative clustering, every observation begins as its own cluster. The algorithm repeatedly merges the two closest clusters until one cluster remains.

The definition of distance between clusters is called linkage:

A dendrogram can be cut at a selected height to obtain a chosen number of clusters. Unlike k-means, hierarchical clustering can reveal relationships at several levels and does not require k at the initial algorithmic stage. However, early merges cannot normally be undone, and results depend strongly on the distance and linkage choices. Large datasets may also make hierarchical methods computationally demanding.

5. Dimensionality reduction and PCA

A dataset with many correlated features can be difficult to visualise, model and interpret. Dimensionality reduction constructs a smaller representation while attempting to preserve selected information.

Principal component analysis

PCA transforms centred numerical variables into orthogonal directions called principal components. The first component captures the greatest possible variance among all unit-length directions; each later component captures the greatest remaining variance subject to being orthogonal to earlier components.

Let X be a centred data matrix. PCA can be obtained from the covariance matrix S. The first principal direction w₁ solves:

The solution is the eigenvector associated with the largest eigenvalue of S. The corresponding component score for observation x is:

The eigenvalue gives the variance along that component. If eigenvalues are λ₁, λ₂, …, λₚ, the proportion of variance explained by the first q components is:

This is a proportion, so it has no units. It is meaningful when PCA is applied to the covariance or correlation structure of appropriately prepared numerical data. If variables are standardised, PCA effectively uses the correlation matrix; otherwise, variables with larger variance can dominate.

Interpreting PCA

A loading is the contribution or weight of an original variable to a component. A component with large positive loadings for spending and purchase frequency might be interpreted cautiously as an “activity” dimension. Opposing signs indicate direction, but the overall sign of a component is arbitrary: multiplying all scores and loadings by −1 produces the same solution.

PCA is not a clustering algorithm. A two-dimensional PCA plot can help visualise possible groups, but apparent separation may disappear in the full feature space, and a plot does not establish causal relationships. PCA maximises variance, not business importance, predictive accuracy or fairness.

When to use dimensionality reduction

PCA can reduce noise, compress data, support visual exploration and mitigate redundancy among correlated features. It can also harm interpretability because each component combines several original variables. For nonlinear structures, methods such as t-SNE or UMAP may produce useful visualisations, but their layouts can depend on hyperparameters and random state and should not automatically be interpreted as global distances or confirmed clusters.

6. Integrated applied case: retail customer segmentation

This is an illustrative case. A regional retailer wants to design differentiated communications but has no existing customer labels. It has twelve months of transaction data for 24,000 customers.

Background and problem

The raw data contain customer identifiers, transaction dates, monetary value, product categories, returns and discount use. Management wants segments that are interpretable, reasonably stable and suitable for campaign planning. The data team must avoid using sensitive personal attributes as segmentation variables and must not assume that a segment is a permanent identity.

Analysis and preparation

The team constructs customer-level features: purchase frequency, average basket value, recency in days, proportion of discounted purchases and return rate. The identifier is excluded. Customers with insufficient history are flagged for a separate analysis rather than silently treated as low-value customers. Missing return information is investigated with the operations team.

Because the variables use different units and have skewed distributions, the team examines histograms, considers justified transformations, and standardises the final numerical features. Several initialisations of k-means are run for k from 2 through 7. Silhouette values, cluster sizes, stability across resamples and managerial interpretability are considered together.

Approach and solution

A four-cluster solution is selected provisionally. The clusters are described using medians and distributions rather than only centroids, because means can be distorted by outliers. An illustrative interpretation is:

These labels are summaries, not inherent truths. The team tests whether the segmentation is stable over time and whether campaigns based on it improve outcomes compared with a baseline strategy. It also checks whether error rates and promotional access differ unfairly across customer groups.

Outcome and learning points

The segmentation helps the retailer tailor experiments, but it does not by itself demonstrate that a particular message causes higher spending. A controlled campaign evaluation is needed. The project also reveals that customer segments change as behaviour changes; periodic review is preferable to treating a cluster assignment as permanent.

The central learning point is that unsupervised learning is a decision-support process: data definition, preparation, algorithm choice, validation and communication are equally important.

7. Evaluating and communicating discovered patterns

Unsupervised evaluation has no single universal accuracy score because there is usually no ground-truth label. Useful evidence includes:

• Internal validation: compactness and separation measures such as silhouette score.
• Stability: whether similar patterns appear under resampling, alternative initialisations or nearby time periods.
• External validation: comparison with known outcomes not used to create the clusters, such as response to a later campaign.
• Interpretability: whether a domain expert can explain the groups using meaningful variables.
• Actionability: whether the result supports a feasible and beneficial decision.

Reports should state the data period, preprocessing, excluded variables, scaling method, algorithm, distance, k or linkage, random seed where relevant, validation results and limitations. A heat map of standardised cluster profiles, a dendrogram or a PCA scatter plot can support communication, but visualisations should include scales and should not imply more certainty than the analysis provides.

8. Risks, ethics and governance

Unsupervised models can reproduce historical inequalities even when protected attributes are excluded. Proxy variables—such as postcode, purchasing patterns or device type—may correlate with protected characteristics. Segmentation can then lead to unequal pricing, service quality or access to opportunities.

Important safeguards include:

• define a legitimate purpose before modelling;
• minimise data collection and protect personal information;
• assess whether variables are sensitive or act as proxies;
• document preprocessing and model decisions;
• restrict access to individual-level assignments;
• provide human review for consequential decisions;
• monitor outcomes across relevant groups;
• allow correction where data are inaccurate;
• reassess clusters as populations and behaviour change.

Privacy law and organisational policy may impose requirements concerning lawful processing, transparency, retention, automated decision-making and individual rights. A technically valid cluster is not automatically a permissible basis for action.

9. Best-practice workflow

A disciplined workflow is:

1. Define the decision or exploratory question.
2. Audit provenance, quality, missingness and representativeness.
3. Select meaningful variables and remove identifiers that have no analytic purpose.
4. Investigate outliers and choose justified treatment.
5. Encode categorical variables appropriately and scale where required.
6. Run more than one reasonable method or configuration.
7. Evaluate internal quality, stability, interpretability and business relevance.
8. Profile clusters using distributions, not only averages.
9. Validate actions with experiments or later outcomes.
10. Document assumptions, limitations, ethical review and monitoring plans.

Contemporary developments

Modern practice increasingly combines classical clustering with scalable computation, representation learning and interactive visual analytics. Autoencoders can learn lower-dimensional representations using neural networks, while density-based methods such as DBSCAN can identify irregular groups and noise. These methods can be valuable, but greater complexity does not guarantee better discovery. For diploma-level practice, transparent baselines such as standardised k-means, hierarchical clustering and PCA remain essential because their assumptions and outputs can be explained and audited.

Chapter Summary

Unsupervised learning discovers structure without a supplied target variable. Clustering groups observations according to a chosen definition of similarity, while dimensionality reduction creates a smaller representation of high-dimensional data. Data preparation—especially feature selection, missing-value treatment, outlier investigation and scaling—strongly affects results.

K-means minimises within-cluster squared Euclidean distance and works best for compact numerical groups, but requires a selected k and is sensitive to initialisation and scale. Hierarchical clustering provides a dendrogram and can expose nested structure, although its results depend on linkage and distance choices. PCA creates orthogonal directions ordered by explained variance; it supports compression and visualisation but is not itself clustering and may reduce interpretability.

Because unsupervised learning lacks a universal accuracy measure, analysts should combine internal metrics, stability, external evidence, domain interpretation and actionability. Ethical practice requires attention to privacy, proxy discrimination, transparency, human oversight and changing populations. The professional goal is not merely to find groups, but to discover patterns that are valid, responsible and useful.

Key Terms

Unsupervised learning: Learning from data without a supplied target label.

Pattern discovery: Identification of structure, regularity or unusual behaviour in data.

Clustering: Assigning observations to groups based on similarity.

Feature: A measured variable used to represent an observation.

Distance measure: A rule for quantifying separation between observations.

Euclidean distance: Straight-line distance based on squared numerical differences.

Standardisation: Rescaling a variable using its mean and standard deviation.

Centroid: The mean vector representing a cluster in k-means.

k-means: A clustering algorithm that minimises within-cluster squared distance.

Linkage: The rule used to define distance between clusters in hierarchical clustering.

Dendrogram: A tree-like display of hierarchical merges.

Silhouette coefficient: A measure comparing an observation’s cohesion with its separation from another cluster.

Dimensionality reduction: Representation of data using fewer variables or components.

PCA: A linear dimensionality-reduction method based on directions of maximum variance.

Principal component: An orthogonal direction constructed from original variables.

Loading: The weight of an original variable in a principal component.

Explained variance: The proportion of total variance represented by selected components.

Anomaly: An observation substantially different from the prevailing pattern.

Stability: Persistence of a discovered pattern under reasonable data or modelling changes.

Actionability: The extent to which an analytical result supports a feasible decision.

Review Questions

Short Answer Questions

1. What distinguishes unsupervised learning from supervised learning?
2. Why can an identification number be harmful in a distance-based clustering analysis?
3. State the purpose of standardising numerical features.
4. What does a dendrogram represent?
5. Why is PCA not the same as clustering?

Long Answer Questions

1. Explain the k-means algorithm, including its objective function, update steps and main assumptions.
2. Compare single, complete, average and Ward’s linkage in hierarchical clustering.
3. Explain how PCA constructs components and how explained variance is interpreted.
4. Discuss how missing values, outliers and feature scaling can alter cluster results.
5. Describe a complete workflow for producing an ethically responsible customer segmentation.

Analytical / Application Questions

1. A dataset contains income in dollars, transaction count and a customer ID. Explain which variables should be transformed, excluded or retained before k-means, and justify your choices.
2. A two-cluster solution has a higher silhouette score than a four-cluster solution, but the four groups are much more stable over time and more useful to managers. How should the analyst make and communicate the choice?
3. A PCA plot appears to show two customer groups, but the groups disappear when original variables are examined. Give at least three possible explanations and propose checks before claiming that two segments exist.

Multiple Choice Questions with Answers

Question 1

Which statement best describes unsupervised learning?

A. It learns only from labelled examples.

B. It discovers structure without a supplied target label.

C. It guarantees causal explanations for observed patterns.

D. It always predicts a numerical outcome.

Correct Answer: B. It discovers structure without a supplied target label.

Question 2

Why is standardisation often used before Euclidean-distance clustering?

A. To remove all missing values automatically.

B. To ensure every feature becomes categorical.

C. To prevent variables with larger numerical scales from dominating distance.

D. To guarantee the correct number of clusters.

Correct Answer: C. To prevent variables with larger numerical scales from dominating distance.

Question 3

In k-means, what does a centroid represent?

A. The median of every feature in the entire dataset.

B. The mean vector of observations assigned to a cluster.

C. The observation with the largest distance from all others.

D. The first principal component.

Correct Answer: B. The mean vector of observations assigned to a cluster.

Question 4

What is the main purpose of the elbow method?

A. To select a potentially useful number of clusters by examining diminishing improvement.

B. To calculate missing values.

C. To identify the most sensitive demographic group.

D. To convert categorical variables into labels.

Correct Answer: A. To select a potentially useful number of clusters by examining diminishing improvement.

Question 5

A silhouette coefficient close to a high positive value generally indicates that an observation is:

A. Closer to another cluster than to its own cluster.

B. Equally distant from all clusters.

C. Well matched to its own cluster and separated from alternatives.

D. Guaranteed to be an outlier.

Correct Answer: C. Well matched to its own cluster and separated from alternatives.

Question 6

Which statement about PCA is correct?

A. It always creates business-relevant categories.

B. It constructs directions ordered by captured variance.

C. It requires a target variable for training.

D. It guarantees that causal relationships are preserved.

Correct Answer: B. It constructs directions ordered by captured variance.

Question 7

For observations 1, 2 and 3 assigned to a one-dimensional cluster, what is the centroid?

A. 1

B. 2

C. 3

D. 6

Correct Answer: B. 2

Question 8

Which linkage method can be especially susceptible to chaining?

A. Single linkage

B. Complete linkage

C. Average linkage

D. Ward’s method

Correct Answer: A. Single linkage

Question 9

Which is an important ethical risk in customer segmentation?

A. Clusters can act as proxies for protected characteristics.

B. Scaling always removes privacy risks.

C. A higher silhouette score proves fairness.

D. Unsupervised models cannot use personal data.

Correct Answer: A. Clusters can act as proxies for protected characteristics.

Question 10

Why should cluster stability be examined?

A. To determine whether patterns persist under reasonable changes in data or modelling.

B. To guarantee that every cluster has the same size.

C. To replace all domain knowledge.

D. To make PCA supervised.

Correct Answer: A. To determine whether patterns persist under reasonable changes in data or modelling.