Linear and Affine Transformations

Introduction

Linear and affine transformations provide the mathematical language for changing representations of data. In artificial intelligence and machine learning, a feature vector may be scaled, rotated, projected, compressed, aligned with another coordinate system, or translated before it is used by an algorithm. These operations are not merely geometric conveniences: they influence distances, similarity, model parameters, numerical stability, and interpretability.

A linear transformation preserves vector addition and scalar multiplication. Once bases are selected, every linear transformation between finite-dimensional vector spaces can be represented by a matrix. This makes matrix multiplication the computational mechanism for applying transformations and composing them. An affine transformation extends a linear transformation by allowing translation. It has the form x ↦ Ax + b and is fundamental in computer vision, robotics, graphics, spatial data analysis, and feature engineering.

This chapter develops kernels and images, rank and nullity, composition and inverses, coordinate changes, affine maps, and practical applications to AI and ML.

Learning Objectives

By the end of this chapter, you should be able to:

1. Define a linear transformation and represent it with a matrix.

2. Determine the kernel, image, rank, and nullity of a transformation.

3. Analyze composition, invertibility, and inverse transformations.

4. Change the matrix representation of a transformation under a change of basis.

5. Distinguish linear maps from affine maps and analyze affine composition and inversion.

6. Apply transformations to feature vectors and geometric data while recognizing their limitations.

1. Linear Transformations

Let V and W be vector spaces over the same field, usually R. A function T: V → W is linear if, for all u, v in V and every scalar c,

T(u + v) = T(u) + T(v), T(cu) = cT(u).

Equivalently, T(au + bv) = aT(u) + bT(v) for all vectors u, v and scalars a, b. The first condition is preservation of addition; the second is preservation of scalar multiplication.

A linear transformation necessarily maps the zero vector to the zero vector. Indeed, T(0) = T(0 + 0) = T(0) + T(0), so T(0) = 0. Therefore, a transformation that moves the origin to a nonzero point is not linear, although it may be affine.

Examples include:

• Scaling: T(x) = 3x.
• Reflection across the x-axis in R²: T(x, y) = (x, −y).
• Projection onto the x-axis: T(x, y) = (x, 0).
• Differentiation on an appropriate vector space of functions.

The function T(x, y) = (x + 1, y) is not linear because T(0, 0) = (1, 0), not (0, 0).

Matrix Representation

Let T: R^n → R^m be linear. The standard basis of R^n is e₁, …, eₙ. Since every x ∈ R^n can be written as x = x₁e₁ + ··· + xₙeₙ,

T(x) = x₁T(e₁) + ··· + xₙT(eₙ).

Thus, if A is the m × n matrix whose columns are T(e₁), …, T(eₙ), then

T(x) = Ax.

This representation is unique relative to the selected bases. The columns of A are not arbitrary: they are the images of the domain basis vectors.

Worked Example 1: Constructing a Matrix

Suppose T: R² → R² is defined by

T(x, y) = (2x − y, x + 3y).

The images of the standard basis vectors are T(1, 0) = (2, 1) and T(0, 1) = (−1, 3). Therefore,

A = [[2, −1], [1, 3]],

and T(x, y) = A[x, y]ᵀ. For x = (4, 2)ᵀ,

T(x) = [ [2, −1], [1, 3] ][4, 2]ᵀ = [6, 10]ᵀ.

2. Kernel, Image, Rank, and Nullity

The kernel, or null space, of T is

ker(T) = {x ∈ V : T(x) = 0}.

For T(x) = Ax, this is the solution set of Ax = 0. The kernel identifies input directions that are completely lost by the transformation. If a nonzero feature variation belongs to the kernel, the transformed representation cannot distinguish that variation.

The image, or range, is

im(T) = {T(x) : x ∈ V}.

For a matrix A, the image is the column space of A. It consists of all outputs that the transformation can produce.

The rank of T is dim(im(T)); the nullity is dim(ker(T)). For T: R^n → R^m, the rank–nullity theorem states

rank(T) + nullity(T) = n.

The domain dimension, not the codomain dimension, appears on the right. The theorem says that every input dimension either contributes an independent output direction or belongs to a lost direction represented through the kernel.

Worked Example 2: Kernel and Image

Let

A = [[1, 2, 3], [2, 4, 6]].

The second row is twice the first, so rank(A) = 1. To find the kernel, solve

x₁ + 2x₂ + 3x₃ = 0.

Taking x₂ = s and x₃ = t gives x₁ = −2s − 3t. Hence

ker(A) = span{(−2, 1, 0), (−3, 0, 1)}.

The nullity is 2, and rank plus nullity is 1 + 2 = 3, equal to the number of columns. The image is the one-dimensional subspace

im(A) = span{(1, 2)}.

Although the input has three coordinates, all outputs lie on a line. This is a simple example of dimension reduction.

Geometric Interpretation

A linear map can stretch, shrink, rotate, reflect, shear, project, or collapse a space. A projection collapses some directions into the kernel. A full-rank square transformation preserves dimension, although it may substantially alter lengths and angles. Rank indicates the effective dimensionality of the output, while the kernel indicates information discarded by the map.

3. Composition of Transformations

If T: R^n → R^m has matrix A and S: R^m → R^p has matrix B, then the composition S ∘ T has matrix BA:

(S ∘ T)(x) = S(T(x)) = B(Ax) = (BA)x.

The order matters. In general, BA ≠ AB, and the dimensions must be compatible. The rightmost matrix acts first.

Composition is associative: (CD)A = C(DA), but matrix multiplication is not generally commutative. This distinction is important in pipelines. For example, a scaling followed by a rotation usually differs from a rotation followed by a scaling.

For compatible transformations,

ker(T) ⊆ ker(S ∘ T),

because any vector already mapped to zero by T remains zero after S. However, the composite may have an even larger kernel if S discards additional directions.

Worked Example 3: Transformation Pipeline

Let A scale the coordinates by different factors,

A = [[2, 0], [0, 1]],

and let B rotate vectors by 90 degrees counterclockwise,

B = [[0, −1], [1, 0]].

Applying A first and then B gives

BA = [[0, −1], [2, 0]].

Applying B first and then A gives

AB = [[0, −2], [1, 0]].

These matrices differ. For the vector (1, 0), BA produces (0, 2), whereas AB produces (0, 1). The order of preprocessing operations therefore changes the representation.

4. Invertibility and Inverse Transformations

A linear transformation T: R^n → R^n is invertible if there exists T⁻¹ such that

T⁻¹(T(x)) = x and T(T⁻¹(y)) = y.

For a square matrix A, the following conditions are equivalent:

• A is invertible.
• det(A) ≠ 0.
• rank(A) = n.
• ker(A) = {0}.
• The columns of A are linearly independent.
• Ax = b has exactly one solution for every b ∈ R^n.

When A is invertible, the inverse transformation is x = A⁻¹y. For a 2 × 2 matrix

A = [[a, b], [c, d]],

if ad − bc ≠ 0, then

A⁻¹ = 1/(ad − bc) [[d, −b], [−c, a]].

The determinant condition is specific to square matrices. Rectangular matrices do not have ordinary two-sided inverses, although they may have left or right inverses under suitable rank conditions.

In numerical computing, explicitly forming an inverse is often less stable and less efficient than solving Ax = b directly using factorization. The mathematical inverse is useful conceptually, but software typically uses methods such as LU, QR, or singular value decomposition.

Conditioning and Near-Singularity

An invertible transformation may still be numerically unreliable if it is close to singular. Small input errors can then produce large output errors after inversion. The condition number measures sensitivity; for a chosen matrix norm, it is commonly defined as

κ(A) = ||A|| ||A⁻¹||.

This definition applies only when A is invertible. A large condition number indicates that inversion or parameter estimation may be sensitive to measurement and rounding errors.

5. Coordinate Systems and Change of Basis

A vector is an abstract geometric or algebraic object; its coordinate column depends on the basis. Let B = (b₁, …, bₙ) and C = (c₁, …, cₙ) be two bases of the same vector space. If P is the matrix whose columns are the B-coordinate vectors of the C-basis vectors, then

[x]_B = P[x]_C.

Consequently,

[x]_C = P⁻¹[x]_B,

provided P is invertible, as it must be when B and C are bases.

Suppose a linear transformation T has matrix A_B in basis B and matrix A_C in basis C. Then

A_C = P⁻¹ A_B P.

This is a similarity transformation. It does not change intrinsic properties such as determinant, trace, eigenvalues, rank, or characteristic polynomial, although the numerical entries of the matrix change.

The expression should be interpreted carefully: P converts C-coordinates into B-coordinates; A_B applies T in B-coordinates; P⁻¹ converts the result back into C-coordinates.

Example 4: A Simple Basis Change

Let T be represented in the standard basis by

A = [[2, 0], [0, 3]].

Let the new basis be c₁ = (1, 1) and c₂ = (1, −1). Then

P = [[1, 1], [1, −1]],

and P⁻¹ = (1/2)[[1, 1], [1, −1]]. The matrix in the new basis is A_C = P⁻¹AP. Computing gives

A_C = [[5/2, −1/2], [−1/2, 5/2]].

The entries changed, but the transformation itself did not. Coordinate changes are essential when a basis is selected to expose meaningful directions, such as principal components or physically meaningful axes.

6. Affine Transformations

An affine transformation has the form

f(x) = Ax + b,

where A is a linear transformation and b is a fixed translation vector. Unless b = 0, f is not linear because f(0) = b. Nevertheless, affine maps preserve affine combinations: if coefficients αᵢ satisfy Σαᵢ = 1, then

f(Σ αᵢxᵢ) = Σ αᵢf(xᵢ).

Therefore, affine maps preserve lines, parallelism, ratios along a line, and convex combinations. They do not necessarily preserve distances, angles, or lengths.

An affine map is invertible precisely when A is square and invertible. In that case,

f⁻¹(y) = A⁻¹(y − b).

The translation must be removed before applying A⁻¹.

Homogeneous Coordinates

Affine transformations can be represented as linear transformations in one higher dimension. Write

x̃ = [x; 1],

and define

H = [[A, b], [0ᵀ, 1]].

Then Hx̃ = [Ax + b; 1]. This representation makes affine transformations easy to compose using matrix multiplication. It is widely used in computer graphics, robotics, and geometric computer vision.

If f(x) = A₁x + b₁ and g(x) = A₂x + b₂, then

g(f(x)) = A₂A₁x + (A₂b₁ + b₂).

Thus the composite affine map is represented by the pair (A₂A₁, A₂b₁ + b₂).

7. Applications in AI and Machine Learning

Feature Transformations

A feature vector x may be transformed as z = Ax + b before model training. Examples include:

• rescaling measurements with different units;
• rotating or projecting data into a more useful coordinate system;
• reducing dimensionality using a projection matrix;
• centering data by subtracting a mean vector;
• aligning measurements collected from different sensors.

Centering uses z = x − μ, which is affine as a function of x. Standardization usually applies a diagonal scaling after centering, z = D(x − μ), where D contains reciprocal scale estimates. The estimates must be computed from the training data only when evaluating a predictive model, otherwise information from the test set can leak into training.

Principal Component Analysis

PCA identifies orthogonal directions of high variance. After centering data, a matrix W whose columns are selected principal directions produces reduced coordinates z = Wᵀ(x − μ). This is an affine map from the original feature space to a lower-dimensional coordinate space. Because W has fewer columns than rows in a reduction setting, the map is generally not invertible; information orthogonal to the selected subspace is discarded.

Neural Network Layers

A dense neural-network layer computes z = Wx + b, followed commonly by a nonlinear activation. The affine part changes coordinates and mixes features; the nonlinearity is what allows a multilayer network to represent functions beyond a single affine map. Bias terms are translations and should not be confused with linear operations.

Geometric Data

In computer vision and robotics, affine maps model image shearing, scaling, rotation, and translation under appropriate approximations. In point-cloud processing, transformations align coordinates from different sensors. In these applications, the transformation convention—row vectors versus column vectors, units, axis orientation, and multiplication order—must be documented explicitly.

8. Illustrative Case: Aligning Two Sensor Feature Spaces

Background

Consider an illustrative autonomous-vehicle system using two sensors. Sensor A records two-dimensional object-location features in metres. Sensor B records the same objects in a coordinate frame that is rotated, scaled slightly, and translated relative to Sensor A.

Problem

A classifier trained using Sensor A coordinates performs poorly on unaligned Sensor B data. The system needs a transformation that maps B measurements into the coordinate convention used by A.

Analysis

Suppose calibration produces the affine map

x_A = Ax_B + b,

where

A = [[1.02, −0.10], [0.10, 1.02]], b = [3, −2]ᵀ.

The matrix represents approximate scaling and rotation; b represents displacement between sensor origins. For a Sensor B observation x_B = [10, 5]ᵀ,

Ax_B = [1.02(10) − 0.10(5), 0.10(10) + 1.02(5)]ᵀ

= [9.7, 6.1]ᵀ.

Therefore,

x_A = [12.7, 4.1]ᵀ.

Approach and Outcome

The transformed coordinates can be passed to the classifier after applying the same calibration learned from training data. If A is invertible, points can also be mapped back using A⁻¹(x_A − b). Validation should compare alignment error on held-out calibration points and evaluate downstream classification performance, not merely inspect a few transformed coordinates.

Learning Points

The case illustrates that translation requires an affine rather than linear model, matrix order matters, units must be consistent, and calibration uncertainty affects downstream predictions. A transformation that is geometrically plausible may still be unsuitable if it amplifies noise or was estimated using leaked evaluation data.

9. Limitations, Governance, and Best Practices

Transformations can discard information, distort distances, and obscure the meaning of original variables. A projection may improve computational efficiency while harming rare-event detection. Scaling can cause a feature with measurement error to receive disproportionate influence. Coordinate alignment can also encode assumptions about privacy-sensitive location data.

Best practices include:

1. Record the transformation matrix, translation vector, basis convention, units, and data version.

2. Fit learned transformations such as centering, scaling, and PCA on training data only.

3. Check rank, conditioning, and reconstruction error when information preservation matters.

4. Use dimensionally meaningful features and avoid arbitrary transformations that reduce interpretability.

5. Validate transformations on representative and subgroup-specific data.

6. Prefer stable linear solves to explicit matrix inversion in numerical software.

7. Protect transformed data when the original features contain personal, health, or location information.

8. Monitor drift: a calibration or feature map estimated in one environment may become invalid after sensor replacement or distribution change.

Contemporary systems increasingly combine affine preprocessing with learned nonlinear representations. Differentiable geometric layers, spatial transformer modules, and equivariant architectures seek to make models respond predictably to transformations such as translation or rotation. These methods do not eliminate the need to specify coordinate conventions and assess numerical stability.

Chapter Summary

A linear transformation preserves vector addition and scalar multiplication and is represented, relative to chosen bases, by matrix multiplication. Its kernel contains input directions mapped to zero, while its image contains attainable outputs. Rank measures image dimension, nullity measures kernel dimension, and rank plus nullity equals the domain dimension. Composition corresponds to matrix multiplication, with the rightmost operation applied first. A square transformation is invertible exactly when it has full rank and a trivial kernel, although numerical conditioning determines practical reliability. Changing coordinates changes the matrix representation according to A_new = P⁻¹A_oldP. Affine maps have the form Ax + b and extend linear maps by translation; they are central to feature preprocessing, neural-network layers, calibration, computer vision, and robotics. Sound applications require attention to information loss, units, conditioning, data leakage, interpretability, and governance.

Key Terms

Linear transformation: A map preserving vector addition and scalar multiplication.

Matrix representation: The matrix whose columns are the images of domain basis vectors.

Kernel: The set of vectors mapped to the zero vector.

Image: The set of all attainable outputs of a transformation.

Rank: The dimension of the image or column space.

Nullity: The dimension of the kernel.

Rank–nullity theorem: The domain dimension equals rank plus nullity.

Composition: Applying one transformation after another.

Invertible map: A map possessing a two-sided inverse.

Determinant: A scalar whose nonzero value characterizes invertibility for square matrices.

Condition number: A measure of sensitivity of a matrix problem to perturbations.

Basis: A linearly independent spanning set.

Change-of-basis matrix: A matrix converting coordinates between bases.

Similarity transformation: A representation change of the form P⁻¹AP.

Affine transformation: A map of the form Ax + b.

Translation: Addition of a fixed vector.

Homogeneous coordinates: An augmented representation allowing affine maps to use matrix multiplication.

Feature transformation: A map applied to input variables before modeling.

Projection: A transformation mapping data onto a subspace, often reducing dimension.

Data leakage: Use of information from evaluation data during model preparation.

Review Questions

Short Answer Questions

1. What two properties define a linear transformation?

2. What does the kernel of a matrix transformation represent?

3. State the rank–nullity theorem for a map from Rⁿ.

4. Why is Ax + b generally affine rather than linear?

5. How does the matrix of a transformation change under a basis change?

Long Answer Questions

1. Explain how a matrix is constructed from the images of basis vectors and illustrate the process with a map from R² to R².

2. Discuss the relationship among kernel, image, rank, and nullity, including their relevance to information loss.

3. Explain composition and invertibility of linear transformations, including the role of matrix order and conditioning.

4. Derive the inverse of an invertible affine map f(x) = Ax + b.

5. Compare linear preprocessing, affine preprocessing, and nonlinear processing in a neural-network pipeline.

Analytical / Application Questions

1. For A = [[1, 2], [2, 4]], determine the rank, kernel, and whether the transformation is invertible. Interpret the result geometrically.

2. A data pipeline first centers x by μ and then applies a matrix W. Express the complete operation as an affine map and explain what information may be lost if W reduces dimension.

3. Two coordinate transformations are proposed for sensor alignment. Design a validation procedure that checks matrix convention, units, calibration error, conditioning, and downstream model performance.

Multiple Choice Questions with Answers

Question 1

Which condition is required for a transformation T to be linear?

A. T(0) must be a nonzero vector

B. T(u+v)=T(u)+T(v) and T(cu)=cT(u)

C. T must preserve lengths for all vectors

D. T must have a square matrix

Correct Answer: B. T(u+v)=T(u)+T(v) and T(cu)=cT(u)

Question 2

For a matrix A representing T: R^n → R^m, what is the image of T?

A. The null space of A

B. The row space of A only

C. The column space of A

D. The set of all vectors in R^n

Correct Answer: C. The column space of A

Question 3

If a transformation from R^5 has rank 3, what is its nullity?

A. 2

B. 3

C. 5

D. 8

Correct Answer: A. 2

Question 4

If T has matrix A and S has matrix B, which matrix represents S composed with T?

A. AB

B. A+B

C. BA

D. B−A

Correct Answer: C. BA

Question 5

Which condition is equivalent to invertibility of a square matrix A?

A. det(A)=0

B. ker(A) contains a nonzero vector

C. rank(A) is less than its dimension

D. ker(A)={0}

Correct Answer: D. ker(A)={0}

Question 6

If coordinates satisfy [x]_B = P[x]_C, what is the matrix of T in basis C when its matrix in basis B is A_B?

A. P A_B P⁻¹

B. P⁻¹ A_B P

C. A_B P

D. P + A_B

Correct Answer: B. P⁻¹ A_B P

Question 7

Which expression generally defines an affine transformation?

A. f(x)=Ax+b

B. f(x)=A/x

C. f(x)=xᵀx only

D. f(x)=det(A)

Correct Answer: A. f(x)=Ax+b

Question 8

Let f(x)=Ax+b, with A invertible. What is f⁻¹(y)?

A. A⁻¹y+b

B. A(y−b)

C. A⁻¹(y−b)

D. Ay−A⁻¹b

Correct Answer: C. A⁻¹(y−b)

Question 9

A projection from R^3 onto a two-dimensional subspace generally has what property?

A. It is always invertible

B. It has a nontrivial kernel

C. Its rank is 3

D. It preserves every distance

Correct Answer: B. It has a nontrivial kernel

Question 10

Why should learned feature transformations usually be fitted using training data only?

A. To guarantee the matrix is square

B. To prevent data leakage from evaluation data

C. To make every transformation nonlinear

D. To force the determinant to equal one

Correct Answer: B. To prevent data leakage from evaluation data