Subject Overview

A rigorous, application-oriented study of vectors, matrices, linear systems, vector spaces, transformations, decompositions, numerical methods, tensors and matrix calculus, with direct connections to machine learning, optimization, data analytics and scalable AI systems.

Learning Goal

Students will be able to reason mathematically about linear algebra, select and implement appropriate computational methods, analyze numerical and ethical risks, and apply linear algebra to build, evaluate and govern modern AI and machine-learning solutions.

Syllabus

1. Mathematical Language, Scalars, Vectors and Matrices

Learning Objective: Master sets, fields, notation, indices, summations, real and complex scalars, vector and matrix operations, transpose, conjugate transpose, norms, distances and basic proof techniques used in AI.

Category: foundation

2. Systems of Linear Equations

Learning Objective: Formulate and solve linear systems using row reduction and Gaussian elimination; analyze consistency, rank, nullity, underdetermined systems, computational cost and numerical stability.

Category: foundation

3. Vector Spaces, Subspaces and Bases

Learning Objective: Understand span, linear independence, basis, dimension, coordinates, row space, column space, null space, orthogonal complements and the rank–nullity theorem.

Category: core

4. Linear and Affine Transformations

Learning Objective: Represent transformations as matrices; analyze kernels, images, composition, inverses, coordinate changes and affine maps, with applications to feature transformations and geometric data.

Category: core

5. Inner-Product Spaces, Orthogonality and Least Squares

Learning Objective: Apply inner products, angles, orthogonality, Gram–Schmidt orthogonalization, projections and QR methods to approximation, regression, overdetermined systems and model fitting.

Category: core

6. Determinants, Rank and Invertibility

Learning Objective: Interpret determinants algebraically and geometrically; use determinant properties, minors, cofactors, rank criteria and equivalent conditions for invertibility while recognizing computational limitations.

Category: core

7. Eigenvalues, Eigenvectors and Spectral Theory

Learning Objective: Compute and interpret eigenvalues and eigenvectors, diagonalization, characteristic polynomials, eigenspaces, symmetric and Hermitian matrices, the spectral theorem, quadratic forms and positive-definite matrices.

Category: core

8. Matrix Factorizations and Low-Rank Approximation

Learning Objective: Use LU, QR, Cholesky, eigendecomposition and singular value decomposition; understand pseudoinverses, compact SVD, low-rank approximation and the Eckart–Young theorem for compression and latent-factor models.

Category: application

9. Numerical and Sparse Linear Algebra

Learning Objective: Analyze floating-point error, conditioning, stability and complexity; apply direct and iterative solvers, power iteration, conjugate gradients, sparse representations and randomized matrix algorithms.

Category: application

10. Matrix Calculus and Tensors

Learning Objective: Work with multidimensional arrays, tensor operations, broadcasting and Einstein notation; compute gradients, Jacobians, Hessians and matrix derivatives using chain rules required for backpropagation.

Category: core

11. Linear Algebra for Optimization

Learning Objective: Connect gradients, Hessians, convexity, positive semidefiniteness, constrained optimization, Lagrange multipliers and regularization to gradient-based learning and optimization diagnostics.

Category: application

12. Probability, Statistics and Data Analytics

Learning Objective: Apply covariance and correlation matrices, multivariate Gaussian geometry, whitening, principal component analysis, canonical ideas and dimensionality reduction to exploratory analytics and uncertainty-aware decisions.

Category: application

13. Linear Algebra in Machine Learning and AI

Learning Objective: Apply data matrices, kernels, embeddings, similarity measures, linear models, neural-network layers, attention, graph operators, convolutional representations and matrix factorization to classification, recommendation and representation learning.

Category: advanced

14. Scalable Implementation and Industry Practice

Learning Objective: Implement reliable computations with contemporary libraries and accelerators; evaluate dense, sparse and distributed workloads, memory and precision trade-offs, reproducibility, benchmarking, deployment cost and build-versus-buy decisions.

Category: strategic

15. Responsible Practice and Emerging Directions

Learning Objective: Evaluate bias in embeddings and dimensionality reduction, privacy leakage, adversarial sensitivity, numerical auditability and governance controls; examine mixed precision, tensor decomposition, federated methods, neuromorphic and quantum linear algebra, and future AI requirements.

Category: contemporary