Linear Algebra in Machine Learning and AI

Introduction and Context

Linear algebra provides the language through which modern machine-learning systems represent data, transform information, measure similarity, and learn parameters. An image can be represented as a matrix of pixel intensities, a document as a vector of word or embedding features, a recommender system as a sparse user–item matrix, and a neural network as a sequence of matrix transformations. Consequently, understanding linear algebra is not merely a mathematical prerequisite: it is essential for designing, interpreting, implementing, and debugging AI systems.

This chapter connects core linear-algebraic objects—vectors, matrices, inner products, norms, linear maps, eigenvectors, singular values, and factorizations—to classification, recommendation, representation learning, neural networks, attention, graphs, and convolution. The emphasis is on dimensions, assumptions, geometric interpretation, computational structure, and practical limitations.

Learning Objectives

After studying this chapter, you should be able to:

1. Represent datasets, features, labels, and embeddings using vectors and matrices.

2. Use inner products, norms, distances, and kernels to quantify similarity.

3. Explain linear and affine models, least squares, and regularization.

4. Interpret neural-network layers as compositions of matrix transformations and nonlinearities.

5. Describe the matrix operations underlying attention, graph learning, and convolution.

6. Explain matrix factorization for recommendation and dimensionality reduction.

7. Analyze computational, statistical, ethical, and deployment limitations of linear-algebraic AI systems.

1. Data as Geometric Objects

Suppose a dataset contains $n$ observations and $d$ measured features. Using the convention that observations are rows, the data matrix is

$$X \in \mathbb{R}^{n\times d}.$$

The $i$th row $x_i^T$ describes observation $i$, and the $j$th column contains feature $j$. A target vector for a regression or binary-classification problem is commonly written as $y\in\mathbb{R}^n$. A categorical target may instead be represented by a one-hot matrix $Y\in\{0,1\}^{n\times C}$, where $C$ is the number of classes.

The orientation convention matters. If observations are columns, the same data may be represented as a $d\times n$ matrix, but every subsequent formula must be changed consistently. Many implementation errors arise from confusing feature dimensions with batch dimensions.

A vector is not only a list of numbers. It is a point or directed displacement in a vector space. A feature vector may have physical units, while an embedding usually has learned coordinates without direct semantic units. Operations such as addition and dot products are meaningful only when dimensions and interpretations are compatible.

Scaling and preprocessing

Features with different scales can distort geometric calculations. Standardization transforms feature $j$ according to

$$z_{ij}=\frac{x_{ij}-\mu_j}{\sigma_j},$$

where $\mu_j$ and $\sigma_j>0$ are computed from the training set. The same training statistics must be applied to validation and test data; recomputing them on test data causes leakage. For sparse or heavy-tailed data, robust scaling, logarithmic transformations, or normalization may be preferable.

For a vector $x$, its Euclidean norm is

$$\|x\|_2=\sqrt{\sum_{j=1}^{d}x_j^2}.$$

The distance between $x$ and $z$ is $\|x-z\|_2$. The Manhattan norm is $\|x\|_1=\sum_j|x_j|$, and the maximum norm is $\|x\|_\infty=\max_j|x_j|$. Norm choice affects both optimization and the geometry of nearest-neighbor methods.

The inner product is

$$\langle x,z\rangle=x^Tz=\sum_{j=1}^{d}x_jz_j.$$

For nonzero vectors, cosine similarity is

$$\operatorname{cos}(x,z)=\frac{x^Tz}{\|x\|_2\|z\|_2}.$$

It measures angular alignment and ignores magnitude. It is useful for text and embedding retrieval, but it may be inappropriate when magnitude carries meaning. Euclidean distance and cosine similarity produce different rankings unless vectors have been normalized.

2. Linear Models for Prediction

A linear regression model with an intercept is

$$\hat y=Xw+b\mathbf{1},$$

where $w\in\mathbb{R}^d$, $b\in\mathbb{R}$, and $\mathbf{1}\in\mathbb{R}^n$ is an all-ones vector. Equivalently, append a column of ones to $X$ and incorporate $b$ into an augmented parameter vector. The model is affine in the original features because of the intercept.

For squared-error regression, the empirical objective is

$$L(w,b)=\frac{1}{n}\|Xw+b\mathbf{1}-y\|_2^2.$$

If $X$ has full column rank and no intercept is written separately, the least-squares minimizer satisfies the normal equations

$$X^TX\hat w=X^Ty,$$

and therefore

$$\hat w=(X^TX)^{-1}X^Ty,$$

when $X^TX$ is invertible. In practice, one should usually solve the least-squares problem using QR or singular-value decomposition rather than explicitly computing an inverse. The formula also fails when features are linearly dependent or when the problem is ill-conditioned.

Regularization

Ridge regression adds an $L_2$ penalty:

$$\hat w=\arg\min_w\left\{\|Xw-y\|_2^2+\lambda\|w\|_2^2\right\},\qquad \lambda\ge 0.$$

For the unscaled objective, its solution is

$$\hat w=(X^TX+\lambda I)^{-1}X^Ty,$$

where $I$ is the $d\times d$ identity matrix. The intercept, if present, is commonly excluded from the penalty. Ridge reduces variance and improves conditioning, at the cost of introducing bias.

Lasso regression uses $\lambda\|w\|_1$ instead. It can produce exact zeros and thus supports feature selection, although correlated features may be selected unpredictably. Regularization strength must be selected using training-only cross-validation.

Classification

A binary linear classifier computes a score

$$s(x)=w^Tx+b,$$

and predicts one class when $s(x)$ exceeds a chosen threshold. The decision boundary $w^Tx+b=0$ is a hyperplane. Logistic regression converts the score into a probability:

$$p(y=1\mid x)=\sigma(s(x))=\frac{1}{1+e^{-s(x)}}.$$

For multiclass classification, a weight matrix $W\in\mathbb{R}^{C\times d}$ produces logits $Wx+b$, and softmax gives

$$p_c=\frac{e^{z_c}}{\sum_{k=1}^{C}e^{z_k}}.$$

Softmax probabilities are normalized scores, not necessarily calibrated probabilities. Calibration must be evaluated separately.

3. Kernels and Similarity-Based Learning

A kernel is a function $k(x,z)$ that acts as an inner product in some feature space. A valid positive-semidefinite kernel has a Gram matrix $K$ with entries $K_{ij}=k(x_i,x_j)$ that is symmetric and satisfies $a^TKa\ge 0$ for every vector $a$.

The linear kernel is $k(x,z)=x^Tz$. A common nonlinear choice is the radial basis function kernel:

$$k(x,z)=\exp\left(-\frac{\|x-z\|_2^2}{2\sigma^2}\right),\qquad \sigma>0.$$

The parameter $\sigma$ controls locality. Small values make similarity decay rapidly; large values make many points appear similar. Kernel methods can represent nonlinear decision boundaries without explicitly constructing the feature map, a principle called the kernel trick.

Kernel matrices can be expensive: storing $K$ requires $O(n^2)$ memory, and many algorithms require at least cubic-time operations in $n$ in their basic form. Approximate feature maps, low-rank approximations, or random features can reduce this cost. A kernel must also be chosen consistently with the data representation and validation procedure.

4. Embeddings and Representation Learning

An embedding maps an object into a vector space in which useful relations may be expressed geometrically. A word, product, user, image, or document can be represented by a vector in $\mathbb{R}^r$. Similarity may be computed with a dot product or cosine similarity, but the meaning of each dimension is generally distributed rather than directly interpretable.

A learned embedding matrix for $V$ discrete entities is $E\in\mathbb{R}^{V\times r}$. If entity $i$ is represented by a one-hot row vector $e_i^T$, then its embedding is $e_i^TE$, equivalently the $i$th row of $E$. This is computationally implemented as a lookup, not necessarily as a dense matrix multiplication.

Representation learning attempts to produce features that make a downstream task easier. A useful representation may preserve semantic similarity, support classification, or make variation factors more separable. However, embeddings can encode historical bias, private attributes, and spurious correlations. Similarity in embedding space is task-dependent rather than an objective measure of human meaning.

5. Neural-Network Layers as Matrix Transformations

A fully connected layer maps an input batch $H\in\mathbb{R}^{n\times d_{in}}$ to

$$Z=HW^T+\mathbf{1}b^T,$$

where $W\in\mathbb{R}^{d_{out}\times d_{in}}$ and $b\in\mathbb{R}^{d_{out}}$. An elementwise activation produces $H' = \phi(Z)$, such as ReLU, $\phi(t)=\max(0,t)$. A multilayer perceptron composes these affine maps and nonlinearities. Without nonlinearities, a composition of linear maps is still one linear map and cannot express genuinely nonlinear decision boundaries.

The Jacobian describes local sensitivity. For $f(x)=Wx+b$, the Jacobian is $W$. For a composition, the chain rule multiplies Jacobian matrices. This explains both gradient propagation and numerical difficulties: repeated singular values greater than one can cause exploding gradients, while values below one can cause vanishing gradients.

Biases translate representations, whereas weight matrices rotate, scale, shear, or project them. Rank limits the number of independent directions a layer can transmit. A bottleneck layer with output dimension smaller than the input can encourage compression, but compression may discard information needed for a particular task.

6. Attention and Similarity Matrices

Given a sequence representation $H\in\mathbb{R}^{n\times d}$, attention constructs queries, keys, and values:

$$Q=HW_Q,\qquad K=HW_K,\qquad V=HW_V.$$

For model dimension $d_k$, scaled dot-product attention is

$$\operatorname{Attention}(Q,K,V)=\operatorname{softmax}\left(\frac{QK^T}{\sqrt{d_k}}+M\right)V.$$

Here $QK^T\in\mathbb{R}^{n\times n}$ contains pairwise query–key scores, $M$ is an optional mask whose entries may be zero or $-\infty$ before softmax, and softmax is applied row-wise. The scaling factor reduces the tendency of dot products to become large when $d_k$ is large. The result is a weighted combination of rows of $V$ for each query.

Attention is therefore a data-dependent linear combination of values, preceded by a learned similarity calculation. Its principal computational cost is $O(n^2d_k)$ for a sequence of length $n$, making long contexts expensive in memory and time. Sparse, local, low-rank, and kernel-inspired attention methods seek to reduce this cost, but may alter the information available to the model.

Attention weights should not automatically be interpreted as explanations. They indicate one computational weighting mechanism, not necessarily causal importance or human-understandable reasoning.

7. Graph Operators and Graph Neural Networks

A graph with $n$ nodes can be represented by an adjacency matrix $A\in\mathbb{R}^{n\times n}$. For an unweighted undirected graph, $A_{ij}=1$ when nodes $i$ and $j$ are connected and $A_{ij}=0$ otherwise. The degree matrix is diagonal:

$$D_{ii}=\sum_j A_{ij}.$$

The unnormalized graph Laplacian is

$$L=D-A.$$

For any vector $x$ on the nodes,

$$x^TLx=\frac{1}{2}\sum_{i,j}A_{ij}(x_i-x_j)^2,$$

for a symmetric adjacency matrix. Thus, the Laplacian quadratic form penalizes differences between connected nodes and measures graph smoothness.

A normalized adjacency operator often used in graph convolution is

$$\tilde A=\tilde D^{-1/2}\tilde A_{raw}\tilde D^{-1/2},$$

where self-loops are added by $\tilde A_{raw}=A+I$ and $\tilde D_{ii}=\sum_j\tilde A_{raw,ij}$. A basic graph-convolution layer is

$$H^{(l+1)}=\phi\left(\tilde A H^{(l)}W^{(l)}\right).$$

The ordering of multiplication is dimensionally valid: $H^{(l)}$ has one row per node, $W^{(l)}$ transforms features, and $\tilde A$ mixes information among neighboring nodes. Repeated propagation can cause over-smoothing, in which node representations become too similar. Graph construction, missing edges, directionality, and temporal changes can all substantially affect predictions.

8. Convolutional Representations

For a one-dimensional signal $x$ and filter $w$, a discrete cross-correlation commonly used in neural-network libraries is

$$y[t]=\sum_{k=0}^{m-1}w[k]x[t+k],$$

subject to a specified padding convention. Strict mathematical convolution reverses the filter, using $w[k]$ with $x[t-k]$; the distinction should be stated because many machine-learning implementations call cross-correlation convolution.

For an image, a small filter is applied across local spatial patches. Sharing the same filter weights across locations gives translation equivariance: translating the input generally translates intermediate responses, apart from boundary effects and stride. Multiple filters create output channels. Stride reduces spatial resolution, while padding controls output size. Local connectivity and parameter sharing make convolution more efficient than a fully connected layer for images, but they impose assumptions about locality and stationarity.

A convolution can be expressed as multiplication by a structured sparse matrix, often called an im2col or Toeplitz representation. This reveals its linear-algebraic nature but is not always the most memory-efficient implementation.

9. Matrix Factorization for Recommendation and Compression

Let $R\in\mathbb{R}^{m\times n}$ denote a user–item rating matrix, with missing ratings represented by a mask $\Omega$. Matrix factorization approximates observed entries by

$$R_{ui}\approx p_u^Tq_i,$$

where $p_u,q_i\in\mathbb{R}^r$ are user and item latent vectors. A regularized objective is

$$\min_{P,Q}\sum_{(u,i)\in\Omega}(R_{ui}-p_u^Tq_i)^2+\lambda\left(\|P\|_F^2+\|Q\|_F^2\right).$$

Here $\|P\|_F^2=\sum_{ij}P_{ij}^2$ is the squared Frobenius norm. The objective is not jointly convex in $P$ and $Q$, although it is convex in either factor when the other is fixed. Alternating least squares and stochastic gradient descent are common optimization methods.

Bias terms can model overall, user, and item tendencies:

$$\hat R_{ui}=\mu+b_u+b_i+p_u^Tq_i.$$

The latent dimension $r$ controls capacity. Larger values can improve training fit but increase overfitting and computation. Recommendation evaluation must respect time and exposure: a system cannot fairly be evaluated only on items users chose to rate, because unobserved does not necessarily mean disliked.

Singular value decomposition (SVD) factorizes a real matrix as

$$X=U\Sigma V^T,$$

where $U$ and $V$ have orthonormal columns and $\Sigma$ contains nonnegative singular values in descending order. The rank-$r$ truncated SVD retains the largest $r$ singular values and is the best rank-$r$ approximation under both the Frobenius norm and the spectral norm. This result applies to a fully observed matrix and should not be confused with ordinary SVD applied directly to a sparse, missing-data rating matrix.

SVD supports principal component analysis. After centering columns of $X$, the right singular vectors identify principal directions, while squared singular values are proportional to variance explained. PCA is sensitive to scaling and outliers, and its components are not automatically causal or semantically meaningful.

Worked Example: A Small Recommendation Model

Consider two users and three items, with observed ratings

$$R=\begin{bmatrix}5&4&?\\2&?&1\end{bmatrix}.$$

Use one-dimensional latent factors $p_1=2$, $p_2=1$, $q_1=2$, $q_2=1.5$, and $q_3=0.5$. The predicted rating for user 1 and item 2 is

$$\hat R_{12}=p_1q_2=2(1.5)=3.$$

The prediction error for the observed rating 4 is $4-3=1$, and its squared error is $1$. For user 2 and item 3,

$$\hat R_{23}=p_2q_3=1(0.5)=0.5,$$

so the squared error for the observed rating 1 is $(1-0.5)^2=0.25$. The sum of these two squared errors is $1.25$, before including other observed entries or regularization. This example illustrates both the strength and limitation of latent factors: a small number of dimensions can capture broad preferences, but individual predictions may be inaccurate and the factors are not uniquely identifiable. For example, multiplying every user factor by a nonzero constant and dividing every item factor by the same constant leaves dot-product predictions unchanged.

Applied Case: Illustrative News Recommendation System

Background

Consider an illustrative university news platform serving articles to students and staff. It records clicks, reading duration, search queries, and article metadata. The platform wants to recommend relevant articles while avoiding excessive repetition and protecting sensitive information.

Problem

The raw interaction matrix is extremely sparse: most users have interacted with only a small fraction of articles. New articles have no interaction history, and click data reflects both user interest and the ranking policy that exposed items.

Analysis

An engineering team first creates content embeddings from article text and categorical metadata. It normalizes embeddings before using cosine similarity for content retrieval. A collaborative factorization model is trained on time-split interactions, with regularization and negative examples selected carefully. A hybrid score combines content similarity, collaborative preference, freshness, and diversity. The team monitors matrix dimensions, missing values, leakage, and numerical stability.

The system also examines whether recommendations disproportionately expose certain topics or publishers. Sensitive attributes are not used for personalization unless there is a documented, lawful, and beneficial purpose. Access to raw interaction data is restricted, and retention periods are defined.

Approach and Outcome

Candidate generation uses approximate nearest-neighbor search over article embeddings, while a ranking model combines user and item factors with contextual features. Evaluation includes recall at a fixed cutoff, normalized discounted cumulative gain, coverage, novelty, calibration, and subgroup analysis. A controlled online experiment is used only after offline checks and safety review.

The resulting system improves relevance for users with sufficient history, but performance for new users remains weaker. This leads to a cold-start strategy based on broad, diverse content and optional preference selection rather than silent inference of sensitive characteristics.

Learning Points

1. Sparse matrices require explicit treatment of missingness and exposure bias.

2. Similarity, prediction, diversity, and fairness are distinct objectives.

3. Time-aware evaluation is more realistic than random splitting for evolving content.

4. Latent representations are useful but can encode institutional and social biases.

5. Linear-algebraic models must be embedded within governance, privacy, and monitoring processes.

Challenges, Limitations, and Best Practices

Numerical conditioning. Nearly dependent columns make least-squares estimates unstable. Centering, scaling, regularization, and QR or SVD-based solvers are safer than explicit matrix inversion.

Dimension and shape errors. Document whether examples are rows or columns. Track every tensor shape through a model and test small cases with known outputs.

Computational cost. Dense matrix multiplication may be prohibitive for high-dimensional data. Use sparsity, batching, low-rank approximation, quantization, or hardware-appropriate kernels when accuracy permits.

Distribution shift. A learned embedding or factorization reflects its training distribution. New users, new products, changed language, and altered policies can invalidate geometric relationships.

Interpretability. A mathematically transparent operation is not necessarily an interpretable decision. Linear coefficients can be unstable under collinearity, and attention weights or latent factors should not be treated as causal explanations without additional evidence.

Fairness and privacy. Similarity systems can reproduce unequal representation, enable profiling, or reveal membership in sensitive groups. Apply data minimization, access control, privacy-preserving training where appropriate, subgroup evaluation, and human oversight for consequential decisions.

Reproducibility. Record preprocessing statistics, feature order, model dimensions, random seeds where relevant, solver settings, and training–evaluation splits. Version data and embeddings because a changed vocabulary or normalization rule changes the geometry.

Contemporary Developments

Modern AI increasingly combines linear operators with nonlinear, probabilistic, and symbolic components. Efficient attention approximations, retrieval-augmented systems, graph transformers, multimodal embeddings, neural operators, and hardware-aware low-rank adaptation all rely on structured matrix computation. Quantization and sparsity reduce deployment costs, while differentiable retrieval and learned similarity functions blur the boundary between database search and representation learning.

These developments do not eliminate foundational linear algebra. They make careful reasoning about rank, conditioning, normalization, geometry, and computational complexity more important. A model may be large and nonlinear, but its individual operations remain subject to dimensional compatibility, numerical precision, and data-dependent assumptions.

Chapter Summary

Linear algebra underlies the representation and transformation of data in machine learning. Data matrices organize observations and features; vectors, norms, inner products, and kernels define similarity and geometry. Linear and logistic models use affine transformations for prediction, while regularization controls variance and ill-conditioning. Neural layers compose matrix maps with nonlinearities. Attention forms a similarity matrix and uses it to mix value vectors. Graph operators propagate information over adjacency structures, and convolution applies shared local linear filters. Matrix factorization and SVD provide tools for recommendation, compression, and dimensionality reduction.

Successful application requires more than manipulating formulas. Dimensions, scaling, missingness, conditioning, computational cost, evaluation design, privacy, bias, and distribution shift must all be considered. Linear algebra supplies powerful abstractions, but the validity of an AI system depends on how those abstractions are matched to data, tasks, and social context.

Key Terms

Vector: An ordered collection of values representing a point, feature set, or direction.

Matrix: A rectangular array used to organize data or represent a linear transformation.

Inner product: A scalar product, commonly $x^Tz$, measuring alignment between vectors.

Norm: A function measuring vector size, such as the Euclidean or $L_1$ norm.

Cosine similarity: The normalized inner product measuring angular alignment.

Affine map: A linear transformation plus a translation, such as $Wx+b$.

Gram matrix: A matrix of pairwise inner products or kernel evaluations.

Kernel: A valid similarity function corresponding to an inner product in a feature space.

Embedding: A learned vector representation of an object.

Logit: An unnormalized score produced before a classification normalization such as softmax.

Regularization: A penalty or constraint used to control model complexity.

Rank: The dimension of the span of a matrix's rows or columns.

Singular value decomposition: A factorization $X=U\Sigma V^T$ using orthonormal directions and singular values.

Principal component analysis: A variance-oriented dimensionality-reduction method based on centered data.

Attention: A mechanism that weights value vectors using query–key similarities.

Adjacency matrix: A matrix encoding connections or edge weights in a graph.

Graph Laplacian: The operator $D-A$ that measures graph smoothness.

Convolution: A local, shared-weight linear operation over an ordered signal or image.

Matrix factorization: Approximation of a matrix using products of lower-dimensional factors.

Cold start: The problem of making predictions for new users, items, or entities with little data.

Review Questions

Short Answer Questions

1. Why must preprocessing statistics be estimated using the training set only?

2. What is the geometric difference between Euclidean distance and cosine similarity?

3. Why are nonlinear activation functions necessary in a multilayer neural network?

4. What does the graph Laplacian quadratic form measure?

5. Why is explicit matrix inversion generally discouraged in least-squares implementations?

Long Answer Questions

1. Explain how a data matrix, a linear classifier, and a neural-network layer are connected through matrix operations. Include dimensions.

2. Compare ridge regression, lasso regression, and unregularized least squares in terms of objective, geometry, and practical use.

3. Derive the dimensions and interpret the operations in scaled dot-product attention.

4. Explain how SVD supports dimensionality reduction and discuss its limitations for sparse recommendation data.

5. Discuss the advantages and limitations of graph-convolutional representations compared with ordinary fully connected layers.

Analytical / Application Questions

1. A dataset has 10,000 observations and 40 features. Design a preprocessing and linear-classification pipeline, specifying matrix dimensions, scaling decisions, regularization, and evaluation safeguards.

2. A recommender system performs well for frequent users but poorly for new users and new items. Analyze the causes using matrix-factorization concepts and propose a hybrid solution.

3. An attention model becomes prohibitively expensive when sequence length doubles. Analyze the computational scaling and recommend two approaches, discussing their likely trade-offs.

Multiple Choice Questions with Answers

Question 1

For a data matrix X with n observations and d features stored as rows, what is its shape?

A. d × n

B. n × d

C. n × n

D. d × d

Correct Answer: B. n × d

Question 2

Which similarity measure is invariant to multiplying either nonzero vector by a positive scalar?

A. Euclidean distance

B. Manhattan distance

C. Cosine similarity

D. Squared error

Correct Answer: C. Cosine similarity

Question 3

What is the main effect of the L2 penalty in ridge regression?

A. It forces every coefficient to zero

B. It reduces coefficient magnitude and can improve conditioning

C. It guarantees a nonlinear decision boundary

D. It removes the intercept automatically

Correct Answer: B. It reduces coefficient magnitude and can improve conditioning

Question 4

In scaled dot-product attention, why is QK^T divided by the square root of d_k?

A. To make the output sparse

B. To remove all negative scores

C. To control the scale of dot products before softmax

D. To reduce the number of sequence positions

Correct Answer: C. To control the scale of dot products before softmax

Question 5

For an undirected graph, what is the unnormalized graph Laplacian?

A. A + D

B. A − D

C. D − A

D. D^{-1}A

Correct Answer: C. D − A

Question 6

Which factorization provides the best rank-r approximation under the Frobenius norm for a fully observed real matrix?

A. Truncated SVD

B. An arbitrary QR factorization

C. The adjacency factorization

D. A softmax factorization

Correct Answer: A. Truncated SVD

Question 7

If H has shape n × d_in and W has shape d_out × d_in, what is the shape of HW^T?

A. d_in × d_out

B. n × d_out

C. d_out × n

D. n × d_in

Correct Answer: B. n × d_out

Question 8

In a one-dimensional convolutional neural-network implementation, what property results from using the same filter weights at different positions?

A. Translation equivariance, subject to boundary and stride effects

B. Guaranteed translation invariance at every layer

C. Elimination of all nonlinear behavior

D. A full connection between every input pair

Correct Answer: A. Translation equivariance, subject to boundary and stride effects

Question 9

In a one-factor recommendation model, if a user factor is 3 and an item factor is 2, what is the predicted interaction without bias terms?

A. 1

B. 5

C. 6

D. 9

Correct Answer: C. 6

Question 10

What is a major limitation of a dense kernel Gram matrix for n training examples?

A. It always has zero diagonal entries

B. It requires O(n^2) storage

C. It cannot represent similarities

D. It is defined only for categorical labels

Correct Answer: B. It requires O(n^2) storage