Eigenvalues, Eigenvectors and Spectral Theory

Introduction and Context

Eigenvalues and eigenvectors describe directions in which a linear transformation acts by simple scaling. Although the definition is algebraic, the idea has wide significance: eigenvectors reveal stable directions in dynamical systems, principal directions in data, vibration modes in engineering structures, and latent patterns in machine-learning models. Spectral theory provides the mathematical framework for studying these quantities, especially for symmetric and Hermitian matrices.

For a bachelor-level understanding of AI and machine learning, eigenanalysis is particularly important because principal component analysis (PCA), covariance analysis, graph-based learning, spectral clustering, matrix factorization, and many optimization methods depend on it. This chapter develops the concepts from first principles, explains when diagonalization is possible, and connects the theory with practical computation.

Learning Objectives

By the end of this chapter, you should be able to:

1. Define eigenvalues, eigenvectors, eigenspaces, and characteristic polynomials.

2. Compute eigenvalues and eigenvectors for small matrices.

3. Explain algebraic and geometric multiplicity.

4. Determine when a matrix is diagonalizable and construct a diagonalization.

5. State and apply the spectral theorem for real symmetric and complex Hermitian matrices.

6. Interpret quadratic forms and classify matrices as positive definite, positive semidefinite, negative definite, or indefinite.

7. Relate eigenvalues and eigenvectors to PCA, covariance matrices, stability, and spectral algorithms.

8. Recognize numerical, modeling, and governance limitations associated with spectral methods.

1. Linear Transformations and Invariant Directions

Let A be an n × n matrix representing a linear transformation from R^n to R^n. A nonzero vector v is an eigenvector of A if

A v = λ v,

where λ is a scalar called the eigenvalue associated with v. The vector v is not required to remain unchanged: its magnitude may be enlarged, reduced, or reversed. Its direction, however, remains on the same line through the origin.

If λ > 1, the transformation expands v; if 0 < λ < 1, it contracts v; if λ = 1, it leaves v unchanged; if λ < 0, it reverses direction as well as scaling; and if λ = 0, it maps v to the zero vector. Eigenvectors are never zero vectors, because every matrix maps the zero vector to zero and this would not identify a meaningful direction.

An eigenvalue is associated with infinitely many eigenvectors whenever one eigenvector exists: every nonzero scalar multiple c v is also an eigenvector associated with λ. Thus, an eigenvector is properly understood as a direction, or more precisely as a one-dimensional subspace in the simplest case.

Example: A diagonal matrix

For

A = [[4, 0], [0, -2]],

we have

A [1, 0]^T = 4[1, 0]^T,

and

A [0, 1]^T = -2[0, 1]^T.

Therefore, the coordinate directions are eigenvector directions, with eigenvalues 4 and −2. A vector on the x-axis is stretched by a factor of 4, whereas a vector on the y-axis is reflected and stretched by a factor of 2.

2. Characteristic Polynomial and Eigenspaces

The eigenvalue equation can be rearranged as

(A − λI)v = 0,

where I is the n × n identity matrix. A nonzero solution exists precisely when A − λI is singular. Therefore, eigenvalues satisfy

det(A − λI) = 0.

The polynomial

p_A(λ) = det(A − λI)

is called the characteristic polynomial of A. Some texts use det(λI − A); for an n × n matrix the two conventions differ only by a factor of (−1)^n and have the same roots.

Computing a 2 × 2 characteristic polynomial

For

A = [[a, b], [c, d]],

using the convention det(A − λI),

p_A(λ) = det([[a − λ, b], [c, d − λ]])

= (a − λ)(d − λ) − bc.

The roots of this polynomial are the eigenvalues. Once λ is known, its eigenvectors are obtained by solving

(A − λI)v = 0.

The eigenspace corresponding to λ is

E_λ = {v ∈ R^n : Av = λv} = null(A − λI).

It is a subspace and therefore includes the zero vector, although the zero vector is not itself called an eigenvector.

Worked Example 1: Eigenvalues and eigenspaces

Let

A = [[3, 1], [0, 2]].

The characteristic polynomial is

p_A(λ) = det([[3 − λ, 1], [0, 2 − λ]])

= (3 − λ)(2 − λ).

Thus the eigenvalues are λ = 3 and λ = 2.

For λ = 3,

A − 3I = [[0, 1], [0, −1]].

The equation (A − 3I)v = 0 implies y = 0. Hence

E_3 = span{[1, 0]^T}.

For λ = 2,

A − 2I = [[1, 1], [0, 0]].

The equation x + y = 0 gives x = −y. Thus

E_2 = span{[−1, 1]^T}.

The two eigenspaces have independent direction vectors, so A has a basis of eigenvectors and is diagonalizable.

3. Algebraic and Geometric Multiplicity

The algebraic multiplicity of an eigenvalue is its multiplicity as a root of the characteristic polynomial. The geometric multiplicity is the dimension of its eigenspace:

geometric multiplicity of λ = dim null(A − λI).

For every eigenvalue,

1 ≤ geometric multiplicity ≤ algebraic multiplicity.

If the algebraic and geometric multiplicities are equal for every eigenvalue, the matrix is diagonalizable. Repeated eigenvalues do not automatically prevent diagonalization.

Example of a repeated eigenvalue that is diagonalizable

The identity matrix I has characteristic polynomial (1 − λ)^n. Its only eigenvalue is 1 with algebraic multiplicity n. Its eigenspace is all of R^n, so its geometric multiplicity is also n. It is diagonalizable—indeed, it is already diagonal.

Example of a repeated eigenvalue that is not diagonalizable

Consider

B = [[2, 1], [0, 2]].

Its only eigenvalue is 2, with algebraic multiplicity 2. However,

B − 2I = [[0, 1], [0, 0]],

so y = 0 and the eigenspace is span{[1, 0]^T}, which has dimension 1. The geometric multiplicity is less than the algebraic multiplicity; therefore B is not diagonalizable.

A matrix that cannot be diagonalized may sometimes be represented using Jordan form, but Jordan form is beyond the essential scope of this chapter. The important practical point is that a full set of independent eigenvectors is required for ordinary diagonalization.

4. Diagonalization

An n × n matrix A is diagonalizable if there exists an invertible matrix P and a diagonal matrix D such that

A = P D P^−1.

The columns of P are linearly independent eigenvectors of A, and the corresponding eigenvalues appear in the same order along the diagonal of D:

P = [v_1 v_2 ... v_n], D = diag(λ_1, λ_2, ..., λ_n).

Equivalently,

A P = P D.

Multiplying by P^−1 gives the diagonalization formula.

Diagonalization is useful because powers and matrix functions become easy to compute:

A^k = P D^k P^−1,

where D^k = diag(λ_1^k, ..., λ_n^k). This is valuable in discrete-time dynamical systems, Markov models, and repeated application of a linear transformation.

Worked Example 2: Constructing a diagonalization

For

A = [[3, 1], [0, 2]],

we found eigenvectors

v_1 = [1, 0]^T for λ_1 = 3,

v_2 = [−1, 1]^T for λ_2 = 2.

Therefore,

P = [[1, −1], [0, 1]], D = [[3, 0], [0, 2]].

The determinant of P is 1, so P is invertible. Its inverse is

P^−1 = [[1, 1], [0, 1]].

Consequently,

A = P D P^−1.

For example,

A^k = P [[3^k, 0], [0, 2^k]] P^−1.

This avoids multiplying A by itself k times. The formula is valid for nonnegative integer k and, with suitable conditions, can be extended to other matrix functions.

5. Fundamental Properties of Eigenvalues

Several useful facts follow from the characteristic polynomial and basic matrix theory.

Trace and determinant

For an n × n matrix over the complex numbers, counting algebraic multiplicity,

trace(A) = λ_1 + λ_2 + ... + λ_n,

det(A) = λ_1 λ_2 ... λ_n.

Thus, a matrix is singular if and only if at least one eigenvalue is zero. The trace is the sum of diagonal entries and the determinant is the product of eigenvalues.

Triangular matrices

For an upper- or lower-triangular matrix, the eigenvalues are its diagonal entries. This follows because the determinant of a triangular matrix is the product of its diagonal entries.

Similar matrices

If B = P^−1 A P for an invertible P, then A and B are similar. Similar matrices represent the same linear transformation in different bases and have the same characteristic polynomial, determinant, trace, and eigenvalues. Their coordinate eigenvectors differ because the basis has changed.

Eigenvectors for distinct eigenvalues

Eigenvectors associated with distinct eigenvalues are linearly independent. This result explains why a matrix with n distinct eigenvalues is automatically diagonalizable.

6. Real Symmetric and Complex Hermitian Matrices

A real matrix A is symmetric if

A^T = A.

A complex matrix A is Hermitian if

A^* = A,

where A* is the conjugate transpose. Real symmetric matrices are a special case of Hermitian matrices.

These matrices are especially important because their eigenvalues are real and eigenvectors belonging to distinct eigenvalues are orthogonal. In fact, they possess an orthonormal basis of eigenvectors.

Why orthogonality occurs

Suppose A is real symmetric, Av = λv and Aw = μw, with λ ≠ μ. Then

v^T A w = μ v^T w.

Since A^T = A,

v^T A w = (A v)^T w = λ v^T w.

Therefore, (λ − μ)v^T w = 0. Since λ ≠ μ, v^T w = 0. Thus v and w are orthogonal.

The same argument uses conjugate transposes for Hermitian matrices.

7. The Spectral Theorem

Real version

If A is a real symmetric n × n matrix, then there exists an orthogonal matrix Q and a real diagonal matrix Λ such that

A = Q Λ Q^T,

where

Q^T Q = Q Q^T = I.

The columns of Q form an orthonormal basis of eigenvectors, and Λ contains the real eigenvalues.

Complex version

If A is a complex Hermitian matrix, then there exists a unitary matrix U such that

A = U Λ U*,

where U*U = I and Λ is real diagonal.

The real symmetric form is most common in introductory machine learning because data matrices and covariance matrices are usually real.

The spectral theorem is stronger than ordinary diagonalizability. Every real symmetric matrix is diagonalizable, and the diagonalizing matrix can be chosen orthogonal. Consequently,

A^k = Q Λ^k Q^T,

and for a suitable scalar function f,

f(A) = Q f(Λ) Q^T.

Here f(Λ) is formed by applying f to each diagonal eigenvalue, provided the function is defined on the spectrum of A.

Orthogonal projection interpretation

If q_1, ..., q_n are orthonormal eigenvectors, then

A = Σ_i λ_i q_i q_i^T.

Each matrix q_i q_i^T is the orthogonal projection onto the direction q_i. Therefore, A acts by decomposing a vector into orthogonal eigen-directions and scaling each component by its eigenvalue.

8. Quadratic Forms

A quadratic form associated with a real symmetric matrix A is

q(x) = x^T A x,

where x ∈ R^n. Although x^T A x can be defined for any square A, only the symmetric part affects the value because

x^T A x = x^T [(A + A^T)/2] x.

Quadratic forms appear in energy functions, least squares, optimization, covariance analysis, and geometry.

Using the spectral decomposition A = QΛQ^T and setting y = Q^T x, we obtain

x^T A x = y^T Λ y = Σ_i λ_i y_i^2.

This expression makes definiteness transparent.

Classification by quadratic forms

A real symmetric matrix A is:

• Positive definite if x^T A x > 0 for every nonzero x.
• Positive semidefinite if x^T A x ≥ 0 for every x.
• Negative definite if x^T A x < 0 for every nonzero x.
• Negative semidefinite if x^T A x ≤ 0 for every x.
• Indefinite if the quadratic form takes both positive and negative values.

For a real symmetric matrix, these conditions are equivalent to conditions on eigenvalues:

• Positive definite: every eigenvalue is positive.
• Positive semidefinite: every eigenvalue is nonnegative.
• Negative definite: every eigenvalue is negative.
• Negative semidefinite: every eigenvalue is nonpositive.
• Indefinite: at least one eigenvalue is positive and at least one is negative.

Example: Definiteness

Let

A = [[2, 0], [0, 5]].

Then

x^T A x = 2x_1^2 + 5x_2^2.

For every nonzero x, this is positive, so A is positive definite. Its eigenvalues are 2 and 5, confirming the eigenvalue criterion.

A positive semidefinite matrix may have zero eigenvalues. For example, diag(3, 0) gives 3x_1^2, which is never negative but equals zero for nonzero vectors on the second coordinate axis.

9. Positive-Definite Matrices and Machine Learning

Positive-definite matrices are central to optimization and statistics. If A is symmetric positive definite, then the function

f(x) = 1/2 x^T A x − b^T x + c

is strictly convex, where b is a compatible vector and c is a scalar. Its gradient is

∇f(x) = A x − b,

and its Hessian is A. Since A is positive definite, the unique minimizer satisfies

A x = b.

In machine learning, covariance matrices are positive semidefinite. A covariance matrix becomes positive definite when the measured features are linearly independent in the relevant sample or population setting. Regularization often replaces a covariance or Hessian matrix H with

H + αI,

where α > 0. This shifts every eigenvalue upward by α, improving conditioning and ensuring positive definiteness when H is positive semidefinite.

10. Eigenvalues in Principal Component Analysis

PCA seeks orthogonal directions that capture the greatest variance in centered data. Let X be an m × d data matrix whose columns represent d centered features. A common sample covariance matrix is

S = (1/(m − 1)) X^T X,

assuming m > 1 and the rows of X are observations. The matrix S is symmetric positive semidefinite.

PCA computes eigenvectors of S. If

S q_i = λ_i q_i,

with eigenvalues ordered λ_1 ≥ λ_2 ≥ ... ≥ λ_d ≥ 0, then:

• q_i is the i-th principal direction;
• λ_i is the variance along that direction;
• the first k eigenvectors define the best k-dimensional linear subspace in the least-squares reconstruction sense.

The proportion of total variance explained by the first k components is

(λ_1 + ... + λ_k)/(λ_1 + ... + λ_d),

provided the denominator is nonzero.

This interpretation depends on centering the data. Without centering, the leading direction may primarily represent the mean rather than variation around the mean. Feature scaling also matters: variables measured in large units can dominate the covariance matrix.

Worked Example 3: PCA interpretation

Suppose the covariance matrix has eigenvalues 9, 3, and 0. The total variance is 12. The first principal component explains 9/12 = 0.75, or 75%, and the first two explain 12/12 = 100%. Thus a two-dimensional representation preserves all variance in this example, while a one-dimensional representation preserves 75% according to the covariance-based criterion.

The eigenvectors, not only the eigenvalues, are required to transform observations into principal-component coordinates.

11. Spectral Radius and Dynamical Behavior

The spectral radius of A is

ρ(A) = max_i |λ_i|,

where the maximum is taken over all eigenvalues, including complex eigenvalues when necessary. For a diagonalizable matrix, repeated application of A is governed by powers λ_i^k. If all eigenvalues satisfy |λ_i| < 1, then A^k tends to the zero matrix as k becomes large. If at least one eigenvalue has magnitude greater than 1, some directions generally grow. Eigenvalues of magnitude exactly 1 require more careful analysis, particularly if the matrix is not diagonalizable.

This principle appears in recurrent models, iterative algorithms, population models, and Markov processes. For a stochastic transition matrix, eigenvalue 1 is always associated with stationary behavior under appropriate assumptions, while other eigenvalues influence the rate of convergence.

12. Numerical Computation and Best Practices

For small matrices, characteristic polynomials are useful by hand. For large matrices, explicitly computing a characteristic polynomial is usually inefficient and numerically unstable. Practical software commonly uses algorithms based on orthogonal transformations, such as QR methods, or specialized methods for symmetric matrices.

Important distinctions include:

• Exact eigenvalues versus numerical approximations.
• Eigenvalue accuracy versus eigenvector accuracy.
• Dense versus sparse matrices.
• Full eigendecomposition versus computing only dominant eigenpairs.

For symmetric matrices, use a symmetric/Hermitian eigensolver whenever possible. It exploits structure, returns real eigenvalues up to numerical error, and is generally more stable than treating the matrix as an arbitrary nonsymmetric matrix.

Eigenvectors are not unique: their sign may be reversed, and vectors within a repeated eigenspace may be replaced by another orthonormal basis. Therefore, comparing eigenvectors from two software runs by their raw entries can be misleading. Compare subspaces or account for sign and basis ambiguity.

The condition of an eigenvalue problem also matters. Closely spaced eigenvalues can make individual eigenvectors sensitive to small perturbations, even when the corresponding invariant subspace remains relatively stable. Data preprocessing, scaling, and numerical tolerance should be documented.

13. Applied Case: PCA for Industrial Sensor Monitoring

Background

This illustrative case concerns a manufacturing plant that monitors temperature, pressure, vibration, and flow rate. Many sensor readings are correlated because they respond to a smaller number of underlying operating conditions.

Problem

Engineers want to reduce the four-dimensional sensor stream to a smaller representation for visualization and anomaly detection. Storing and modeling every variable is possible, but correlated measurements may obscure the dominant operating patterns.

Analysis

The data matrix is centered, and a covariance matrix S is computed. Suppose its eigenvalues are approximately

6.4, 1.1, 0.4, 0.1.

The total variance is 8.0. The first component explains 80%, and the first two explain 93.75%. The corresponding eigenvectors identify combinations of sensors associated with dominant modes of operation.

Approach and Solution

The engineering team retains the first two principal components. Each new centered sensor observation x is projected using

z = Q_2^T x,

where Q_2 contains the first two unit eigenvectors as columns. A monitoring model is built in this two-dimensional coordinate system. Reconstruction error, or the size of the discarded component, is used as one anomaly indicator.

Outcome

The reduced representation captures most ordinary operating variation while lowering computational cost and making visual inspection easier. An unusual sensor pattern that does not conform to the learned two-dimensional subspace produces a relatively large reconstruction error.

Learning Points

1. Eigenvalues quantify variance captured by principal directions.

2. Eigenvectors provide interpretable linear combinations of sensor variables.

3. Centering is necessary for ordinary covariance-based PCA.

4. A high explained-variance ratio does not guarantee that rare, safety-critical events are preserved.

5. Engineers must inspect loadings, units, missing-data handling, and sensor bias rather than treating PCA as an automatic explanation.

14. Limitations, Ethics, and Governance

Eigenanalysis is mathematically precise, but its interpretation depends on the matrix and data-generating process. PCA captures linear variance, not necessarily predictive information, causal structure, or fairness-relevant variation. A low-variance feature may be crucial for detecting a rare safety event or a protected-group disparity.

In high-stakes applications, practitioners should document the data used to construct the matrix, preprocessing decisions, scaling choices, missing-value treatment, and the number of retained components. If eigenvectors are used to support decisions about people, they should be audited for proxy discrimination and monitored for distribution shift.

Numerical approximations should not be reported with unjustified precision. Reproducibility requires recording software versions, tolerances, random seeds where applicable, and whether covariance matrices were regularized. Privacy requirements may also affect whether the underlying data can be used to calculate and retain spectral summaries.

15. Best Practices

• Verify dimensions before forming products or eigenvalue equations.
• For hand calculations, use det(A − λI) consistently.
• Check eigenpairs by evaluating the residual ||Av − λv||.
• Normalize eigenvectors when comparing directions or constructing orthogonal bases.
• Use structure-aware numerical solvers for symmetric and Hermitian matrices.
• Distinguish an eigenvector’s sign ambiguity from a genuine change in direction.
• Inspect eigenvalue multiplicities and eigenspace dimensions before claiming diagonalizability.
• Center and, when justified, scale data before PCA.
• Use explained variance as one criterion, not as the sole measure of usefulness.
• Regularize ill-conditioned covariance or Hessian matrices when appropriate.
• Communicate uncertainty, approximation error, and limitations to technical and nontechnical stakeholders.

Chapter Summary

Eigenvalues identify scaling factors associated with invariant directions of a linear transformation, while eigenvectors identify those directions. They are computed from the characteristic equation det(A − λI) = 0, and their associated eigenspaces are null spaces of A − λI. Algebraic multiplicity counts repeated roots; geometric multiplicity measures eigenspace dimension. A matrix is diagonalizable when it has a basis of linearly independent eigenvectors.

Real symmetric and complex Hermitian matrices have especially strong structure: their eigenvalues are real, eigenvectors for distinct eigenvalues are orthogonal, and an orthonormal eigenbasis exists. The spectral theorem expresses such matrices as QΛQ^T or UΛU*. This representation supports efficient powers, matrix functions, geometric interpretation, and analysis of quadratic forms.

Eigenvalues characterize positive definiteness for symmetric matrices and determine convexity of quadratic objectives. In AI and machine learning, covariance eigenvectors and eigenvalues form the foundation of PCA, while spectral radius helps explain repeated linear dynamics. Reliable practice requires attention to numerical stability, scaling, preprocessing, repeated-eigenvalue ambiguity, and the limitations of variance-based interpretations.

Key Terms

Eigenvalue: A scalar λ for which Av = λv has a nonzero solution.

Eigenvector: A nonzero vector whose direction is preserved by a linear transformation, up to scaling.

Eigenspace: The null space of A − λI associated with eigenvalue λ.

Characteristic polynomial: The polynomial det(A − λI), whose roots are the eigenvalues.

Algebraic multiplicity: The multiplicity of an eigenvalue as a root of the characteristic polynomial.

Geometric multiplicity: The dimension of the eigenspace associated with an eigenvalue.

Diagonalizable matrix: A matrix that has a basis of eigenvectors and can be written PDP^−1.

Similar matrices: Matrices related by B = P^−1AP and therefore having the same eigenvalues.

Symmetric matrix: A real matrix satisfying A^T = A.

Hermitian matrix: A complex matrix satisfying A* = A.

Orthogonal matrix: A real matrix Q satisfying Q^TQ = I.

Unitary matrix: A complex matrix U satisfying U*U = I.

Spectral theorem: The theorem guaranteeing orthonormal diagonalization for symmetric or Hermitian matrices.

Quadratic form: An expression x^TAx associated with a matrix A.

Positive definite: A symmetric matrix for which x^TAx > 0 for every nonzero x.

Positive semidefinite: A symmetric matrix for which x^TAx ≥ 0 for every x.

Spectral radius: The largest absolute value of the eigenvalues of a matrix.

Principal component: An eigenvector direction of a covariance matrix, ordered by its eigenvalue.

Explained variance ratio: The fraction of total eigenvalue sum represented by selected principal components.

Regularization: A modification such as H + αI used to improve conditioning or enforce positive definiteness.

Review Questions

Short Answer Questions

1. Define an eigenvalue and an eigenvector of a square matrix.

2. What is the relationship between an eigenspace and the null space of A − λI?

3. Distinguish algebraic multiplicity from geometric multiplicity.

4. State the spectral theorem for a real symmetric matrix.

5. What does the largest eigenvalue represent in covariance-based PCA?

Long Answer Questions

1. Explain how to compute the characteristic polynomial, eigenvalues, and eigenspaces of a 2 × 2 matrix. Include a suitable example.

2. Explain the conditions under which a matrix is diagonalizable and discuss why repeated eigenvalues require care.

3. Derive the orthogonality of eigenvectors corresponding to distinct eigenvalues of a real symmetric matrix.

4. Explain the classification of a symmetric matrix as positive definite, positive semidefinite, negative definite, or indefinite using both quadratic forms and eigenvalues.

5. Describe how PCA uses the spectral decomposition of a covariance matrix and discuss the importance of centering and scaling.

Analytical / Application Questions

1. A symmetric covariance matrix has eigenvalues 12, 4, 2, and 0.5. Calculate the percentage of variance explained by the first two principal components and discuss whether retaining two components is justified.

2. Consider a matrix with characteristic polynomial (λ − 3)^2(λ + 1). Explain what additional information is needed to determine whether the matrix is diagonalizable.

3. An iterative system x_(k+1) = Ax_k has eigenvalues 0.8, −0.4, and 1.05. Analyze the likely long-run behavior and explain why eigenvectors and diagonalizability may affect the detailed trajectory.

Multiple Choice Questions with Answers

Question 1

Which equation determines the eigenvalues of a square matrix A?

A. det(A + λI) = 0

B. det(A − λI) = 0

C. det(A) − λ = 0

D. Aλ = I

Correct Answer: B. det(A − λI) = 0

Question 2

The geometric multiplicity of an eigenvalue λ is the:

A. Multiplicity of λ in the determinant of A

B. Number of diagonal entries equal to λ

C. Dimension of null(A − λI)

D. Absolute value of λ

Correct Answer: C. Dimension of null(A − λI)

Question 3

A matrix with n distinct eigenvalues is necessarily:

A. Singular

B. Diagonalizable

C. Nilpotent

D. Positive definite

Correct Answer: B. Diagonalizable

Question 4

For a real symmetric matrix, eigenvectors corresponding to distinct eigenvalues are:

A. Parallel

B. Orthogonal

C. Always zero

D. Complex and non-orthogonal

Correct Answer: B. Orthogonal

Question 5

Which factorization is guaranteed by the spectral theorem for a real symmetric matrix A?

A. A = QΛQ^T with Q orthogonal

B. A = QΛQ with Q arbitrary

C. A = PDP with P singular

D. A = ΛQ^TQ^T with Λ rectangular

Correct Answer: A. A = QΛQ^T with Q orthogonal

Question 6

A symmetric matrix is positive definite when:

A. All eigenvalues are negative

B. At least one eigenvalue is zero

C. All eigenvalues are positive

D. It has a nonzero determinant only

Correct Answer: C. All eigenvalues are positive

Question 7

A covariance matrix has eigenvalues 8, 2, and 0. The first principal component explains:

A. 20% of the variance

B. 40% of the variance

C. 80% of the variance

D. 100% of the variance

Correct Answer: C. 80% of the variance

Question 8

If A = PDP^−1, then for a nonnegative integer k, A^k equals:

A. P^kD(P^−1)^k

B. PD^kP^−1

C. P^−1D^kP

D. D^k only

Correct Answer: B. PD^kP^−1

Question 9

The spectral radius of a matrix is:

A. The sum of its diagonal entries

B. The determinant of the matrix

C. The largest absolute value of its eigenvalues

D. The smallest eigenvalue

Correct Answer: C. The largest absolute value of its eigenvalues

Question 10

Adding αI, where α > 0, to a symmetric matrix shifts each eigenvalue by:

A. α

B. α^2

C. 1/α

D. 0

Correct Answer: A. α