The Cauchy–Schwarz Inequality is one of the most important results in linear algebra.
It places a fundamental bound on the inner product of two vectors.
If you see angle, cosine, similarity, or inner product bounds
→ think Cauchy–Schwarz InequalityKey Idea: The inner product (dot product) can never exceed the product of magnitudes. This ensures all geometric interpretations (angles, cosine) are valid.
For any vectors:
March 18, 2026Decompositions reveal structure in matrices and power algorithms like PCA.
Matrix decompositions break complex matrices into simpler parts.
From the lecture introduction, matrices are used to describe mappings and transformations of vectors.
That is why decomposition is important: it lets us understand a complicated transformation by rewriting it using simpler building blocks.
In the slides, the topic is introduced as part of three closely connected goals: how to summarise matrices, how matrices can be decomposed, and how the decompositions can be used for matrix approximations.
The characteristic polynomial of a square matrix is the key tool used to compute eigenvalues.
It connects:
Let
\( A \in \mathbb{R}^{n \times n} \)
and
\( \lambda \in \mathbb{R} \)
.
The characteristic polynomial of (A) is defined as:
\[ p_A(\lambda) = \det(A - \lambda I) \]
It is a polynomial in \( \lambda \) of degree (n).
The minor of an element \( a_{ij} \) is the determinant of the smaller square matrix formed by:
The minor is denoted \( M_{ij} \) .
Minors are used to compute cofactors, which are used for determinants and inverses (via adjoint/adjugate).
The cofactor of \( a_{ij} \) , denoted \( C_{ij} \) , is:
Eigenvalues and eigenvectors describe directions that remain unchanged under a linear transformation, except for scaling.
From lectures:
matrix multiplication represents a transformation of space.
Most vectors change direction and magnitude.
Some special vectors only scale.
These are eigenvectors.
Key Idea: A matrix transformation stretches or compresses vectors. Eigenvectors are directions that remain unchanged. Eigenvalues tell how much scaling happens.
Cholesky decomposition is a special matrix factorisation used for symmetric positive definite matrices.
From lecture discussions, this decomposition is powerful because it reduces a matrix into a triangular form, making computations easier and more stable.
Key Idea: Cholesky decomposition expresses a matrix as a product of a lower triangular matrix and its transpose. It is efficient and numerically stable.
For a symmetric positive definite matrix:
Eigen decomposition expresses a matrix using its eigenvectors and eigenvalues.
From lecture discussions, this is one of the most important ways to understand the internal structure of a matrix.
Instead of treating the matrix as a black box, eigen decomposition reveals its fundamental directions and scaling behaviour.
Key Idea: Eigen decomposition rewrites a matrix in terms of directions (eigenvectors) and scaling factors (eigenvalues). This makes complex transformations easier to understand and compute.
Diagonalisation expresses a matrix using its eigenvectors and eigenvalues when possible.
From lecture explanation, diagonalisation is one of the most powerful tools because it converts a complicated matrix into a much simpler form.
Instead of working with a full matrix, we work with a diagonal matrix, which is much easier to analyse and compute.
Key Idea: If a matrix has enough independent eigenvectors, it can be rewritten as a diagonal matrix using a change of basis. This simplifies matrix operations significantly.
March 18, 2026Singular Value Decomposition (SVD) is one of the most important matrix decomposition techniques in linear algebra and machine learning.
It factorises any matrix into three simpler matrices that reveal its structure.
Key Idea: SVD decomposes a matrix into rotations + scaling. It tells us how data is transformed along orthogonal directions.
For any matrix in real space: \[ A \in \mathbb{R}^{m \times n} \]
Low-rank approximation keeps the most important structure while reducing noise and computation.
Used for:
Forms the basis of PCA.