Eigen Decomposition #
Eigen decomposition expresses a matrix using its eigenvectors and eigenvalues.
From lecture discussions, this is one of the most important ways to understand the internal structure of a matrix.
Instead of treating the matrix as a black box, eigen decomposition reveals its fundamental directions and scaling behaviour.
Key Idea: Eigen decomposition rewrites a matrix in terms of directions (eigenvectors) and scaling factors (eigenvalues). This makes complex transformations easier to understand and compute.
Core Formula #
For a square matrix:
\[ A = P D P^{-1} \]Where:
- P = matrix of eigenvectors
- D = diagonal matrix of eigenvalues
Why It Works (Lecture Insight) #
From lectures:
- Matrix multiplication represents a transformation
- Most vectors change direction
- Eigenvectors do NOT change direction
They only scale:
\[ A v = \lambda v \]So if we express everything in eigenvector basis:
- transformation becomes simple scaling
Connection to Diagonalization #
Eigen decomposition is essentially diagonalization.
\[ D = P^{-1} A P \]This shows:
- A becomes diagonal in eigenvector basis
- diagonal entries = eigenvalues
Conditions for Eigen Decomposition #
From lecture:
Matrix must have:
- n independent eigenvectors
If not:
- matrix is not diagonalizable
- decomposition fails
Special Case: Symmetric Matrices #
From slides:
Symmetric matrices have stronger properties:
\[ A = Q \Lambda Q^T \]Where:
- Q is orthogonal
- eigenvectors are orthonormal
Lecture insight: This makes computations more stable and simpler.
Why Eigen Decomposition is Useful #
1. Matrix Powers #
\[ A^k = P D^k P^{-1} \]2. Understanding Structure #
- Eigenvalues → scaling
- Eigenvectors → directions
3. Simplifying Computation #
- Diagonal matrix is easy to compute
- No cross interactions
Connection to SVD #
From lectures:
- Eigen decomposition works for square matrices
- SVD generalises this to all matrices
Also:
\[ \sigma_i = \sqrt{\lambda_i(A^T A)} \]Example #
Given matrix:
\[ A = \begin{bmatrix} 4 & 2 \\ 1 & 3 \end{bmatrix} \]Steps:
- Find eigenvalues
- Find eigenvectors
- Form P and D
Geometric Interpretation #
From lecture:
- Eigenvectors define axes
- Eigenvalues define scaling along axes
So transformation becomes:
- stretch or shrink along fixed directions
Hidden Exam Pattern #
From lectures:
- Eigen decomposition is combined with:
- diagonalization
- matrix powers
- SVD
Common Mistakes #
- Not checking independence of eigenvectors
- Confusing eigenvalues with singular values
- Incorrect inverse of P
- Skipping verification step
Strategy to Prepare #
- Practice eigenvalue problems
- Solve eigenvector systems
- Form decomposition clearly
- Link with diagonalization
Quick Summary Table #
| Concept | Meaning |
|---|---|
| A = PDP⁻¹ | Eigen decomposition |
| D | Eigenvalues |
| P | Eigenvectors |
| Condition | independent eigenvectors |
References #
Lecture slides
Course handout
Webinar discussions