Diagonalization #

Diagonalisation expresses a matrix using its eigenvectors and eigenvalues when possible.

From lecture explanation, diagonalisation is one of the most powerful tools because it converts a complicated matrix into a much simpler form.

Instead of working with a full matrix, we work with a diagonal matrix, which is much easier to analyse and compute.

Key Idea: If a matrix has enough independent eigenvectors, it can be rewritten as a diagonal matrix using a change of basis. This simplifies matrix operations significantly.

Singular Value Decomposition (SVD)

March 18, 2026

AI, ML

Machine Learning, Linear Algebra, SVD

Singular Value Decomposition (SVD) #

Singular Value Decomposition (SVD) is one of the most important matrix decomposition techniques in linear algebra and machine learning.

It factorises any matrix into three simpler matrices that reveal its structure.

Key Idea: SVD decomposes a matrix into rotations + scaling. It tells us how data is transformed along orthogonal directions.

Definition #

For any matrix in real space: \[ A \in \mathbb{R}^{m \times n} \]

Matrix Approximation

AI, ML

Machine Learning, Mathematics, Linear Algebra

Matrix Approximation #

Low-rank approximation keeps the most important structure while reducing noise and computation.

Low-Rank Approximation #

Used for:

Dimensionality reduction
Noise removal
Efficient computation

Forms the basis of PCA.

Home | Matrix Decompositions

Vector Calculus

AI, ML

Machine Learning, Mathematics, Vector Calculus

Vector Calculus #

Vector calculus extends differentiation to multivariate and vector-valued functions.

Gradients power learning. This section builds differentiation skills needed for backpropagation.

flowchart TD

    %% Core Node
    PD["Partial Derivatives"]

    %% Supporting Concepts
    DQ["Difference Quotient"]
    JH["Jacobian / Hessian"]
    TS["Taylor Series"]

    %% Application Chapters
    CH6["<br/>Probability"]
    CH7["<br/>Optimization"]
    CH9["<br/>Regression"]
    CH10["<br/>Dimensionality Reduction"]
    CH11["<br/>Density Estimation"]
    CH12["<br/>Classification"]

    %% Relationships
    DQ -->|defines| PD
    PD -->|collected in| JH
    JH -->|used in| TS
    JH -->|used in| CH6
	
    PD -->|used in| CH7
    PD -->|used in| CH9
    PD -->|used in| CH10
    PD -->|used in| CH11
    PD -->|used in| CH12

    %% Styling (Your Soft Academic Palette)
    style PD fill:#90CAF9,stroke:#1E88E5,color:#000

    style DQ fill:#CE93D8,stroke:#8E24AA,color:#000
    style JH fill:#CE93D8,stroke:#8E24AA,color:#000
    style TS fill:#CE93D8,stroke:#8E24AA,color:#000
    style CH6 fill:#CE93D8,stroke:#8E24AA,color:#000
	
    style CH7 fill:#C8E6C9,stroke:#2E7D32,color:#000
    style CH9 fill:#C8E6C9,stroke:#2E7D32,color:#000
    style CH10 fill:#C8E6C9,stroke:#2E7D32,color:#000
    style CH11 fill:#C8E6C9,stroke:#2E7D32,color:#000
    style CH12 fill:#C8E6C9,stroke:#2E7D32,color:#000

Home | Calculus

Continuous Optimisation

AI, ML

Machine Learning, Mathematics, Linear Algebra

Continuous Optimisation #

Optimisation finds parameters that minimise (or maximise) an objective function.

Home | Calculus

Optimisation using Gradient Descent

AI, ML

Machine Learning, Mathematics, Linear Algebra

Optimisation using Gradient Descent #

Gradient descent is an optimisation algorithm used to train ML and neural networks.

Gradient descent updates parameters by moving opposite the gradient.

Trains ML models by minimising errors:

between predicted and actual results
by iteratively adjusting its parameters
moves step‑by‑step in the direction of the steepest decrease in the loss function, it helps ML models learn the best possible weights for better predictions