Linear Algebra #

Linear Algebra forms the foundation for representing data and transformations in machine learning.

Linear Algebra provides the mathematical language used to represent data, transformations, and structure in machine learning.

• Linear Systems
  • Systems of Linear Equations
  • Matrices
  • Matrix Transposition
  • Solving Linear Systems
  • Inverse Matrix
  • Convex Combination
• Vector Spaces
  • Linear Independence
  • Basis and Rank
  • Norm
  • Inner Products
  • Lengths and Distances
  • Angles and Orthogonality
  • Orthonormal Basis
  • Feature Space
• Matrix Decompositions
  • Special Matrices
  • Determinant and Trace
  • Eigenvalues and Eigenvectors
  • Diagonalization
  • Cholesky Decomposition
  • Eigen Decomposition
  • Singular Value Decomposition (SVD)
  • Matrix Approximation
• Dimensionality reduction and PCA
  • Principal Component Analysis (PCA)
  • PCA Theory
  • PCA in Practice
  • Latent Variable Perspective
  • Mathematical Preliminaries of SVM
  • Nonlinear SVM and Kernels

Why Linear Algebra Matters in ML #

• Every machine learning model uses matrices
• All data in ML is represented using vectors and matrices
• Neural networks are pipelines of matrix operations
• Models apply matrix transformations to data
• Optimisation relies on linear algebra operations

What to Learn #

• Scalars, vectors, and matrices
• Vector operations (addition, dot product)
• Matrix multiplication (critical)
• Identity matrices and transpose
• Eigenvalues and eigenvectors (conceptual understanding)

• Scalar → a number
• Vector → a directed point
• Matrix → a space transformer
• Linear transformation → structured mapping
• Feature → one axis
• Feature space → where data lives
• Vector space → where vectors live

Home | Mathematical Foundation