MFML Lecture to Course Content Map

MFML Lecture to Course Content Map #

This file maps the uploaded Maths lecture PDFs and webinar PDFs against the official course handout/contact-session plan. It is intended as an exam preparation index and as a source map for future Hugo Markdown notes.

Course identity #

• Course: Mathematical Foundations for Machine Learning
• Course code: AIML ZC416
• Main areas: linear algebra, vector spaces, matrix decompositions, vector calculus, optimisation, PCA, and SVM.

Official module structure #

Contact session by lecture #

Webinar mapping #

Mid-Sem focus #

The course handout states that the Mid-Semester Test covers Weeks 1-8. So for Mid-Sem, focus on:

MFML Exam Revision Index

MFML Exam Revision Index #

This is a practical revision index for the uploaded Mathematical Foundations for Machine Learning material.

Exam split #

High-priority concept checklist #

Linear systems and matrices #

• Convert equations into matrix form Ax = b
• Understand solution types: no solution, unique solution, infinite solutions
• Identify pivot and free variables
• Understand row operations, REF/RREF, rank, nullity
• Know matrix inverse and transpose properties

Vector spaces #

• Definition of vector space and subspace
• Closure under addition and scalar multiplication
• Span, linear combination, linear independence
• Basis, dimension, rank
• Column space, row space, nullspace

Analytic geometry #

• Norm properties
• Manhattan norm and Euclidean norm
• Inner product definition
• Symmetric positive-definite matrices
• Distance, angle, orthogonality
• Orthonormal basis and Gram-Schmidt

Matrix decompositions #

• Determinant and trace
• Cofactor expansion
• Row operation effect on determinant
• Eigenvalue equation Av = λv
• Characteristic equation det(A - λI) = 0
• Diagonalisation A = PDP^{-1}
• Spectral theorem for symmetric matrices
• Cholesky decomposition
• SVD A = UΣV^T
• Low-rank approximation

Vector calculus #

• Derivative from first principles
• Partial derivatives
• Gradient as direction of steepest ascent
• Gradient of vector-valued functions
• Matrix-gradient identities
• Chain rule
• Backpropagation and automatic differentiation

Taylor series and Hessian #

• Taylor polynomial
• Taylor series and Maclaurin series
• Remainder term
• Taylor series in two variables
• Hessian matrix
• First derivative and second derivative tests
• Maxima, minima and saddle points

Gradient descent and optimisation #

• Negative gradient direction
• Learning rate/step size
• Line search
• Convergence and local minima
• Constrained vs unconstrained optimisation
• Lagrange multipliers
• Convex optimisation
• SGD and optimisation in ML
• Feature preprocessing and scaling
• Overfitting in optimisation examples

Nonlinear optimisation algorithms #

• Difficult surfaces: cliffs, valleys, flat regions
• Curvature and why first-order methods can struggle
• Momentum update and intuition
• AdaGrad
• RMSProp
• Adam
• Learning rate decay

PCA #

• Dimensionality reduction problem
• Centred data and covariance matrix
• Maximum variance view
• Projection/reconstruction view
• Principal components as eigenvectors of covariance matrix
• SVD relation to PCA
• Low-rank approximation and Eckart-Young theorem
• PCA in high dimensions
• Practical PCA steps

SVM #

• Linear classifiers
• Margin and support vectors
• Hard-margin SVM primal formulation
• Lagrangian for SVM
• KKT conditions
• Primal vs dual perspective
• Role of inner products
• Kernel trick
• Hinge loss
• Soft-margin SVM

Suggested revision order #

Phase 1: Foundations #

1. Lecture 1
2. Lecture 2
3. Lecture 3
4. Webinar 1 problems related to REF, nullspace, column space and subspaces

Phase 2: Matrix decompositions #

1. Lecture 4
2. Lecture 5
3. Webinar 1 and Webinar 2 eigenvalue/eigendecomposition problems

Phase 3: Calculus and optimisation foundations #

1. Lecture 6
2. Lecture 7
3. Lecture 8
4. Webinar 2 maxima/minima and Hessian problems

Phase 4: Optimisation for ML #

1. Lecture 9
2. Lecture 10
3. Lecture 11
4. Webinar 3 gradient-descent step-size problems

Phase 5: PCA and SVM #

1. Lecture 12
2. Lecture 13
3. Lecture 14
4. Lecture 15
5. Webinar 4 / SVM problems

What to ask me next #

Use these prompts when generating Hugo pages:

MFML Topic to Source Index

MFML Topic to Source Index #

This index tells you where to look when you want to create future notes or revise a topic.

Formula Sheet

March 12, 2026

Formula Sheet #

This page is a quick reference of definitions + formulas, grouped by the modules.

Notation #

• Sample size: \( n \) (sample), \( N \) (population)
• Sample mean: \( \bar{x} \) , population mean: \( \mu \)
• Sample variance: \( s^2 \) , population variance: \( \sigma^2 \)
• Sample SD: \( s \) , population SD: \( \sigma \)
• Complement: \( A^c \)
• Intersection (“and”): \( A\cap B \) , union (“or”): \( A\cup B \)
• Conditional probability: \( P(A\mid B) \)

1. Basic Probability & Statistics #

1.1 Measures of Central Tendency #

Arithmetic mean #

Sample mean (ungrouped):

Supervised Learning

January 3, 2026

Supervised Learning #

Trained using labelled data.
Each example in the training set includes the correct output.
The algorithm learns to generalise and make predictions on unseen data.
Generally more accurate than unsupervised methods.
Requires human intervention for labelling and setup.
Widely used due to its accuracy and efficiency.
Produces highly accurate results when trained on good-quality labelled data.

Classification #

Output is discrete (e.g. Yes/No, Spam/Not Spam).
Used for categorising data into predefined classes.
Support Vector Machine (SVM) is a common classifier (a linear classifier with margin-based separation).

Differentiation of Univariate Functions

Differentiation of Univariate Functions #

Differentiation measures rate of change.

For a function f(x), the derivative measures the rate of change.

Interpretation:

• Slope of tangent
• Instantaneous rate of change

Home | Vector Calculus

Artificial Intelligence

July 4, 2024

My AI Notes #

Learning how machines learn! My working notes as I learn AI.

flowchart LR
    AI[Artificial Intelligence]
    ML[Machine Learning]
    DL[Deep Learning]
    FM[Foundation Models]
    LLM[LLM Models]

    AI --> ML
    ML --> DL
    DL --> FM
    FM --> LLM

    style AI fill:#E1F5FE
    style ML fill:#C8E6C9
    style DL fill:#90CAF9
    style FM fill:#64B5F6
    style LLM fill:#FFCCBC

• Mathematical Foundations for Machine Learning
• Statistical Methods
• Machine Learning
• Deep Neural Networks

• AI Foundation
  • AI Stages: ANI, AGI, ASI
  • AI Stack
  • AI Pipeline
  • AI Learning Resources
• Machine Learning
  • Supervised Learning
  • Semi-Supervised Learning
  • Reinforcement Learning
  • ML Workflow
  • Regression (Linear)
  • Ordinary Least Squares
  • Cost Function
  • Gradient Descent
  • Classification (Linear)
  • Decision Tree
  • Instance-based Learning
  • Support Vector Machine
  • Bayesian Learning
  • Ensemble Learning
  • Unsupervised Learning
  • Evaluation/Comparison
  • ML Pipeline
• Generative AI
  • Foundation Models
  • LLM - Model
  • AI Agents
  • Retrieval-Augmented Generation (RAG)
• Deep Learning
  • Neural Networks
  • Artificial Neuron and Perceptron
  • LNN for Regression
  • Gradient Descent Algorithm
  • LNN for Classification
  • Deep Feedforward Neural Networks (DFNN) for Classification
  • Convolutional Neural Networks
  • Deep CNN Architectures
  • CNN Pipeline
  • Recurrent Neural Networks
  • Deep Recurrent Neural Networks
  • Attention Mechanism
  • Transformer
  • Optimisation of Deep models
  • Gradient Descent and Mini-Batch Gradient Descent
  • Momentum Methods
  • Regularisation for Deep models
  • DNN Formula and Numerical Sheet
• Mathematical Foundation
  • MFML Lecture to Course Content Map
  • MFML Exam Revision Index
  • MFML Topic to Source Index
  • Linear Algebra
    • Linear Systems
      • Systems of Linear Equations
      • Matrices
      • Matrix Transposition
      • Solving Linear Systems
      • Forward and Backward Substitution
      • Inverse Matrix
      • Convex Combination
    • Vector Spaces
      • Basis and Rank
      • Linear Independence
      • Norm
      • Inner Products and Dot Product
      • Lengths and Distances
      • Angles and Orthogonality
      • Orthonormal Basis
      • Feature Space
      • Cauchy–Schwarz
    • Matrix Decompositions
      • Special Matrices
      • Characteristic Polynomial
      • Determinant and Trace
      • Eigenvalues and Eigenvectors
      • Cholesky Decomposition
      • Eigen Decomposition
      • Diagonalization
      • Singular Value Decomposition (SVD)
      • Matrix Approximation
    • Dimensionality reduction and PCA
      • Primal and Dual Perspective for Linear SVM
      • Mathematical Preliminaries for SVM
      • Nonlinear SVM
  • Calculus
    • Vector Calculus
      • Differentiation of Univariate Functions
      • Partial Differentiation and Gradients
      • Gradients of Vector-Valued and Matrix Functions
      • Useful Gradient Identities
      • Backpropagation and Automatic Differentiation
      • Higher-order derivatives
      • Taylor’s series
      • Maxima and Minima
    • Continuous Optimisation
      • Optimisation using Gradient Descent
      • Constrained Optimisation
      • Lagrange Multipliers
      • Convex Optimisation
    • Nonlinear Optimisation
      • Challenges in Gradient-Based Optimisation
      • Stochastic Gradient Descent (SGD)
      • Momentum-Based Learning
      • Adaptive Methods: AdaGrad, RMSProp, Adam
      • Tuning Hyperparameters and Preprocessing
• Statistics
  • Formula Sheet
  • Stats Formula Sheet
  • Basic Statistics
  • Basic Probability
  • Hypothesis Testing
  • Prediction & Forecasting
  • Gaussian Mixture Model & Expectation Maximization
  • Conditional Probability & Bayes’ Theorem
    • Conditional Probability
    • Bayes’ Theorem
    • Naïve Bayes
  • Probability Distributions
    • Random Variables
    • Common Probability Distributions

• Machine Learning → The broad field where systems learn patterns from data to make predictions or decisions.
• Neural Networks → A subset of machine learning that uses interconnected artificial neurons to model complex relationships.
• Deep Learning → A subset of neural networks that uses many hidden layers to learn high-level features from large datasets.
• Foundation Models → Large deep learning models trained on massive datasets and reused across many tasks using transfer learning.
• LLMs (Large Language Models) → A specialised type of foundation model focused on understanding and generating human language.

flowchart TD
AI["Artificial<br/>Intelligence"]
ML["Machine<br/>Learning"]
NN["Neural<br/>Networks"]
DL["Deep<br/>Learning"]
FM["Foundation<br/>Models"]
LLM["LLM<br/>Models"]

AI --> ML
ML --> NN
NN --> DL
DL --> FM
FM --> LLM

LR["Linear<br/>Regression"]
DT["Decision<br/>Trees"]
ML --> LR
ML --> DT

MLP["MLP"]
CNN["CNN"]
NN --> MLP
NN --> CNN

CNNDL["CNN<br/>(deep)"]
RNN["RNN"]
DL --> CNNDL
DL --> RNN

BERT["BERT"]
CLIP["CLIP"]
FM --> BERT
FM --> CLIP

GPT["GPT"]
LLAMA["LLaMA"]
LLM --> GPT
LLM --> LLAMA

TEXT["Text"]
IMAGE["Images"]
AUDIO["Audio"]
VIDEO["Video"]
LLM --> TEXT
LLM --> IMAGE
LLM --> AUDIO
LLM --> VIDEO

style AI fill:#90CAF9,stroke:#1E88E5,color:#000
style ML fill:#90CAF9,stroke:#1E88E5,color:#000
style NN fill:#90CAF9,stroke:#1E88E5,color:#000

style DL fill:#CE93D8,stroke:#8E24AA,color:#000
style FM fill:#CE93D8,stroke:#8E24AA,color:#000

style LLM fill:#C8E6C9,stroke:#2E7D32,color:#000
style LR fill:#C8E6C9,stroke:#2E7D32,color:#000
style DT fill:#C8E6C9,stroke:#2E7D32,color:#000
style MLP fill:#C8E6C9,stroke:#2E7D32,color:#000
style CNN fill:#C8E6C9,stroke:#2E7D32,color:#000
style CNNDL fill:#C8E6C9,stroke:#2E7D32,color:#000
style RNN fill:#C8E6C9,stroke:#2E7D32,color:#000
style BERT fill:#C8E6C9,stroke:#2E7D32,color:#000
style CLIP fill:#C8E6C9,stroke:#2E7D32,color:#000
style GPT fill:#C8E6C9,stroke:#2E7D32,color:#000
style LLAMA fill:#C8E6C9,stroke:#2E7D32,color:#000
style TEXT fill:#C8E6C9,stroke:#2E7D32,color:#000
style IMAGE fill:#C8E6C9,stroke:#2E7D32,color:#000
style AUDIO fill:#C8E6C9,stroke:#2E7D32,color:#000
style VIDEO fill:#C8E6C9,stroke:#2E7D32,color:#000

Stats Formula Sheet

February 25, 2026

Stats Formula Sheet #

Keep this page as a quick reference of definitions + formulas.

Notation #

• Sample size: \( n \) (sample), \( N \) (population)
• Mean: \( \bar{x} \) (sample), \( \mu \) (population)
• Variance: \( s^2 \) (sample), \( \sigma^2 \) (population)
• Standard deviation: \( s \) (sample), \( \sigma \) (population)

Module 1: Basic Statistics #

Measures of Central Tendency #

Sample mean (ungrouped):

Partial Differentiation and Gradients

Partial Differentiation and Gradients #

For f(x1, x2, …, xn):

Gradient vector:

Gradient points in direction of steepest ascent.

flowchart LR
    Input --> Function
    Function --> Gradient
    Gradient --> Optimisation

Home | Vector Calculus

Linear Independence

Linear Independence #

A set of vectors is linearly independent if none of them can be written as a linear combination of the others.

Independence means each vector adds new information.