Support Vector Machine

Support Vector Machine (SVM) #

A Support Vector Machine (SVM) is a supervised machine learning algorithm used for:

• Classification (most common)
• Regression (SVR – Support Vector Regression)

Find the decision boundary that separates classes with the maximum margin.

A Support Vector Machine is a supervised learning algorithm that finds an optimal hyperplane by maximising the margin between classes, using support vectors and kernel functions to handle non-linear data.

Deep Recurrent Neural Networks

April 19, 2026

Deep Recurrent Neural Networks #

Vanilla RNNs introduce the hidden-state idea, but they struggle on longer and more complex sequences because gradients can vanish across time. Deep recurrent models extend the RNN idea in two important ways:

1. make the recurrent architecture richer, for example by stacking multiple recurrent layers or using information from both directions,
2. use gates and memory cells to control what should be remembered, forgotten, updated, and exposed.

This is why practical recurrent modelling usually moves from a simple RNN to stacked RNNs, bidirectional RNNs, GRUs, or LSTMs.

Attention Mechanism

Attention Mechanism #

• Queries, Keys, and Values
• Attention Pooling by Similarity
• Attention Pooling via Nadaraya–Watson Regression
• Attention Scoring Functions
• Dot Product Attention
• Convenience Functions
• Scaled Dot Product Attention
• Additive Attention
• Bahdanau Attention Mechanism
• Multi-Head Attention
• Self-Attention
• Positional Encoding
• Code implementation (webinar)

Reference #

• Dive into deep learning. Cambridge University Press.. (Ch 10, Ch7

Home | Deep Learning

Bayesian Learning

Bayesian Learning #

MLE Hypothesis #

MAP Hypothesis #

Bayes Rule #

Optimal Bayes Classifier #

Naïve Bayes Classifier #

Probabilistic Generative Classifiers #

Bayesian Linear Regression #

Home | Machine Learning

Transformer

December 15, 2025

Transformer #

• is an architecture of neural networks

• based on the multi-head attention mechanism

• text is converted to numerical representations called tokens, and each token is converted into a vector via lookup from a word embedding table

• takes a text sequence as input and produces another text sequence as output

• foundation for modern Large Language Models (LLMs) like ChatGPT and Gemini

• Transformer architecture

• Model, Positionwise Feed-Forward Networks, Residual Connection and Layer Normalization

Ensemble Learning

Ensemble Learning #

Combining Classifiers #

Bagging #

Random Forest #

Boosting #

ADABoost #

Gradient Boosting #

XGBoost #

Home | Machine Learning

Optimisation of Deep models

Optimisation of Deep models #

• Goal of Optimization
• Optimization Challenges in Deep Learning
• Gradient Descent
• Stochastic Gradient Descent
• Minibatch Stochastic Gradient Descent
• Momentum
• Adagrad and Algorithm
• RMSProp and Algorithm
• Adadelta and Algorithm
• Adam and Algorithm
• Code Implementation and comparison of algorithms (webinar)

Reference #

• Dive into deep learning. Cambridge University Press.. (Ch12)

Home | Deep Learning

Evaluation/Comparison

Machine Learning Model Evaluation/Comparison #

Comparing Machine Learning Models #

Emerging requirements e.g., bias, fairness, interpretability of ML models #

Home | Machine Learning

Regularisation for Deep models

Regularisation for Deep models #

• Generalization for regression
• Training Error and Generalization Error
• Underfitting or Overfitting
• Model Selection
• Weight Decay and Norms
• Generalization in Classification
• Environment and Distribution Shift
• Generalization in Deep Learning
• Dropout
• Batch Normalization
• Layer Normalization
• Code implementation (webinar)

Reference #

• Dive into deep learning. Cambridge University Press.. (T1 – Ch 3.6, 3.7, T1 - Ch 4.6, 4.7, T1 - Ch 5.5, 5.6, T1 - Ch 8.5, T1 - Ch 11.7

Home | Deep Learning

Linear Algebra

March 18, 2026

Linear Algebra #

The study of vectors and matrices is called Linear Algebra.

Linear Algebra provides the mathematical language used to represent data, transformations, and structure in ML.

• Linear Systems
  • Systems of Linear Equations
  • Matrices
  • Matrix Transposition
  • Solving Linear Systems
  • Forward and Backward Substitution
  • Inverse Matrix
  • Convex Combination
• Vector Spaces
  • Basis and Rank
  • Linear Independence
  • Norm
  • Inner Products and Dot Product
  • Lengths and Distances
  • Angles and Orthogonality
  • Orthonormal Basis
  • Feature Space
  • Cauchy–Schwarz
• Matrix Decompositions
  • Special Matrices
  • Characteristic Polynomial
  • Determinant and Trace
  • Eigenvalues and Eigenvectors
  • Cholesky Decomposition
  • Eigen Decomposition
  • Diagonalization
  • Singular Value Decomposition (SVD)
  • Matrix Approximation
• Dimensionality reduction and PCA
  • Principal Component Analysis (PCA)
  • PCA Theory
  • PCA in Practice
  • Latent Variable Perspective
  • Mathematical Preliminaries of SVM
  • Nonlinear SVM and Kernels

Why Linear Algebra Matters in ML #

• Every machine learning model uses matrices
• All data in ML is represented using vectors and matrices
• Neural networks are pipelines of matrix operations
• Models apply matrix transformations to data
• Optimisation relies on linear algebra operations

What to Learn #

• Scalars, vectors, and matrices
• Vector operations (addition, dot product)
• Matrix multiplication (critical)
• Identity matrices and transpose
• Eigenvalues and eigenvectors (conceptual understanding)

• Scalar → a number
• Vector → a directed point
• Matrix → a space transformer
• Linear transformation → structured mapping
• Feature → one axis
• Feature space → where data lives
• Vector space → where vectors live

Home | Mathematical Foundation