Support Vector Machine

Support Vector Machine (SVM) #

A Support Vector Machine (SVM) is a supervised machine learning algorithm used for:

• Classification (most common)
• Regression (SVR – Support Vector Regression)

Find the decision boundary that separates classes with the maximum margin.

A Support Vector Machine is a supervised learning algorithm that finds an optimal hyperplane by maximising the margin between classes, using support vectors and kernel functions to handle non-linear data.

Attention Mechanism

Attention Mechanism #

• Queries, Keys, and Values
• Attention Pooling by Similarity
• Attention Pooling via Nadaraya–Watson Regression
• Attention Scoring Functions
• Dot Product Attention
• Convenience Functions
• Scaled Dot Product Attention
• Additive Attention
• Bahdanau Attention Mechanism
• Multi-Head Attention
• Self-Attention
• Positional Encoding
• Code implementation (webinar)

Reference #

• Dive into deep learning. Cambridge University Press.. (Ch 10, Ch7

Home | Deep Learning

Bayesian Learning

Bayesian Learning #

MLE Hypothesis #

MAP Hypothesis #

Bayes Rule #

Optimal Bayes Classifier #

Naïve Bayes Classifier #

Probabilistic Generative Classifiers #

Bayesian Linear Regression #

Home | Machine Learning

Transformer

December 15, 2025

Transformer #

• is an architecture of neural networks

• based on the multi-head attention mechanism

• text is converted to numerical representations called tokens, and each token is converted into a vector via lookup from a word embedding table

• takes a text sequence as input and produces another text sequence as output

• foundation for modern Large Language Models (LLMs) like ChatGPT and Gemini

• Transformer architecture

• Model, Positionwise Feed-Forward Networks, Residual Connection and Layer Normalization

Ensemble Learning

Ensemble Learning #

Combining Classifiers #

Bagging #

Random Forest #

Boosting #

ADABoost #

Gradient Boosting #

XGBoost #

Home | Machine Learning

Optimisation of Deep models

Optimisation of Deep models #

• Goal of Optimization
• Optimization Challenges in Deep Learning
• Gradient Descent
• Stochastic Gradient Descent
• Minibatch Stochastic Gradient Descent
• Momentum
• Adagrad and Algorithm
• RMSProp and Algorithm
• Adadelta and Algorithm
• Adam and Algorithm
• Code Implementation and comparison of algorithms (webinar)

Reference #

• Dive into deep learning. Cambridge University Press.. (Ch12)

Home | Deep Learning

Evaluation/Comparison

Machine Learning Model Evaluation/Comparison #

Comparing Machine Learning Models #

Emerging requirements e.g., bias, fairness, interpretability of ML models #

Home | Machine Learning

Regularisation for Deep models

Regularisation for Deep models #

• Generalization for regression
• Training Error and Generalization Error
• Underfitting or Overfitting
• Model Selection
• Weight Decay and Norms
• Generalization in Classification
• Environment and Distribution Shift
• Generalization in Deep Learning
• Dropout
• Batch Normalization
• Layer Normalization
• Code implementation (webinar)

Reference #

• Dive into deep learning. Cambridge University Press.. (T1 – Ch 3.6, 3.7, T1 - Ch 4.6, 4.7, T1 - Ch 5.5, 5.6, T1 - Ch 8.5, T1 - Ch 11.7

Home | Deep Learning

AI Learning Resources

January 3, 2026

AI Learning Resources #

A curated list of high-quality online courses to learn Artificial Intelligence, Machine Learning, and Deep Learning from reputable universities and organisations.

Recommended Books & References #

Deep Neural Networks (DNN) #

• Deep Learning. MIT Press.
  Goodfellow, I., Bengio, Y., & Courville, A. (2016). (Vol. 1, No. 2).

• Introduction to Deep Learning. MIT Press.
  Eugene, C. (2019).

• Deep Learning with Python. Simon & Schuster.
  Chollet, F. (2021).