ML

Multi-Armed Bandit Problem #

The Multi-Armed Bandit (MAB) problem is the simplest setting for studying decision-making under uncertainty.

An agent repeatedly chooses one of $ k $ actions. Each action produces a numerical reward drawn from an unknown distribution. The objective is to maximise the expected total reward over time.

The central challenge is deciding when to exploit current knowledge and when to explore uncertain alternatives.

The k-Armed Bandit Problem ☆ #

At every time step:

Parallel and Distributed Algorithms

AI, ML

ML System Optimization, MLSysOps, Parallel Computing, Amdahl's Law, Data Parallelism, Task Parallelism, AI, ML

Parallel and Distributed Algorithms #

Parallelisation divides computational work into parts that can execute concurrently. The purpose is to reduce completion time or increase throughput, but the gain depends on how much work is genuinely independent and how much overhead is introduced.

This page covers:

speedup, maximum speedup, and processor efficiency
Amdahl’s Law
data-level parallelism
task-level parallelism
algorithm-specific parallelism
communication, synchronisation, scheduling, and load-balancing overhead
parallel merge sort and matrix multiplication

Learning Objectives #

By the end of this page, you should be able to:

Markov Decision Process Framework

AI, ML

AI, ML, Reinforcement Learning, Markov Decision Process

Markov Decision Process Framework #

A Markov Decision Process (MDP) is a mathematical framework for modelling sequential decisions. It describes the situations an agent may encounter, the actions it may take, how the environment may change, and the rewards produced by those changes.

Bandit problems ask which action is best in a single recurring situation. An MDP adds changing states: an action affects not only the immediate reward but also the situation faced next.

Parallel Programming Models

AI, ML

ML System Optimization, MLSysOps, Multi-Core CPU, GPGPU, SIMD, MIMD, SIMT, GPU, TPU, AI, ML

Parallel Programming Models #

Parallel algorithms need hardware that can execute independent work efficiently. Modern systems therefore combine multiple CPU cores, memory hierarchies, threads, instruction pipelines, GPUs, clusters, and specialised matrix processors.

This page covers:

multi-core CPU organisation
cache and memory hierarchy
processes, threads, scheduling, and synchronisation
instruction pipelining and clock-cycle time
SIMD, MIMD, and SIMT execution
GPGPU architecture and GPU memory behaviour
CPU-only and GPU-accelerated clusters
Tensor Processing Units and systolic arrays

Learning Objectives #

By the end of this page, you should be able to:

Regression (Linear)

AI, ML

Linear Regression #

Linear Regression is a supervised ML method used to predict a numerical target by fitting a model that is linear in its parameters.

In ML , linear models are a core baseline: they’re fast, often surprisingly strong, and usually easy to interpret.

Key takeaway: Linear Regression learns parameters by minimising a squared-error cost. You can solve it directly (closed form) or iteratively (gradient descent), and you can extend it using basis functions and regularisation.

Ordinary Least Squares

February 21, 2026

AI, ML

AI, ML, Linear Regression, OLS

Direct solution method - Ordinary Least Squares and the Line of Best Fit #

Revision:
OLS is the direct method for linear regression. It finds the best-fit line by minimising the sum of squared residuals without iterative updates.

Direct Method vs Iterative Method ☆ #

Linear regression parameters can be found in two main ways.

Method	Main idea	When used
Ordinary Least Squares	Compute the best parameters directly	Small or moderate datasets
Gradient Descent	Start with parameters and update repeatedly	Large datasets or many features

flowchart LR
    A["Linear Regression"] --> B["Direct Solution<br/>OLS"]
    A --> C["Iterative Solution<br/>Gradient Descent"]

    B --> B1["Normal Equation"]
    B --> B2["No learning rate"]
    B --> B3["One-shot solution"]

    C --> C1["Learning rate"]
    C --> C2["Repeated updates"]
    C --> C3["Stops after convergence"]

    style A fill:#E1F5FE,stroke:#5b7db1,color:#000
    style B fill:#C8E6C9,stroke:#5f8f6a,color:#000
    style C fill:#FFF9C4,stroke:#b59b3b,color:#000
    style B1 fill:#EDE7F6,stroke:#8a6fb3,color:#000
    style B2 fill:#EDE7F6,stroke:#8a6fb3,color:#000
    style B3 fill:#EDE7F6,stroke:#8a6fb3,color:#000
    style C1 fill:#EDE7F6,stroke:#8a6fb3,color:#000
    style C2 fill:#EDE7F6,stroke:#8a6fb3,color:#000
    style C3 fill:#EDE7F6,stroke:#8a6fb3,color:#000

Why It Is Called “Least Squares” ☆ #

OLS is called least squares because it chooses parameters that make the squared residual errors as small as possible.

Cost Function

February 21, 2026

AI, ML

AI, ML, Linear Regression, OLS

Cost Function #

Revision:
A cost function converts model error into a single number. Training means changing the model parameters until this number becomes as small as possible.

Why Cost Function Matters in ML ☆ #

A machine learning model needs a way to decide whether one set of parameters is better than another.

For linear regression, every possible value of the parameters gives a different line. The cost function tells us which line is better by measuring how far the predictions are from the true values.

Gradient Descent

February 21, 2026

AI, ML

AI, ML, Linear Regression, Gradient Descent

Gradient Descent for Linear Regression #

Revision:
Gradient descent is the step-by-step method for reducing the cost function when a direct closed-form solution is not convenient.

Where Gradient Descent Fits in ML ☆ #

Gradient descent is used when we want the model to learn parameters by repeatedly improving them.

For linear regression, it adjusts the slope and intercept until the prediction error becomes small.

flowchart LR
    A["Initial Parameters"] --> B["Make Predictions"]
    B --> C["Compute Cost"]
    C --> D["Compute Gradient"]
    D --> E["Update Parameters"]
    E --> B

    style A fill:#E1F5FE,stroke:#5b7db1,color:#000
    style B fill:#C8E6C9,stroke:#5f8f6a,color:#000
    style C fill:#FFF9C4,stroke:#b59b3b,color:#000
    style D fill:#EDE7F6,stroke:#8a6fb3,color:#000
    style E fill:#C8E6C9,stroke:#5f8f6a,color:#000

Core Idea ☆ #

The gradient tells us the direction in which the cost increases fastest.

Classification (Linear)

AI, ML

Linear models for Classification #

categorises data by finding a linear boundary (hyperplane) that separates classes
calculating a weighted sum of input features plus bias

flowchart TD
T["Linear<br/>classification<br/>models"] --> P["Perceptron"]
T --> LR["Logistic<br/>regression"]
T --> SVM["Linear<br/>SVM"]

P -->|uses| STEP["Step<br/>activation"]
LR -->|uses| SIG["Sigmoid<br/>+ log loss"]
SVM -->|uses| HNG["Hinge<br/>loss"]

style T fill:#90CAF9,stroke:#1E88E5,color:#000

style P fill:#C8E6C9,stroke:#2E7D32,color:#000
style LR fill:#C8E6C9,stroke:#2E7D32,color:#000
style SVM fill:#C8E6C9,stroke:#2E7D32,color:#000

style STEP fill:#CE93D8,stroke:#8E24AA,color:#000
style SIG fill:#CE93D8,stroke:#8E24AA,color:#000
style HNG fill:#CE93D8,stroke:#8E24AA,color:#000

Discriminant Functions
Decision Theory
Probabilistic Discriminative Classifiers
Logistic Regression

Logistic Regression #

Supervised machine learning algorithm
Binary classification algorithm
requires data to be linearly separable
predicts the probability that an input belongs to a specific class
uses Sigmoid function to convert inputs into a probability value between 0 and 1

Key takeaway: Logistic regression predicts $P(y=1\mid x)$ using a sigmoid of a linear score $z=w\cdot x+b$, then learns $w,b$ by maximising likelihood (equivalently minimising log-loss).

Hypothesis Testing

AI, ML, Stats

AI, ML, Stats, Hypothesis Testing, ANOVA, Maximum Likelihood

Hypothesis Testing #

Hypothesis testing is a statistical decision-making method used to decide whether sample evidence is strong enough to reject an initial assumption about a population.

It connects probability, sampling distributions, confidence intervals, significance levels, and decision rules.

Key takeaway:
Hypothesis testing is not about proving something with certainty.
It is about asking:
If the null hypothesis were true, how surprising would this sample result be?