Supervised Learning

January 3, 2026

Supervised Learning #

Trained using labelled data.
Each example in the training set includes the correct output.
The algorithm learns to generalise and make predictions on unseen data.
Generally more accurate than unsupervised methods.
Requires human intervention for labelling and setup.
Widely used due to its accuracy and efficiency.
Produces highly accurate results when trained on good-quality labelled data.

Classification #

Output is discrete (e.g. Yes/No, Spam/Not Spam).
Used for categorising data into predefined classes.
Support Vector Machine (SVM) is a common classifier (a linear classifier with margin-based separation).

Semi-Supervised Learning

January 3, 2026

Semi-Supervised Learning #

• A combination of labelled and unlabelled data.
• Useful when labelling large datasets is expensive or time-consuming.
• Works well with high-volume datasets (e.g. millions of images).
• Only a small fraction of data is labelled (e.g. a few thousand).
• The algorithm learns from both labelled examples and structure in unlabelled data.
• Ideal for medical imaging where labelled data is limited.
• For example, a radiologist can label a small set of medical scans,
  and the model uses that to learn from thousands of unlabelled scans.
• Helps improve accuracy and generalisation with minimal manual effort.

Home | Machine Learning

Machine Learning

August 6, 2024

Machine Learning #

stateDiagram-v2

    %% ===== CLASS DEFINITIONS (Math-based colours) =====
    classDef algebra fill:#cfe8ff,stroke:#1e3a8a,stroke-width:1px
    classDef probability fill:#d1fae5,stroke:#065f46,stroke-width:1px
    classDef geometry fill:#ffedd5,stroke:#9a3412,stroke-width:1px
    classDef logic fill:#ede9fe,stroke:#5b21b6,stroke-width:1px
    classDef category font-style:italic,font-weight:bold,fill:#aaaaaa,stroke:#374151,stroke-width:3px

    %% ===== ROOT =====
    ML: Machine Learning

    %% ===== SUPERVISED =====
    ML --> SL:::category
    SL: Supervised Learning

    SL --> Regression
    Regression --> LR:::algebra
    LR: Linear Regression

    LR --> NN:::algebra
    NN: Neural Network

    NN --> DT:::logic
    DT: Decision Tree

    SL --> Classification
    Classification --> NB:::probability
    NB: Naive Bayes

    NB --> KNN:::geometry
    KNN: k-Nearest Neighbours

    KNN --> SVM:::algebra
    SVM: Support Vector Machine
    
    %% ===== UNSUPERVISED =====
    ML --> USL:::category
    USL: Unsupervised Learning

    USL --> Clustering
    Clustering --> KM:::geometry
    KM: K-Means

    KM --> GMM:::probability
    GMM: Gaussian Mixture Model

    GMM --> HMM:::probability
    HMM: Hidden Markov Model

    %% ===== REINFORCEMENT =====
    ML --> RL:::category
    RL: Reinforcement Learning

    RL --> DM:::logic
    DM: Decision Making

Deep Reinforcement Learning

Deep Reinforcement Learning #

Deep Reinforcement Learning (DRL) studies how an agent learns to make a sequence of decisions by interacting with an environment and receiving feedback through rewards.

Reinforcement learning provides the framework for sequential decision-making. Deep learning extends this framework with powerful function approximators that can handle large or complex state and action spaces.

Deep Reinforcement Learning = Reinforcement Learning + Deep Neural Networks

Classical reinforcement learning foundations → progresses towards → value-based deep learning → policy-gradient methods → model-based approaches → imitation learning.

Natural Language Processing

Natural Language Processing #

Natural Language Processing (NLP) studies how computers can analyse, understand, represent, and generate human language.

It combines ideas from linguistics, computer science, machine learning, and deep learning to work with text and language-based information.

Natural Language Processing = Linguistics + Computation + Machine Learning

The learning path begins with language understanding and vector representations, progresses through language modelling, tagging, and parsing, and then moves towards transformers, knowledge graphs, Retrieval-Augmented Generation, and modern NLP applications.

M1: Foundations

M1: Foundations #

This module establishes how to measure ML system performance, identify bottlenecks, and decide which work can run in parallel. It connects algorithmic complexity with the capabilities and limits of CPUs, GPUs, and clusters.

Module coverage:

• Performance Metrics and Complexity Analysis: time and space complexity, throughput, latency, and speedup through Amdahl’s Law.
• Parallelisation Paradigms: data-level, task-level, and algorithm-specific parallelism.
• Modern Hardware Architectures: multi-core CPUs, GPGPUs, GPU/CPU clusters, and resource management.

Learning Objectives #

• Distinguish complexity from measured running time, throughput, and latency.
• Explain the limits of speedup and the cost of communication.
• Choose between data, task, and request parallelism for a workload.
• Relate memory hierarchy, bandwidth, and hardware organisation to performance.

Pages #

Topic Outline #

1. ML and DL System Performance #

• Time complexity of algorithms and running time
• Performance scaling and tuning
• Training versus deployment
• Distributed and cloud environments, embedded devices, and mobile systems
• Throughput and response time

2. Parallel and Distributed Algorithms #

• Systems and performance
• Speedup: approaches, issues, and Amdahl’s Law ☆
• Data parallelism, task parallelism, and request parallelism
• Scale-out clusters: communication cost and its impact on speedup
• Scale-up versus scale-out

3. Parallel Programming Models #

• Parallel programming models and the MapReduce pattern
• Task-parallel and request-parallel execution
• Multi-core and GPGPU parallelism: SIMD versus MIMD
• Memory hierarchies and bandwidth considerations
• CPU-only and GPU-accelerated distributed clusters
• Horizontal and vertical data sharding
• Parameter-server and all-reduce paradigms

How This Module Connects #

Performance measurement and parallel execution provide the basis for M2: Distributed ML Algorithms. Communication and memory limits remain relevant throughout the later modules.

ML and DL System Performance

ML and DL System Performance #

Machine learning system optimisation begins with measurement. Before changing an algorithm, adding processors, or moving work to a GPU, we need to understand what is slow, which resource is limiting performance, and how performance changes as the workload grows.

Course coverage:

1. Metrics: time complexity, running time, space, throughput, latency, and response time
2. Performance scaling and tuning: measuring bottlenecks and predicting how a workload grows
3. Training versus deployment: different objectives, workloads, and constraints
4. Deployment environments: distributed and cloud systems, embedded devices, and mobile systems

Learning Objectives #

By the end of this page, you should be able to:

Reinforcement Learning

Reinforcement Learning #

Reinforcement Learning (RL) is a learning approach in which an agent improves its behaviour by interacting with an environment and observing the rewards produced by its actions.

Unlike supervised learning, the agent is not given a correct action label for every situation. It must discover useful behaviour through trial, feedback, and repeated interaction.

Why Reinforcement Learning? #

RL is useful when decisions affect what happens next and the quality of an action may only become clear later.

Neural Networks

Neural Networks #

• A network of artificial neurons inspired by how neurons function in the human brain.
• At its core - a mathematical model designed to process and learn from data.
• Neural networks form the foundation of Deep Learning (involves training large and complex networks on vast amounts of data).

flowchart LR
 subgraph subGraph0["Input Layer"]
        I1(("Input 1"))
        I2(("Input 2"))
        I3(("Input 3"))
  end
 subgraph subGraph1["Hidden Layer"]
        H1(("Hidden 1"))
        H2(("Hidden 2"))
        H3(("Hidden 3"))
  end
 subgraph subGraph2["Output Layer"]
        O(("Output"))
  end
    I1 --> H1 & H2 & H3
    I2 --> H1 & H2 & H3
    I3 --> H1 & H2 & H3
    H1 --> O
    H2 --> O
    H3 --> O

    style I1 fill:#C8E6C9
    style I2 fill:#C8E6C9
    style I3 fill:#C8E6C9
    style H1 stroke:#2962FF,fill:#BBDEFB
    style H2 fill:#BBDEFB
    style H3 fill:#BBDEFB
    style O fill:#FFCDD2
    style subGraph0 stroke:none,fill:transparent
    style subGraph1 stroke:none,fill:transparent
    style subGraph2 stroke:none,fill:transparent

Structure of a Neural Network #

A typical neural network has three main layers:

Artificial Neuron and Perceptron

Artificial Neuron and Perceptron #

knowledge in neural networks is stored in connection weights, and learning means modifying those weights.

Biological Neuron #

A biological neuron is a specialised cell that processes and transmits information through electrical and chemical signals.

Core components:

• Dendrites: receive signals from other neurons
• Cell body (soma): processes incoming signals
• Axon: transmits the output signal
• Synapses: connection points between neurons

Biological intuition:

• many inputs arrive to one neuron
• one neuron can connect out to many neurons
• massive parallelism enables fast perception and recognition

Artificial Neuron #

An artificial neuron is a simplified computational model inspired by biological neurons.