flowchart TD
A[Words and Documents] --> B[Observe Their Context]
B --> C[Represent Them as Vectors]
C --> D[Compare Vector Directions]
D --> E[Estimate Semantic Similarity]
E --> F[Search, Classify, Retrieve or Generate]
style A fill:#E1F5FE
style B fill:#C8E6C9
style C fill:#FFF9C4
style D fill:#EDE7F6
style E fill:#E1F5FE
style F fill:#C8E6C9
Probability distributions are the bridge between:
real-world randomness and mathematical modelling.
A random experiment produces outcomes.
A random variable turns those outcomes into numbers.
A probability distribution tells you how likely each number (or range of numbers) is.
Key takeaway:
A distribution is a complete “story” about uncertainty:
what values are possible, how likely they are, and how we summarise them (mean, variance).
Many ML models are probabilistic:
they assume data (or errors) follow a distribution.
Loss functions often come from distribution assumptions:
squared loss aligns with Gaussian noise.
Naïve Bayes (from the previous module) becomes practical once you can model:
\( P(X\mid Y) \)
using suitable distributions.
In practice:
choosing a distribution is a modelling decision.
It affects:
prediction, uncertainty estimates, and what “rare” or “typical” means in your data.
The AI pipeline is a continuous process where data is collected, prepared, used to train models, evaluated for performance, and continuously improved after deployment.
timeline
title AI Pipeline
Collect Data : Data Ingestion
: Data Understanding
Prepare Data : Cleaning
: Feature Engineering
: Sampling
Train Model : Model Training
: Validation & Metrics
Deploy Model : Deployment
: Monitoring & Retraining
This module examines the platforms, communication patterns, and hardware organisation used to train across larger systems. It extends distributed algorithms into coordinated training services and federated settings.
Module coverage:
ML Platforms and Frameworks: the Parameter Server model, Spark MLlib, and TensorFlow/PyTorch distributed training.
Distributed Deep Learning: decentralised SGD, all-reduce, asynchronous parallelism, and large-scale neural networks.
Federated Learning: training that preserves data privacy, communication efficiency, and hardware utilisation.
Page available; parameter-server synchronisation, compression, overlap, and placement
10
Locality and Large-Scale Parallelism
Planned
11
GPU Architecture and Federated Learning
Planned
The outline below describes the full module scope. Planned topics have no chapter page yet. Chapter 9 currently covers parameter-server execution policies and communication optimisation; collective communication, matrix multiplication, and Local SGD remain to be added from their source material.
A Markov Decision Process (MDP) is the mathematical framework used to describe sequential decision-making in reinforcement learning. It brings together the agent, environment, states, actions, transition dynamics, rewards, returns, policies, and value functions in one model.
A basic bandit asks which action is best in a recurring situation. An MDP goes further: the action taken now can change the next state, which changes the decisions and rewards that become possible later.
Parallel algorithms need hardware that can execute independent work efficiently. Modern systems therefore combine multiple CPU cores, memory hierarchies, threads, instruction pipelines, GPUs, clusters, and specialised matrix processors.
Course coverage:
Parallel programming models for expressing independent work
MapReduce, task-parallel, and request-parallel patterns
Multi-core CPUs and GPGPUs: SIMD, MIMD, SIMT, memory hierarchy, and bandwidth
Distributed clusters: CPU-only and GPU-accelerated systems, data sharding, parameter servers, and all-reduce
The hardware sections also cover specialised matrix processors and the performance limits introduced by memory and communication.
A problem-solving agent converts a goal into a precisely defined search problem. An uninformed search algorithm then explores the state space using only the information in that problem definition.
flowchart TD
G["Goal"] --> F["Formulate problem"]
F --> S["Search for actions"]
S --> X["Execute solution"]
style G fill:#E1F5FE
style F fill:#C8E6C9
style S fill:#FFF9C4
style X fill:#EDE7F6
