M1: Foundations on Arshad Siddiqui

ML and DL System Performance

Mon, 01 Jan 0001 00:00:00 +0000

ML and DL System Performance #

Machine learning system optimisation begins with measurement. Before changing an algorithm, adding processors, or moving work to a GPU, we need to understand what is slow, which resource is limiting performance, and how performance changes as the workload grows.

Course coverage:

Metrics: time complexity, running time, space, throughput, latency, and response time
Performance scaling and tuning: measuring bottlenecks and predicting how a workload grows
Training versus deployment: different objectives, workloads, and constraints
Deployment environments: distributed and cloud systems, embedded devices, and mobile systems

Learning Objectives #

By the end of this page, you should be able to:

Parallel and Distributed Algorithms

Mon, 01 Jan 0001 00:00:00 +0000

Parallel and Distributed Algorithms #

Parallelisation divides computational work into parts that can execute concurrently. The purpose is to reduce completion time or increase throughput, but the gain depends on how much work is genuinely independent and how much overhead is introduced.

Course coverage:

Systems and performance: execution time, throughput, efficiency, and bottlenecks
Speedup—approaches and issues: Amdahl’s Law, serial work, overhead, and load imbalance
Data parallelism versus task parallelism versus request parallelism
Scale-out clusters: communication cost and its impact on speedup

The worked algorithms also show how divide-and-conquer and matrix operations expose parallel work.

Parallel Programming Models

Mon, 01 Jan 0001 00:00:00 +0000

Parallel Programming Models #

Parallel algorithms need hardware that can execute independent work efficiently. Modern systems therefore combine multiple CPU cores, memory hierarchies, threads, instruction pipelines, GPUs, clusters, and specialised matrix processors.

Course coverage:

Parallel programming models for expressing independent work
MapReduce, task-parallel, and request-parallel patterns
Multi-core CPUs and GPGPUs: SIMD, MIMD, SIMT, memory hierarchy, and bandwidth
Distributed clusters: CPU-only and GPU-accelerated systems, data sharding, parameter servers, and all-reduce

The hardware sections also cover specialised matrix processors and the performance limits introduced by memory and communication.