K-Means

Parallelisation of ML Algorithms

AI, ML
ML System Optimization, MLSysOps, K-Means, Decision Trees, Random Forest, SVM, Parallel Machine Learning, AI, ML

Parallelisation of ML Algorithms #

Machine-learning algorithms expose different kinds of parallel work. The correct decomposition depends on whether independent work occurs across records, features, trees, clusters, kernel entries, or optimisation steps.

Course coverage:

  1. Problem decomposition for parallel machine learning
  2. Ensemble methods and XGBoost-style tree ensembles
  3. Parallel k-means assignment and centroid reduction
  4. Distributed decision trees and random forests
  5. Support vector machine parallelisation using block and MapReduce-style computation

Learning Objectives #

By the end of this page, you should be able to:

Communication-Aware Distributed ML

AI, ML
ML System Optimization, MLSysOps, Distributed Machine Learning, K-Means, KNN, Gradient Descent, SGD, Communication, AI, ML

Communication-Aware Distributed ML #

Distributed machine learning is effective only when saved computation exceeds the cost of moving data and synchronising workers. Algorithm design must therefore account for message size, message frequency, barriers, and parameter placement.

Course coverage:

  1. Communication overhead, illustrated through distributed k-means
  2. Model parallelism when parameters do not fit or compute must be divided
  3. Distributed k-nearest neighbours: partitioning, indexing, and approximate search
  4. Gradient descent, SGD, and mini-batch optimisation
  5. Synchronous versus asynchronous updates

Learning Objectives #

By the end of this page, you should be able to:

Unsupervised Learning

AI, ML
AI, ML, Unsupervised Learning, K-Means, EM, GMM, Clustering

Unsupervised Learning #

Unsupervised Learning is used when we have input data but no target labels.

The model is not told the correct answer. Instead, it tries to discover hidden structure in the data.

  • K-means Clustering and variants
  • Review of EM algorithm
  • GMM based Soft Clustering
  • Applications

Supervised vs Unsupervised Learning #

AspectSupervised LearningUnsupervised Learning
Data contains target label?YesNo
Learns fromInput-output pairsInput features only
Main goalPredict outputDiscover structure
Example taskClassification, regressionClustering
Example algorithmLogistic regression, decision treeK-means, GMM
  • Works on unlabelled raw data.
  • The algorithm discovers hidden patterns without prior knowledge of outcomes.
  • Requires no human intervention during training.
  • Does not make direct predictions — it groups or organises data instead.
  • Carries a higher risk because there’s no ground truth to verify results.
  • Common techniques include Clustering, Association, and Dimensionality Reduction.

The most common example is clustering, where similar records are grouped together.