Distributed CNN

Clusters, Hadoop, and Spark

AI, ML
ML System Optimization, MLSysOps, Clusters, Hadoop, Spark, MapReduce, Distributed K-Means, Distributed CNN, AI, ML

Clusters, Hadoop, and Spark #

Scale-out frameworks distribute data and computation across networked machines. Their value comes from aggregate compute and storage, while their main costs are communication, scheduling, serialisation, and repeated synchronisation.

Course coverage:

  1. Cluster computing with Hadoop and Spark
  2. MapReduce k-means, mini-batch and streaming variants
  3. Convergence and communication reduction
  4. Distributed convolutional neural networks: data and model parallelism
  5. Layer or tensor slicing, mixed precision, pipelining, and distributed execution frameworks

Learning Objectives #

By the end of this page, you should be able to: