Optimisation of Deep models

Optimisation of Deep models #

• Goal of Optimization
• Optimization Challenges in Deep Learning
• Gradient Descent
• Stochastic Gradient Descent
• Minibatch Stochastic Gradient Descent
• Momentum
• Adagrad and Algorithm
• RMSProp and Algorithm
• Adadelta and Algorithm
• Adam and Algorithm
• Code Implementation and comparison of algorithms (webinar)

Reference #

• Dive into deep learning. Cambridge University Press.. (Ch12)

Home | Deep Learning

Evaluation/Comparison

Machine Learning Model Evaluation/Comparison #

Comparing Machine Learning Models #

Emerging requirements e.g., bias, fairness, interpretability of ML models #

Home | Machine Learning

Regularisation for Deep models

Regularisation for Deep models #

• Generalization for regression
• Training Error and Generalization Error
• Underfitting or Overfitting
• Model Selection
• Weight Decay and Norms
• Generalization in Classification
• Environment and Distribution Shift
• Generalization in Deep Learning
• Dropout
• Batch Normalization
• Layer Normalization
• Code implementation (webinar)

Reference #

• Dive into deep learning. Cambridge University Press.. (T1 – Ch 3.6, 3.7, T1 - Ch 4.6, 4.7, T1 - Ch 5.5, 5.6, T1 - Ch 8.5, T1 - Ch 11.7

Home | Deep Learning