Temporal-Difference Learning I
#
Temporal-Difference (TD) learning updates predictions from one transition at a time. It learns directly from experience like Monte Carlo methods, but it does not need to wait until the episode ends.
Course Content covered in this module:
- Temporal-Difference Learning
- TD(0), SARSA, Q-Learning and Expected SARSA
TD learning updates an estimate using an immediate reward and another current estimate.
Learning Objectives
#
By the end of this module, you should be able to:
Temporal-Difference Learning II and DRL Taxonomy
#
One-step TD methods bootstrap after a single transition. Multi-step methods instead use several observed rewards before bootstrapping, forming a bridge between TD(0) and Monte Carlo learning.
Course Content covered in this module:
- Temporal-Difference Learning: n-step returns and TD(λ)
- Classification of Reinforcement Learning approaches, algorithms and applications:
- Model-Based versus Model-Free
- Value-Based versus Policy-Based
- On-Policy versus Off-Policy
The number of sampled steps controls how far learning looks ahead before it bootstraps.
Learning Objectives
#
By the end of this module, you should be able to: