Value-Based DRL: Function Approximation #

Tabular methods store a separate value for every state or state-action pair. This becomes impractical when the state space is very large, continuous or represented by raw observations. Function approximation replaces this table with a parameterised function that can learn from observed states, generalise to related states and predict values for states not seen during training.

Course Content covered in this module:

Function approximation
Feature Construction for Linear Methods:
- Tile Coding
- Asymmetric Tile Coding
Linear function approximation
Semi-Gradient TD methods
Off-policy function approximation and TD divergence
TD-Gammon

Instead of memorising one value per state, the agent learns parameters shared across many states.

Learning Objectives #

By the end of this module, you should be able to: