RL is learning by trial and error.
Reinforcement Learning (RL) is a type of machine learning where an autonomous agent learns to make decisions by interacting with an environment.
Instead of being told the correct answer, the agent:
Reinforcement Learning teaches an agent how to act, not what to predict.
Deep Reinforcement Learning (DRL) studies how an agent learns to make a sequence of decisions by interacting with an environment and receiving feedback through rewards.
Reinforcement learning provides the framework for sequential decision-making. Deep learning extends this framework with powerful function approximators that can handle large or complex state and action spaces.
Deep Reinforcement Learning = Reinforcement Learning + Deep Neural Networks
Classical reinforcement learning foundations → progresses towards → value-based deep learning → policy-gradient methods → model-based approaches → imitation learning.
Reinforcement Learning (RL) is a learning approach in which an agent improves its behaviour by interacting with an environment and observing the rewards produced by its actions.
Unlike supervised learning, the agent is not given a correct action label for every situation. It must discover useful behaviour through trial, feedback, and repeated interaction.
Reinforcement learning is goal-oriented learning through interaction.RL is useful when decisions affect what happens next and the quality of an action may only become clear later.
The Multi-Armed Bandit (MAB) problem is the simplest setting for studying decision-making under uncertainty.
An agent repeatedly chooses one of \( k \) actions. Each action produces a numerical reward drawn from an unknown distribution. The objective is to maximise the expected total reward over time.
The central challenge is deciding when to exploit current knowledge and when to explore uncertain alternatives.At every time step:
A Markov Decision Process (MDP) is the mathematical framework used to describe sequential decision-making in reinforcement learning. It brings together the agent, environment, states, actions, transition dynamics, rewards, returns, policies, and value functions in one model.
A basic bandit asks which action is best in a recurring situation. An MDP goes further: the action taken now can change the next state, which changes the decisions and rewards that become possible later.
Dynamic Programming (DP) solves a reinforcement learning problem by repeatedly applying Bellman equations. It uses a complete model of the environment to calculate value functions and improve a policy.
Dynamic Programming turns a known model of an MDP into a policy for choosing actions.DP assumes that the environment dynamics are known. For every state and action, the agent needs the probabilities of possible next states and rewards:
Monte Carlo (MC) methods learn value functions and policies by averaging returns observed in complete episodes. Unlike Dynamic Programming, they do not require transition probabilities or a complete model of the environment.
Monte Carlo learning replaces model-based expectation with averages from sampled experience.Suppose the agent repeatedly reaches a state \( s \) . Each time, it records the return received from that point until the episode ends. The average of these returns estimates the state’s value.
Temporal-Difference (TD) learning updates predictions from one transition at a time. It learns directly from experience like Monte Carlo methods, but it does not need to wait until the episode ends.
Course Content covered in this module:
By the end of this module, you should be able to:
One-step TD methods bootstrap after a single transition. Multi-step methods instead use several observed rewards before bootstrapping, forming a bridge between TD(0) and Monte Carlo learning.
Course Content covered in this module:
By the end of this module, you should be able to:
Tabular methods store a separate value for every state or state-action pair. This becomes impractical when the state space is very large, continuous or represented by raw observations. Function approximation replaces this table with a parameterised function that can learn from observed states, generalise to related states and predict values for states not seen during training.
Course Content covered in this module:
By the end of this module, you should be able to: