Reinforcement Learning

Reinforcement Learning

Reinforcement Learning (RL) #

RL is learning by trial and error.

Reinforcement Learning (RL) is a type of machine learning where an autonomous agent learns to make decisions by interacting with an environment.

Instead of being told the correct answer, the agent:

  • takes actions
  • observes outcomes
  • receives rewards or penalties
  • gradually learns a strategy that maximises long-term reward

Reinforcement Learning teaches an agent how to act, not what to predict.

Deep Reinforcement Learning

Deep Reinforcement Learning #

Deep Reinforcement Learning (DRL) studies how an agent learns to make a sequence of decisions by interacting with an environment and receiving feedback through rewards.

Reinforcement learning provides the framework for sequential decision-making. Deep learning extends this framework with powerful function approximators that can handle large or complex state and action spaces.

Deep Reinforcement Learning = Reinforcement Learning + Deep Neural Networks

The learning path begins with classical reinforcement learning foundations and progresses towards value-based deep learning, policy-gradient methods, model-based approaches, and imitation learning.

Reinforcement Learning

Reinforcement Learning #

Reinforcement Learning (RL) is a learning approach in which an agent improves its behaviour by interacting with an environment and observing the rewards produced by its actions.

Unlike supervised learning, the agent is not given a correct action label for every situation. It must discover useful behaviour through trial, feedback, and repeated interaction.

Reinforcement learning is goal-oriented learning through interaction.

Why Reinforcement Learning? #

RL is useful when decisions affect what happens next and the quality of an action may only become clear later.

Multi-Armed Bandit Problem

Multi-Armed Bandit Problem #

The Multi-Armed Bandit (MAB) problem is the simplest setting for studying decision-making under uncertainty.

An agent repeatedly chooses one of \( k \) actions. Each action produces a numerical reward drawn from an unknown distribution. The objective is to maximise the expected total reward over time.

The central challenge is deciding when to exploit current knowledge and when to explore uncertain alternatives.

The k-Armed Bandit Problem ☆ #

At every time step:

Markov Decision Process Framework

Markov Decision Process Framework #

A Markov Decision Process (MDP) is a mathematical framework for modelling sequential decisions. It describes the situations an agent may encounter, the actions it may take, how the environment may change, and the rewards produced by those changes.

Bandit problems ask which action is best in a single recurring situation. An MDP adds changing states: an action affects not only the immediate reward but also the situation faced next.

Rewards, Returns, Policies and Value Functions

Rewards, Returns, Policies and Value Functions #

An MDP describes how states, actions, rewards and transitions fit together. The next task is to evaluate behaviour: what should the agent try to achieve, how should future rewards be counted, and how good is a state or action over the long term?

Rewards define the objective, returns combine rewards across time, a policy describes behaviour, and value functions predict the long-term quality of that behaviour.