Multi-Armed Bandits

Multi-Armed Bandit Problem

AI, ML
AI, ML, Reinforcement Learning, Multi-Armed Bandits

Multi-Armed Bandit Problem #

The Multi-Armed Bandit (MAB) problem is the simplest setting for studying decision-making under uncertainty.

An agent repeatedly chooses one of \( k \) actions. Each action produces a numerical reward drawn from an unknown distribution. The objective is to maximise the expected total reward over time.

The central challenge is deciding when to exploit current knowledge and when to explore uncertain alternatives.
  • Multi-armed Bandit Problem - Motivation and Problem Statement
  • Incremental solution to the stationary & non-stationary MAB problems
  • Exploration vs. Exploitation trade-off
  • Bandit Gradient Algorithm as Stochastic Gradient Ascent
  • Associative Search

The k-Armed Bandit Problem ☆ #

At every time step: