Monte Carlo Methods
Monte Carlo Methods #
Monte Carlo (MC) methods learn value functions and policies by averaging returns observed in complete episodes. Unlike Dynamic Programming, they do not require transition probabilities or a complete model of the environment.
Monte Carlo learning replaces model-based expectation with averages from sampled experience.1. The Main Idea ☆ #
Suppose the agent repeatedly reaches a state \( s \) . Each time, it records the return received from that point until the episode ends. The average of these returns estimates the state’s value.