Plain-English companion pages for Richard S. Sutton & Andrew G. Barto — Reinforcement Learning: An Introduction, 2nd edition (MIT Press). These notes follow the mid-semester CS1–CS7 arc: what the RL problem is, how a single action's value is learned in bandits, how sequential decisions are modelled as MDPs, how a known MDP is solved by dynamic programming, and how an unknown one is learned from experience by Monte Carlo methods.
What RL is, trial-and-error and delayed reward, the four elements (policy, reward, value, model), exploration vs exploitation, and the tic-tac-toe example.
chapter 2Action values, sample-average and incremental estimates, \(\varepsilon\)-greedy, the 10-armed testbed, nonstationary step sizes, optimistic initial values, UCB, and gradient bandits.
chapter 3The agent–environment interface, the dynamics function, the reward hypothesis, returns and episodes, policies, value functions, and the Bellman expectation and optimality equations.
chapter 4Policy evaluation, policy improvement, policy iteration, value iteration, asynchronous DP, generalized policy iteration, and the curse of dimensionality.
chapter 5MC prediction (first/every-visit), action-value estimation, MC control with exploring starts, on-policy \(\varepsilon\)-soft methods, and off-policy learning via importance sampling.