Lecture 3: Immediate RL

Model:

t: action at, reward (payoff, evaluation, cost) Rt

outcome is immediate, no state inputs, inputs are the rewards,

Exploration vs. Exploit

Action → Reward sampled from a distribution (stochastic)

issue is when we do not know the distribution.

q*(a) - True expected payoff (this remains unknown)

Lecture 4: Bandit Optimalities

Bandit problems -

1. Asymptotic correctness - over a period of time converge to the correct option.

2. Regret optimality - maxmise payoff even during learning

   

   tradeoff being exploration is given up and edge cases might get missed out.

3. PAC optimality - Probably Approximately Correct

   Given (ε, δ) what is the smallest number of times we have select arms to give PAC guarantee.

Lecture 5: Value functions based methods

Estimate for q*

1. Greedy: Select the maximum (Exploitation)