Model Based RL


Today I covered lecture 9 in Berkeley’s deep rl course. This covered model based reinforcement learning, which tries to approximate the model. Note that model based planning covers the ways to optimize a policy given a model, so these two make a good couple.

The basic concept is to approximate the partials of f with respect to the state and action, and we can do this by approximating the global (general) and local (specific) models and using iLQR to find an optimal policy. There are many optimization techniques including KL Divergence which helps us keep the model constrained to various parameters. Overall, I really enjoyed the extensive amounts of math used to derive the statistics based methods, but it could’ve used more intuition and impetus.

Below are my full notes

Model Based Reinforcement Learning