Paper
Learning Dynamic State Abstractions for Model-Based Reinforcement Learning
A key challenge in model-based reinforcement learning (RL) is to synthesize computationally efficient and accurate environment models. We show that carefully designed models that learn predictive and compact state representations, also called state-space models, substantially reduce the computational costs for predicting outcomes of sequences of actions. Extensive experiments establish that state-space models accurately capture the dynamics of Atari games from the Arcade Learning Environment (ALE) from raw pixels. Furthermore, RL agents that use Monte-Carlo rollouts of these models as features for decision making outperform strong model-free baselines on the game MS_PACMAN, demonstrating the benefits of planning using learned dynamic state abstractions.
Authors: Lars Buesing · Théophane Weber · Sébastien Racanière · S. M. Ali Eslami · Danilo Jimenez Rezende · David Reichert · Fabio Viola · Frederic Besse · Karol Gregor · Demis Hassabis