Paper
The Variational Bandwidth Bottleneck: Stochastic Evaluation on an\n Information Budget
In many applications, it is desirable to extract only the relevant\ninformation from complex input data, which involves making a decision about\nwhich input features are relevant. The information bottleneck method formalizes\nthis as an information-theoretic optimization problem by maintaining an optimal\ntradeoff between compression (throwing away irrelevant input information), and\npredicting the target. In many problem settings, including the reinforcement\nlearning problems we consider in this work, we might prefer to compress only\npart of the input. This is typically the case when we have a standard\nconditioning input, such as a state observation, and a "privileged" input,\nwhich might correspond to the goal of a task, the output of a costly planning\nalgorithm, or communication with another agent. In such cases, we might prefer\nto compress the privileged input, either to achieve better generalization\n(e.g., with respect to goals) or to minimize access to costly information\n(e.g., in the case of communication). Practical implementations of the\ninformation bottleneck based on variational inference require access to the\nprivileged input in order to compute the bottleneck variable, so although they\nperform compression, this compression operation itself needs unrestricted,\nlossless access. In this work, we propose the variational bandwidth bottleneck,\nwhich decides for each example on the estimated value of the privileged\ninformation before seeing it, i.e., only based on the standard input, and then\naccordingly chooses stochastically, whether to access the privileged input or\nnot. We formulate a tractable approximation to this framework and demonstrate\nin a series of reinforcement learning experiments that it can improve\ngeneralization and reduce access to computationally costly information.\n
Authors: Goyal, Anirudh · Bengio, Yoshua · Botvinick, Matthew · Levine, Sergey