Paper

Learning Multilevel Distributed Representations for High-Dimensional Sequences

We describe a family of non-linear sequence models that is substantially more powerful than hidden Markov models or linear dynamical systems. Our models have simple approximate inference and learning procedures that work well in practice. Multilevel representations of sequential data can be learned one hidden layer at a time, and adding extra hidden layers improves the resulting generative models. The models can be trained with very high-dimensional, very non-linear data such as raw pixel sequences. Their performance is demonstrated using synthetic video sequences of two balls bouncing in a box.

Authors: Ilya Sutskever · Geoffrey E. Hinton

Related discussions