Paper
Fast Inference and Learning for Modeling Documents with a Deep Boltzmann Machine
We introduce a type of Deep Boltzmann Machine (DBM) that is suitable for ex-tracting distributed semantic representations from a large unstructured collection of docu-ments. We propose an approximate inference method that interacts with learning in a way that makes it possible to train the DBM more efficiently than previously proposed methods. Even though the model has two hidden lay-ers, it can be trained just as efficiently as a standard Restricted Boltzmann Machine. Our experiments show that the model as-signs better log probability to unseen data than the Replicated Softmax model. Fea-tures extracted from our model outperform LDA, Replicated Softmax, and DocNADE models on document retrieval and document classification tasks. 1.
Authors: Nitish Srivastava · Ruslan Salakhutdinov · Geoffrey E. Hinton