Paper
Discovering High Order Features with Mean Field Modules
A new form of the deterministic Boltzmann machine (DBM) learning procedure is presented which can efficiently train network to discriminate between input vectors according to some criterion. The new technique directly utilizes the free energy of these field modules to represent the probability that the criterion is met, the free energy being readily manipulated by the learning procedure. Although conventional deterministic Boltzmann learning fails to extract the higher order feature of shift at a network bottleneck, combining the new mean field with the mutual information objective function rapidly produces that perfectly extract this important higher order feature without direct external supervision.
Authors: Conrad C. Galland · Geoffrey E. Hinton