Paper
Connectionist Models: Proceedings of the Summer School Held in San Diego, California on 1990
The simplicity and locality of the contrastive Hebb synapse (CHS) used in Boltzmann machine learning makes it an attractive model for real biological synapses. The slow learning exhibited by the stochastic Boltzmann machine can be greatly improved by using a mean field approximation and it has been shown (Hinton, 1989) that the CHS also performs steepest descent in these deterministic mean field networks. A major weakness of the learning procedure, from a biological perspective, is that the derivation assumes detailed symmetry of the connectivity. Using networks with purely asymmetric connectivity, we show that the CHS still works in practice provided the connectivity is grossly symmetrical so that if unit i sends a connection to unit j, there are numerous indirect feedback paths from j to i.
Authors: David S. Touretzky · Jeffrey L. Elman · Terrence J. SeJnowski · Geoffrey E. Hinton