Paper
Learning Representations by Recirculation
Summary form only given, as follows. A new learning procedure for networks that contain several groups of nonlinear units arranged in a closed loop is described. The procedure modifies the weights on the connections between groups so that the training patterns over the input group return unaltered after passing around the loop. The learning rule amounts to changing each weight by an amount proportional to the product of the presynaptic activity and the rate of change of the postsynaptic activity. It is much simpler to implement in hardware than methods like back-propagation. Simulations show that it usually converges rapidly, and analysis shows that in certain restricted cases it performs gradient descent in a measure of how much the training patterns are altered by passing around the loop.
Authors: Geoffrey E. Hinton · James L. McClelland