Paper
Learning Sparse Topographic Representations with Products of Student-t Distributions
We propose a model for natural images in which the probability of an image is proportional to the product of the probabilities of some outputs. We encourage the system to find sparse features by using a Student-t distribution to model each output. If the t-distribution is used to model the combined outputs of sets of neurally adjacent filters, the system learns a topographic map in which the orientation, spatial frequency and location of the filters change smoothly across the map. Even though maximum likelihood learning is intractable in our model, the product form allows a relatively efficient learning procedure that works well even for highly overcomplete sets of filters. Once the model has been learned it can be used as a prior to derive the iterated Wiener filter for the purpose of denoising images.
Authors: Max Welling · Simon Osindero · Geoffrey E. Hinton