Paper
Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure
Abstract We show how to pretrain and fine-tune a mul-tilayer neural network to learn a nonlinear transformation from the input space to a low-dimensional feature space in which K-nearest neighbour classification performs well. We alsoshow how the non-linear transformation can be improved using unlabeled data. Our methodachieves a much lower error rate than Support Vector Machines or standard backpropagation ona widely used version of the MNIST handwritten digit recognition task. If some of the dimen-sions of the low-dimensional feature space are not used for nearest neighbor classification, ourmethod uses these dimensions to explicitly represent transformations of the digits that do notaffect their identity. 1 Introduction Learning a similarity measure or distance metric over theinput space
Authors: Ruslan Salakhutdinov · Geoffrey E. Hinton