《ICML-2016-Deep Learning via Semi-Supervised Embedding》.pdf

Deep Learning via Semi-Supervised Embedding Jason Weston∗ jasonw@ Fr´ed´eric Ratle† frederic.ratle@ Ronan Collobert∗ collober@ (∗) NEC Labs America, 4 Independence Way, Princeton, NJ 08540 USA (†) IGAR, University of Lausanne, Amphipˆole, 1015 Lausanne, Switzerland Abstract tance measure based on a nonlinear manifold embed- We show how nonlinear embedding algo- ding as a first step (Chapelle et al., 2003; Chapelle rithms popular for use with shallow semi- Zien, 2005). Transductive Support Vector Machines supervised learning techniques such as ker- (TSVMs) (Vapnik, 1998) (which employs a kind of nel methods can be applied to deep multi- clustering) and LapSVM (Belkin et al., 2006) (which layer architectures, either as a regularizer at employs a kind of embedding) are examples of meth- the output layer, or on each layer of the ar- ods that are joint in their use of unlabeled data and chitecture. This provides a simple alterna- labeled data, but their architecture is still shallow. tive to existing approaches to deep learning Deep architectures seem a natural choice in hard AI whilst yielding competitive error rates com- tasks which involve several sub-tasks which can be pared to those methods, and existing shallow coded into the layers of the architecture. As argued by semi-supervised techniques.