Non-linear latent factor models for revealing structure in high-dimensional data

Abstract

Real world data is not random: The variability in the data-sets that arise in computer vision,\nsignal processing and other areas is often highly constrained and governed by a number of\ndegrees of freedom that is much smaller than the superficial dimensionality of the data.\nUnsupervised learning methods can be used to automatically discover the “true”, underlying\nstructure in such data-sets and are therefore a central component in many systems that deal\nwith high-dimensional data.\n\nIn this thesis we develop several new approaches to modeling the low-dimensional structure\nin data. We introduce a new non-parametric framework for latent variable modelling, that in\ncontrast to previous methods generalizes learned embeddings beyond the training data and its\nlatent representatives. We show that the computational complexity for learning and applying\nthe model is much smaller than that of existing methods, and we illustrate its applicability\non several problems.\n\nWe also show how we can introduce supervision signals into latent variable models using\nconditioning. Supervision signals make it possible to attach “meaning” to the axes of a latent\nrepresentation and to untangle the factors that contribute to the variability in the data. We\ndevelop a model that uses conditional latent variables to extract rich distributed representations\nof image transformations, and we describe a new model for learning transformation\nfeatures in structured supervised learning problems.

Keywords

Affiliated Institutions

• University of Toronto CA

Related Publications