Abstract
Restricted Boltzmann machines were developed using binary stochastic hidden units. These can be generalized by replacing each binary unit by an infinite number of copies that all have the same weights but have progressively more negative biases. The learning and inference rules for these Stepped Sigmoid Units are unchanged. They can be approximated efficiently by noisy, rectified linear units. Compared with binary units, these units learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset. Unlike binary units, rectified linear units preserve information about relative intensities as information travels through multiple layers of feature detectors.
Keywords
Boltzmann machineBinary numberSigmoid functionComputer scienceObject (grammar)Restricted Boltzmann machineArtificial intelligenceInferencePattern recognition (psychology)Feature (linguistics)Cognitive neuroscience of visual object recognitionAlgorithmMathematicsComputer visionDeep learningArithmeticArtificial neural network
Affiliated Institutions
Related Publications
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two as...
LRBM: A Restricted Boltzmann Machine Based Approach for Representation Learning on Linked Data
Linked data consist of both node attributes, e.g., Preferences, posts and degrees, and links which describe the connections between nodes. They have been widely used to represen...
Feature selection for multiclass discrimination via mixed-integer linear programming
We reformulate branch-and-bound feature selection employing L/sub /spl infin// or particular L/sub p/ metrics, as mixed-integer linear programming (MILP) problems, affording con...
CSPNet: A New Backbone that can Enhance Learning Capability of CNN
Neural networks have enabled state-of-the-art approaches to achieve incredible results on computer vision tasks such as object detection. However, such success greatly relies on...
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, ...
Publication Info
- Year
- 2010
- Type
- article
- Pages
- 807-814
- Citations
- 13197
- Access
- Closed
External Links
Citation Metrics
13197
OpenAlex
Cite This
Vinod Nair,
Geoffrey E. Hinton
(2010).
Rectified Linear Units Improve Restricted Boltzmann Machines.
International Conference on Machine Learning
, 807-814.