Spectral Networks and Locally Connected Networks on Graphs

Abstract

Convolutional Neural Networks are extremely efficient architectures in image and audio recognition tasks, thanks to their ability to exploit the local translational invariance of signal classes over their domain. In this paper we consider possible generalizations of CNNs to signals defined on more general domains without the action of a translation group. In particular, we propose two constructions, one based upon a hierarchical clustering of the domain, and another based on the spectrum of the graph Laplacian. We show through experiments that for low-dimensional graphs it is possible to learn convolutional layers with a number of parameters independent of the input size, resulting in efficient deep architectures.

Keywords

Computer scienceExploitTranslation (biology)Convolutional neural networkTheoretical computer scienceCluster analysisGraphSpectral clusteringDomain (mathematical analysis)Spectral graph theoryLaplace operatorArtificial intelligencePattern recognition (psychology)MathematicsLine graph

Affiliated Institutions

Related Publications

On the Quality of Spectral Separators

Stephen Guattery , Gary L. Miller

Computing graph separators is an important step in many graph algorithms. A popular technique for finding separators involves spectral methods. However, there has not been much ...

1998 SIAM Journal on Matrix Analysis and A... 148 citations

Partitioning Sparse Matrices with Eigenvectors of Graphs

Alex Pothen , Horst D. Simon , K. N. Liou

The problem of computing a small vertex separator in a graph arises in the context of computing a good ordering for the parallel factorization of sparse, symmetric matrices. An ...

1990 SIAM Journal on Matrix Analysis and A... 1670 citations

Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering

Mikhail Belkin , Partha Niyogi

Drawing on the correspondence between the graph Laplacian, the Laplace-Beltrami operator on a manifold, and the connections to the heat equation, we propose a geometrically moti...

2002 The MIT Press eBooks 4494 citations

Learning Eigenfunctions Links Spectral Embedding and Kernel PCA

Yoshua Bengio , Olivier Delalleau , Nicolas Le Roux +3 more

In this letter, we show a direct relation between spectral embedding methods and kernel principal components analysis and how both are special cases of a more general learning p...

2004 Neural Computation 335 citations

Laplacian Eigenmaps for Dimensionality Reduction and Data Representation

Mikhail Belkin , Partha Niyogi

One of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data. We consider the problem of constructing a rep...

2003 Neural Computation 7514 citations

Publication Info

Year: 2013
Type: preprint
Citations: 2710
Access: Closed

External Links

View on DOI.org

Social Impact

Altmetric

Spectral Networks and Locally Connected Networks on Graphs

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

2710

OpenAlex

Cite This

APA Style

                            
                                    Joan Bruna, 
                                
                                    Wojciech Zaremba, 
                                
                                    Arthur Szlam
                                
                                et al.
                            
                            (2013). 
                            Spectral Networks and Locally Connected Networks on Graphs. 
                            arXiv (Cornell University)
                            
                            .
                            https://doi.org/10.48550/arxiv.1312.6203

Identifiers

DOI: 10.48550/arxiv.1312.6203