Finite-Dimensional Approximation of Gaussian Processes

Abstract

Gaussian process (GP) prediction suffers from O(n3) scaling with the data set size n. By using a finite-dimensional basis to approximate the GP predictor, the computational complexity can be reduced. We derive optimal finite-dimensional predictors under a number of assumptions, and show the superiority of these predictors over the Projected Bayes Regression method (which is asymptotically optimal). We also show how to calculate the minimal model size for a given n. The calculations are backed up by numerical experiments.

Keywords

Gaussian processApplied mathematicsMathematicsGaussianScalingAsymptotically optimal algorithmFinite setComputational complexity theoryMathematical optimizationSet (abstract data type)Bayes' theoremAlgorithmComputer scienceStatisticsBayesian probabilityMathematical analysisPhysicsGeometry

Affiliated Institutions

Related Publications

Gaussian Processes for Machine Learning

Carl Edward Rasmussen , Christopher K. I. Williams

We give a basic introduction to Gaussian Process regression models. We focus on understanding the role of the stochastic process and how it is used to define a distribution over...

2005 The MIT Press eBooks 10408 citations

The Capacity of Linear Channels with Additive Gaussian Noise

R. K. Mueller , G.J. Foschini

The standard method of computing the mutual information between two stochastic processes with finite energy replaces the processes with their Fourier coefficients. This procedur...

1970 Bell System Technical Journal 22 citations

Efficient Kernel Machines Using the Improved Fast Gauss Transform

Changjiang Yang , Ramani Duraiswami , Larry S. Davis

The computation and memory required for kernel machines with N training samples is at least O(N 2). Such a complexity is significant even for moderate size problems and is prohi...

2004 128 citations

Regression and time series model selection in small samples

Clifford M. Hurvich , Chih‐Ling Tsai

A bias correction to the Akaike information criterion, AIC, is derived for regression and autoregressive time series models. The correction is of particular use when the sample ...

1989 Biometrika 6179 citations

Gaussian Process Priors with Uncertain Inputs Application to Multiple-Step Ahead Time Series Forecasting

Agathe Girard , Carl Edward Rasmussen , Joaquin Quiñonero Candela +1 more

We consider the problem of multi-step ahead prediction in time series analysis using the non-parametric Gaussian process model. k-step ahead forecasting of a discrete-time non-l...

2002 370 citations

Publication Info

Year: 1998
Type: article
Volume: 11
Pages: 218-224
Citations: 33
Access: Closed

External Links

Citation Metrics

OpenAlex

Cite This

APA Style

                            
                                    Giancarlo Ferrari‐Trecate, 
                                
                                    Christopher K. I. Williams, 
                                
                                    Manfred Opper
                                
                            (1998). 
                            Finite-Dimensional Approximation of Gaussian Processes. 
                            Neural Information Processing Systems
                            , 11
                            
                            , 218-224.