Abstract
Abstract : In this paper we discuss several recent conjugate-gradient type methods for solving large-scale nonlinear optimization problems. We demonstrate how the performance of these methods can be significantly improved by careful implementation. A method based upon iterative preconditioning will be suggested which performs reasonably efficiently on a wide variety of significant test problems. Our results indicate that nonlinear conjugate-gradient methods behave in a similar way to conjugate-gradient methods for the solution of systems of linear equations. These methods work best on problems whose Hessian matrices have sets of clustered eigenvalues. On more general problems, however, even the best method may require a prohibitively large number of iterations. We present numerical evidence that indicates that the use of theoretical analysis to predict the performance of algorithms on general problems is not straightforward. (Author)
Keywords
Conjugate gradient methodNonlinear conjugate gradient methodScale (ratio)Nonlinear systemComputer scienceMathematical optimizationMathematicsAlgorithmArtificial intelligencePhysicsGeographyGradient descentArtificial neural networkCartography
Related Publications
Preconditioning of Truncated-Newton Methods
In this paper we discuss the use of truncated-Newton methods, a flexible class of iterative methods, in the solution of large-scale unconstrained minimization problems. At each ...
The Conjugate Gradient Method and Trust Regions in Large Scale Optimization
Algorithms based on trust regions have been shown to be robust methods for unconstrained optimization problems. All existing methods, either based on the dogleg strategy or Hebd...
Pegasos
We describe and analyze a simple and effective iterative algorithm for solving the optimization problem cast by Support Vector Machines (SVM). Our method alternates between stoc...
Large-Scale Machine Learning with Stochastic Gradient Descent
During the last decade, the data sizes have grown faster than the speed of processors. In this context, the capabilities of statistical machine learning methods is limited by th...
Conjugate Gradient Methods with Inexact Searches
Conjugate gradient methods are iterative methods for finding the minimizer of a scalar function f(x) of a vector variable x which do not update an approximation to the inverse H...
Publication Info
- Year
- 1979
- Type
- report
- Citations
- 69
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
69
OpenAlex
8
Influential
40
CrossRef
Cite This
Philip E. Gill,
Walter Murray
(1979).
Conjugate-Gradient Methods for Large-Scale Nonlinear Optimization..
.
https://doi.org/10.21236/ada078713
Identifiers
- DOI
- 10.21236/ada078713