Composite objective mirror descent | RDL Research Database

Abstract

We present a new method for regularized convex optimization and analyze it under both online and stochastic optimization settings. In addition to unifying previously known firstorder algorithms, such as the projected gradient method, mirror descent, and forwardbackward splitting, our method yields new analysis and algorithms. We also derive specific instantiations of our method for commonly used regularization functions, such as ℓ1, mixed norm, and trace-norm. 1

Keywords

Stochastic gradient descentGradient descentRegularization (linguistics)Regular polygonComputer scienceConvex optimizationConvex functionNorm (philosophy)Mathematical optimizationProximal gradient methods for learningOptimization problemProximal Gradient MethodsAlgorithmMathematicsArtificial intelligenceConvex analysisArtificial neural network

Affiliated Institutions

Related Publications

Regularization Paths for Generalized Linear Models via Coordinate Descent

Jerome H. Friedman , Trevor Hastie , Robert Tibshirani

We develop fast algorithms for estimation of generalized linear models with convex penalties. The models include linear regression, two-class logistic regression, and multi- nom...

2010 Journal of Statistical Software 15877 citations

HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent

Feng Niu , Benjamin Recht , Christopher Ré +1 more

Stochastic Gradient Descent (SGD) is a popular algorithm that can achieve state-of-the-art performance on a variety of machine learning tasks. Several researchers have recently ...

2011 arXiv (Cornell University) 1224 citations

Reconfigurable Intelligent Surfaces for Energy Efficiency in Wireless Communication

Chongwen Huang , Alessio Zappone , George C. Alexandropoulos +2 more

The adoption of a Reconfigurable Intelligent Surface (RIS) for downlink multi-user communication from a multi-antenna base station is investigated in this paper. We develop ener...

2018 arXiv (Cornell University) 3473 citations

Sparse reconstruction by convex relaxation: Fourier and Gaussian measurements

Mark Rudelson , Roman Vershynin

This paper proves best known guarantees for exact reconstruction of a sparse signal f from few non-adaptive universal linear measurements. We consider Fourier measurements (rand...

2006 230 citations

Atomic Decomposition by Basis Pursuit

Scott Shaobing Chen , David L. Donoho , Michael A. Saunders

The time-frequency and time-scale communities have recently developed a large number of overcomplete waveform dictionaries --- stationary wavelets, wavelet packets, cosine packe...

1998 SIAM Journal on Scientific Computing 6879 citations

Publication Info

Year: 2010
Type: article
Pages: 14-26
Citations: 252
Access: Closed

External Links

Citation Metrics

252

OpenAlex

Cite This

APA Style

                            
                                    John C. Duchi, 
                                
                                    Shai Shalev‐Shwartz, 
                                
                                    Yoram Singer
                                
                                et al.
                            
                            (2010). 
                            Composite objective mirror descent. 
                            
                            , 14-26.