Authors: Ryota Tomioka , Taiji Suzuki , Masashi Sugiyama - Journal of Machine Learning Research, JMLR 2011 cited by 59
We analyze the convergence behaviour of a recently proposed algorithm for regularized estimation called Dual Augmented Lagrangian (DAL). Our analysis is based on a new interpretation of DAL as a proximal minimization algorithm. We theoretically show under some conditions that DAL converges super-linearly in a non-asymptotic and global sense. Due to a special modelling of sparse estimation problems in the context of machine learning, the assumptions we make are milder and more natural than those made in conventional analysis of augmented Lagrangian algorithms. In addition, the new interpretation enables us to generalize DAL to wide varieties of sparse estimation problems. We experimentally confirm our analysis in a large scale $\ell_1$-regularized logistic regression problem and extensively compare the efficiency of DAL algorithm to previously proposed algorithms on both synthetic and benchmark datasets.
✨ Checking sign-in… PDF Cited by View BibTeX Hide BibTeX View BibTeX Cite
An Interior-Point Method for Large-Scale ell… An Interior-Point Method for Large-Scale ell _1-Regularized Least Squares Gradient Projection for Sparse Reconstruction… Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems Bregman Iterative Algorithms for ell… Bregman Iterative Algorithms for ell 1-Minimization with Applications to Compressed Sensing Dual-Augmented Lagrangian Method for… Dual-Augmented Lagrangian Method for Efficient Sparse Reconstruction Efficient Online and Batch Learning Using… Efficient Online and Batch Learning Using Forward Backward Splitting A Fast Iterative Shrinkage-Thresholding… A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems A regularized discriminative framewor… A regularized discriminative framework for EEG analysis with application to brain-computer interface Distributed Optimization and Statistical Learnin… Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers A Fast Augmented Lagrangian Algorithm fo… A Fast Augmented Lagrangian Algorithm for Learning Low-Rank Matrices Estimation of low-rank tensors via convex… Estimation of low-rank tensors via convex optimization A Singular Value Thresholding Algorithm… A Singular Value Thresholding Algorithm for Matrix Completion Alternating Direction Algorithms for… Alternating Direction Algorithms for 1-Problems in Compressive Sensing A Fast Augmented Lagrangian Algorithm fo… A Fast Augmented Lagrangian Algorithm for Learning Low-Rank Matrices Estimation of low-rank tensors via convex… Estimation of low-rank tensors via convex optimization Recent Advances of Large-Scale Linear… Recent Advances of Large-Scale Linear Classification Learning a common substructure of multipl… Learning a common substructure of multiple graphical Gaussian models Scalable Training of Sparse Linear SVMs Scalable Training of Sparse Linear SVMs Dual Averaging and Proximal Gradient… Dual Averaging and Proximal Gradient Descent for Online Alternating Direction Multiplier Method High-Dimensional Feature Selection by… High-Dimensional Feature Selection by Feature-Wise Kernelized Lasso Tractable Optimization in Machine Learning Tractable Optimization in Machine Learning Theoretical and Experimental Analyses o… Theoretical and Experimental Analyses of Tensor-Based Regression and Classification Two-Stage Fuzzy Multiple Kernel Learning Based o… Two-Stage Fuzzy Multiple Kernel Learning Based on Hilbert-Schmidt Independence Criterion A trust region-type normal map-based… A trust region-type normal map-based semismooth Newton method for nonsmooth nonconvex composite optimization FAStEN: An Efficient Adaptive Method for… FAStEN: An Efficient Adaptive Method for Feature Selection and Estimation in High-Dimensional Functional Regressions Super-Linear Convergence of Dual Augmented… Super-Linear Convergence of Dual Augmented Lagrangian Algorithm for Sparsity Regularized Estimation Earlier references Focus paper Citing papers Older Newer Click a node to pin it, click the empty canvas to go back to this paper, or hover to preview. Open a node’s page from its title.
