For citation:
Balashov M. V. The Lezanski – Polyak – Lojasiewicz inequality and the convergence of the gradient projection algorithm. Izvestiya of Saratov University. Mathematics. Mechanics. Informatics, 2023, vol. 23, iss. 1, pp. 4-10. DOI: 10.18500/1816-9791-2023-23-1-4-10, EDN: ZSKZLA
The Lezanski – Polyak – Lojasiewicz inequality and the convergence of the gradient projection algorithm
We consider the Lezanski – Polyak – Lojasiewicz inequality for a real-analytic function on a real-analytic compact manifold without boundary in finite-dimensional Euclidean space. This inequality emerged in 1963 independently in works of three authors: Lezanski and Lojasiewicz from Poland and Polyak from the USSR. The inequality is appeared to be a very useful tool in the convergence analysis of the gradient methods, firstly in unconstrained optimization and during the past few decades in problems of constrained optimization. Basically, it is applied for a smooth in a certain sense function on a smooth in a certain sense manifold. We propose the derivation of the inequality from the error bound condition of the power type on a compact real-analytic manifold. As an application, we prove the convergence of the gradient projection algorithm of a real analytic function on a real analytic compact manifold without boundary. Unlike known results, our proof gives explicit dependence of the error via parameters of the problem: the power in the error bound condition and the constant of proximal smoothness first of all. Here we significantly use a technical fact that a smooth compact manifold without boundary is a proximally smooth set.
- Lee J. M. Manifolds and differential geometry. Graduate Studies in Mathematics. Rhode Island, AMS Providence, 2009. Vol. 107. 671 p. https://doi.org/10.1090/gsm/107
- Lojasiewicz S. Sur le probleme de la division. Studia Mathematica, 1959, vol. 18, pp. 87–136. https://doi.org/10.4064/sm-18-1-87-136
- Balashov M. V., Polyak B. T., Tremba A. A. Gradient projection and conditional gradient methods for constrained nonconvex minimization. Numerical Functional Analysis and Optimization, 2020, vol. 41, iss. 7, pp. 822–849. https://doi.org/10.1080/01630563.2019.1704780
- Karimi H., Nutini J., Schmidt M. Linear convergence of gradient and proximal-gradient methods under the Polyak – Lojasiewicz condition. In: Frasconi P., Landwehr N., Manco G., Vreeken J. (eds.) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2016. Lecture Notes in Computer Science, vol. 9851. Cham, Springer, 2016, pp. 795–811. https://doi.org/10.1007/978-3-319-46128-1_50
- Schneider R., Uschmajew A. Convergence results for projected line-search methods on varieties of low-rank matricies via Lojasiewicz inequality. SIAM Journal on Optimization, 2015, vol. 25, iss. 1, pp. 622–646. https://doi.org/10.1137/140957822
- Merlet B., Nguyen T. N. Convergence to equilibrium for discretizations of gradient-like flows on Riemannian manifolds. Differential Integral Equations, 2013, vol. 26, iss. 5/6, pp. 571–602. https://doi.org/10.57262/die/1363266079
- Vial J.-Ph. Strong and weak convexity of sets and functions. Mathematics of Operations Research, 1983, vol. 8, iss. 2, pp. 231–259. https://doi.org/10.1287/moor.8.2.231
- Balashov M. V. The gradient projection algorithm for smooth sets and functions in nonconvex case. Set-Valued and Variational Analysis, 2021, vol. 29, pp. 341–360. https://doi.org/10.1007/s11228-020-00550-4
- Balashov M. V. Stability of minimization problems and the error bound condition. Set-Valued and Variational Analysis, 2022, vol. 30, pp. 1061–1076. https://doi.org/10.1007/s11228-022-00634-3
- Ivanov G. E. Slabo vypuklye mnozhestva i funktsii: teoriya i prilozhenie [Weakly Convex Sets and Functions: Theory and Application]. Moscow, Fizmatlit, 2006. 351 p. (in Russian).
- Balashov M. V., Kamalov R. A. The gradient projection method with Armijo’s step size on manifolds. Computational Mathematics and Mathematical Physics, 2021, vol. 61, iss. 1, pp. 1776–1786. https://doi.org/10.1134/S0965542521110038
- Adly S., Nacry F., Thibault L. Preservation of prox-regularity of sets with applications to constrained optimization. SIAM Journal on Optimization, 2016, vol. 26, iss. 1, pp. 448–473. https://doi.org/10.1137/15M1032739
- Polyak B. T. Introduction to Optimization. New York, Optimization Software, 1987. 464 p.
- Balashov M. V., Tremba A. A. Error bound conditions and convergence of optimization methods on smooth and proximally smooth manifolds. Optimization: A Journal of Mathematical Programming and Operations Research, 2020, vol. 71, iss. 3, pp. 711–735. https://doi.org/10.1080/02331934.2020.1812066
- 1433 reads