A modified quadratic hybridization of Polak-Ribi`re-Polyak and Fletcher-Reeves conjugate gradient method for unconstrained optimization problems

This article presents a modified quadratic hybridization of the Polak–Ribi`ere– Polyak and Fletcher–Reeves conjugate gradient method for solving unconstrained optimization problems. Global convergence, with the strong Wolfe line search conditions, of the proposed quadratic hybrid conjugate gradient method is established. The new method is tested on a number of benchmark problems that have been extensively used in the literature and numerical results show the competitiveness of the new hybrid method.

PDF

___

[1] Al-Baali, M., Narushima, Y. and Yabe, H., A family of three term conjugate gradient methods with suffi- cient descent property for unconstrained optimization, Comput. Optim. Appl., 60, 89–110 (2015).
[2] Andrei, N., A new three-term conjugate gradient algorithm for unconstrained optimization, Numer. Algorithms, 68, 305–321 (2015).
[3] Andrei, N., An unconstrained optimization test functions collection, Adv. Model. Optim., 10(1), 147–161 (2008).
[4] Babaie-Kafaki, S., A quadratic hybridization of PolakRibiere-Polyak and Fletcher-Reeves conjugate gradient methods, J Optim. Theory Appl., 154(3), 916-932 (2012).
[5] Babaie-Kafaki, S., A Note on the global convergence of the quadratic hybridization of Polak-Ribiere-Polyak and Fletcher-Reeves conjugate gradient methods, J Optim. Theory Appl., 157(1), 297-298 (2013).
[6] Babaie-Kafaki, S., Two modified scaled nonlinear conjugate gradient methods, J Comput. Appl. Math., 261, 172–182 (2014).
[7] Babaie-Kafaki, S., Fatemi, M., and Mahdavi-Amiri, N., Two effective hybrid conjugate gradient algorithms based on modified BFGS updates, Numer. Algorithms, 58, 315–331 (2011).
[8] Babaie-Kafaki, S. and Ghanbari, R., A descent extension of the Polak-Ribi`ere-Polyak conjugate gradient method, Comp. Math. Appl., 68, 2005–2011 (2014).
[9] Dai, Y.-H., Conjugate gradient methods with Armijotype line searches, Acta Math. Appl. Sin. Engl. Ser., 18(1), 123–130 (2002).
[10] Dai, Y.-H. and Liao, L.Z., New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim., 43, 87–101 (2001).
[11] Dai, Y.-H. and Yuan, Y., A class of globally convergent conjugate gradient methods, Sci. China Ser. A., 46(2), 251–261 (2003).
[12] Dai, Y.-H. and Yuan, Y., A nonlinear conjugate gradient method with strong global convergence property, SIAM J. Optim., 10(1), 177-182 (1999).
[13] Dai, Y.-H., and Yuan, Y., An efficient hybrid conjugate gradient method for unconstrained optimization, Ann. Oper. Res., 103, 33–47 (2001).
[14] Dai, Z. and Wen, F., Another improved Wei-Yao-Liu nonlinear conjugate gradient method with sufficient descent property, Appl. Math. Comput., 218:7421– 7430 (2012).
[15] Dai, Z.-F. and Tian, B.-S., Global convergence of some modified PRP nonlinear conjugate gradient methods, Optim. Lett., 5, 615–630 (2011).
[16] Dolan, E.D. and Mor´e, J.J., Benchmarking optimization software with profile performance profiles, Math. Program., 91(2), 201–213 (2002).
[17] Dong, X.L., Liu, H. and He, Y., A self-adjusting conjugate gradient method with sufficient descent condition and conjugacy condition, J Optim. Theory Appl., 165, 225–241 (2015).
[18] Dong, X.L., Liu, H.W., He, Y.B. and Yang, X.M., A modified Hestenes-Stiefel conjugate gradient method with sufficient descent condition and conjugacy condition, J Comput. Appl. Math., 281, 239–249 (2015).
[19] Dong, Y., A practical PR+ conjugate gradient method only using gradient, Appl. Math. Comput., 219(4), 2041-2052 (2012).
[20] Fletcher, R. and Reeves, C., Function minimization by conjugate gradients, Comp. J., 7, 149–154 (1964).
[21] Gilbert, J.C., and Nocedal, J., Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim., 2(1), 21–42 (1992).
[22] Hager, W.W., and Zhang, H., A survey of nonlinear conjugate gradient methods, Pac. J. Optim., 2(1), 35– 58 (2006).
[23] Hestenes, M.R., and Stiefel, E., Methods for conjugate gradients for solving linear systems, J Res. Natl. Bur. Stand., 49, 409–436 (1952)
[24] Jia, W., Zong, J. and Wang, X., An improved mixed conjugate gradient method, Systems Engineering Procedia 2, 219–225 (2012).
[25] Liu, H., A mixture conjugate gradient method for unconstrained optimization, Third International Symposium on Intelligent Information Technology and Security Informatics, IEEE, 26–29 (2010).
[26] Liu, Y. and Storey, C., Efficient generalized conjugate gradient algorithms, Part 1: theory, J Optim. Theory Appl., 69, 129-137 (1992).
[27] Liu, J., A hybrid nonlinear conjugate gradient method, Lobachevskii J. Math., 33(3), 195–199 (2012).
[28] Liu, J.K. and Li, S.J., New hybrid conjugate gradient method for unconstrained optimization, Appl. Math. Comput., 245, 36–43 (2014).
[29] Mo, J., Gu, N. and Wei, Z., Hybrid conjugate gradient methods for unconstrained optimization, Optim. Methods Softw., 22(2), 297–307 (2007).
[30] Mor´e, J.J., Garbow, B.S. and Hillstrom, K.E., Testing unconstrained optimization software, ACM Trans. Math. Software, 7, 17–41 (1981).
[31] Nocedal, J. and Wright, S.J., Numerical Optimization, 2nd Edition, Springer Science+ Business Media, LLC. Printed in the United States of Ameraica (2006).
[32] Polak, E., and Ribi`ere, G., Note sur la convergence de m`ethodes de directions conjugu`ees, Rev. Fran¸caise d’Infom. Recherche Op`erationnelle, 3(16), 35–43 (1969).
[33] Polyak, B.T., The conjugate gradient method in Extreme problems, USSR Comput. Math. Math. Phys., 9(4), 94–112 (1969).
[34] Shi, Z.-J., Convergence of line search methods for unconstrained optimization, Appl. Math. Comput., 157, 393–405 (2004).
[35] Sun, M. and Liu, J., Three modified Polak-Ribi`erePolyak conjugate gradient methods with sufficient descent property, Journal of Inequalities and Applications, 2015–2125 (2015).
[36] Touati-Ahmed, D. and Storey, C., Efficient hybrid conjugate gradient techniques, J Optim. Theory Appl., 64(2), 379–397 (1990).
[37] Yabe, H. and Sakaiwa, N., A new nonlinear conjugate gradient method for unconstrained optimization, J Oper. Res., 48(4), 284–296 (2005).
[38] Yan, H., Chen, L. and Jiao, B., HS-LS-CD hybrid conjugate gradient algorithm for unconstrained optimization, Second International Workshop on Computer Science and Engineering, IEEE, 264–268 (2009).
[39] Yuan, G. and Lu, X., A modified PRP conjugate gradient method, Ann. Oper. Res., 166, 73–90 (2009).
[40] Zhang, L., Zhou, W. and Li, D.-H., A descent modi- fied Polak-Ribi`ere-Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal., 26, 629–640 (2006).
[41] Zhou, A., Zhu, Z., Fan, H. and Qing, Q., Three new hybrid conjugate gradient methods for optimization, Appl. Math., 2(3), 303–308 (2011).