Strong convergence of subgradient viscosity methods for the bilevel Ky Fan inequality

In this paper, building upon subgradient techniques and viscositytype approximations, we propose a simple projection algorithm for solving the bilevel Ky Fan inequality in a real Hilbert space, where the second level of the problem is variational inequalities. By choosing suitable regularization parameters, strong convergence of proposed iteration sequences is established under minimal assumptions of the cost bifunctions. Preliminary computational experience is also reported

Strong convergence of subgradient-viscosity methods for the bilevel Ky Fan inequality ∗ Pham N. Anh†, Le Q. Thuy‡and Tran T.H. Anh § Abstract. In this paper, building upon subgradient techniques and viscosity-type approximations, we propose a simple projection algorithm for solving the bilevel Ky Fan inequality in a real Hilbert space, where the second level of the problem is variational inequalities. By choosing suitable regularization parameters, strong convergence of proposed iteration sequences is established under minimal assumptions of the cost bifunctions. Preliminary computational experience is also reported. Keyword: Ky Fan inequalities, variational inequalities, monotone, subgradient, viscosity. AMS 2010 Mathematics subject classification: 65 K10, 90 C25. 1 Introduction Let H be a real Hilbert space with inner product and induced norm ·, · and · , respectively. Let C be a nonempty closed convex subset of H. The bilevel Ky Fan inequality is formulated as the follows: Find x∗ ∈ C such that g(x∗ , x) ≥ 0 ∀x ∈ Sol(f, C), where g : C × C → R, Sol(f, C) denotes the set of all solutions of the Ky Fan inequality: Find y ∗ ∈ C such that f (y ∗ , y) ≥ 0 ∀y ∈ C, and f : C × C → R. As usual, the bifunctions f and g are called to be the cost bifunctions. In this paper, we consider an important special case of the bilevel Ky Fan inequality, shortly BKF (g, F, C), that f (x, y) = F (x), y − x for all x, y ∈ C, where the mapping F : C → H. So, Problem BKF (g, F, C) is a simple bilevel problem. However, it covers the Ky Fan inequality [9, 14], bilevel minimization problems [11, 12, 22, 24], bilevel variational inequalities [3, 4, 6, 13, 26], mathematical programs with equilibrum constraints [16], the ∗ This work was completed at the Vietnam Institute for Advanced Study in Mathematics and funded by Vietnam National Foundation for Science and Technology Development (NAFOSTED) under grant number ”101.02-2013.03”. † Department of Scientific Fundamentals, Posts and Telecommunications Institute of Technology, Hanoi, Vietnam (anhpn@ptit.edu.vn). ‡ School of Applied Mathematics and Informatics, Ha Noi University of Science and Technology, Vietnam. § Department of Mathematics, Haiphong University, Vietnam. 1 minimum-norm problem [27] and many other problems in science and engineering (see [5, 10, 21]). It is well-known that the viscosity method is a fundamental method for solving variational inequalities with the constraint set is the fixed point set of a nonexpansive mapping (shortly, the hierarchical variational inequality). That is the problem of finding x∗ ∈ F ix(T ) (the fixed point set of a nonexpansive mapping T : C → C) such that (I − V )(x∗ ), x − x∗ ≥ 0 ∀x ∈ F ix(T ), where V : C → C is nonexpansive and I is the identity mapping. In this iteration method in a real Hilbert space, there are two schemes to solve it: One implicit and one explicit as the follows: xt = tf (xt ) + (1 − t)T (xt ) and xk+1 = λk f (xk ) + (1 − λk )T (xk ), where f is contraction on C, t ∈ (0, 1) and {λk } ⊂ [0, 1]. In [19], Mainge and Moudafi considered the viscosity method for the hierarchical variational inequality as the follows: x0 ∈ C, xk+1 := λk f (xk ) + (1 − λk )[αk V (xk ) + (1 − λk )T (xk )], where f : C → C is a contraction. Under certain appropriate conditions onto parameters αk and λk , they showed that the iteration sequence {xk } converges strongly to a solution of the hierarchical variational inequality. It should be noticed that the method can be regarded as a generalized version Halpern’s algorithm. Upon the idea of this, Lu et al. in [15] investigated other hybrid viscosity approximation methods for solving the hierarchical variational inequality. By combining viscosity approximation methods with projected subgradient techniques, Mainge in [17] established a strong convergence theorem for optimization with variational inequality constraints in H. Xu in [25] introduced a viscosity method for hierarchical fixed point approach to variational inequalities. Here, the author showed that the sequence defined by the implicit hierarchical scheme converges strongly in norm to a solution of the hierarchical variational inequality. Recently, the viscosity method has been studied to develop iteration algorithms for variational inequalities, Ky Fan inequalities, and other problems (see [8, 18]). Methods for solving Problem BKF (g, F, C) also have been studied extensively (see [1, 2, 14, 20]). It is the purpose of the present article to extend the above viscosity approximation method, with suitable modifications, by using subgradient techniques to Problem BKF (g, F, C). By this way, we obtain convergent algorithms for solving the problem in a real Hilbert space H. The iteration algorithm is simple and efficient. At each iteration, we only require computing the projection of a point onto the domain. This paper is organized as follows. In Section 2, we recall some definitions, known lemmas with the corresponding references and our main results. Finally, we present some numerical experiments to illustrate the behavior of the proposed algorithms. 2 The algorithm and its convergence Let us recall some definitions and results that will be used in the sequel. Let C be a nonempty closed convex subset of H, for every x ∈ H, there exists an unique element 2 P rC (x), defined by P rC (x) = arg min { y − x : y ∈ C} . It is also known that P rC is firmly nonexpansive, or 1-inverse strongly monotone, i.e., P rC (x) − P rC (y), x − y ≥ P rC (x) − P rC (y) 2 ∀x, y ∈ C. Besides, we recall some other properties as the follows: x − PC (x) 2 + PC (x) − y 2 2 ≤ x−y ∀x, y ∈ C; (2.1) x − P rC (x), P rC (x) − y ≥ 0 ∀x, y ∈ C; x − P rC (x − y) ≤ y 2 z − P rC (x − y) (2.2) ∀x ∈ C, y ∈ H; ≤ x−z 2 (2.3) − 2 x − z, y + 5 y 2 ∀x, z ∈ C, y ∈ H. (2.4) A mapping F : C → H is called monotone on C, if F (x) − F (y), x − y ≥ 0 ∀x, y ∈ C; paramonotone on C, if F is monotone on C and F (x) − F (y), x − y = 0 ⇒ F (x) = F (y); weakly closed on C, if {xk } ⊂ C, xk ⇀ x and F (xk ) ⇀ w ⇒ w = T x. A bifunction g : C × C → R is called monotone on C, if g(x, y) + g(y, x) ≤ 0 ∀x, y ∈ C; ρ-strongly monotone on C, if g(x, y) + g(y, x) ≤ −ρ x − y 2 ∀x, y ∈ C. Throughout this paper, we consider the bifunction g, the mapping F and regularization parameters with the following properties: (A1 ) g is ρ-strongly monotone and for each x ∈ C, g(x, ·) is weakly continuous and convex on the domain C, ∂2 g(·, ·) is upper semicontinuous and g(x, x) = 0 for all x ∈ C; (A2 ) F : C → H is paramonotone and weakly closed on C, and Lipschitz continuous on the domain C; (A3 ) the set SV I(F, C) := {x ∈ C : F (x), y − x ≥ 0 ∀y ∈ C} is nonempty; (A4 ) µ ∈ (0, ∞), for each k ≥ 0, αk and λk are nonnegative real numbers such that ∞ ∞ k=0 ∞ λ2k < ∞, lim λk = 0, lim αk = 0, λk = ∞, k=0 k→∞ k→∞ 3 ∞ αk = ∞, k=0 αk λk = ∞. k=0 It is easy to check again that condition (A4 ) is satisfied by λk := 1 1 , αk := , with λ ∈ λ (k + 1) (k + 1)α 1 ,1 2 and α ∈ (0, 1 − λ) . We establish a series of preliminary results needed for our convergence analysis. Lemma 2.1 [17] Let {bn } be a sequence of real numbers that does not decrease at infinity, in the sense that there exists a subsequence {bnj }j≥0 of {bn } which satisfies bnj < bnj +1 for all j ≥ 0. Also consider the sequence of integers {τn }n≥0 defined by τn = max{k ≤ n : bk < bk+1 }. Then, {τn }n≥n0 is a nondecreasing sequence verifying lim τn = ∞, and for all n ≥ 0, it n→∞ holds that bτn < bτn +1 and bn < bτn +1 . Lemma 2.2 [17] Let {λn } and {βn } be nonnegative sequences such that ∞ ∞ ∞ λ2n λn = ∞, n=0 λn βn < ∞. < ∞ and n=0 n=0 Then, the following two results hold: (i) There exists a subsequence {βnk } of {βn } such that lim βnk = 0. k→∞ (ii) If {λn } and {βn } are also such that βn+1 − βn < θλn (for some positive θ), then lim βn = 0. n→∞ Lemma 2.3 [18] Let φ be the functional defined, for any x in C, by φ(x) := T x, x − q , where q ∈ H and T : C → H is monotone and weakly closed on C, and Lipschitz continuous on bounded subset of C. Then, φ is weakly lower semicontinuous on C. In order to solve Problem BKF (g, F, C), we investigate the convergence analysis of the sequence {xk } given by the following iteration: Algorithm 2.4 Initialization. Take two sequences of nonnegative real numbers {αk } and {λk }, µ ∈ (0, +∞), x0 ∈ C. Step 1. Choose wk ∈ ∂2 g(xk , xk ). Step 2. Set dk = F (xk ) + αk wk , ηk = max{µ, dk } and xk+1 = P rC xk − k := k + 1 and go to Step 1. λk k ηk d . We now turn to analyse the convergence of Algorithm 2.4. Theorem 2.5 Let the above assumptions (A1 )−(A4 ) hold. Then, the sequence {xk } generated by Algorithm 2.4 converges strongly to the unique solution of Problem BKF (g, F, C). Proof. We divide the proof into several claims. 4 3 Examples and numerical results In this section, we illustrate the proposed algorithms by a class of the bilevel Ky Fan inequality as defined by BKF (g, F, C), where C is a polyhedral convex set given by C := {x ∈ Rn : Ax ≤ b}, and the bifunction g : C × C → R is of the form g(x, y) := G(x) + Qy + q, y − x , where G : C → Rn , Q ∈ Rn×n is a symmetric positive semidefinite matrix and q ∈ Rn . Since Q is symmetric positive semidefinite, g(x, ·) is convex for each fixed x ∈ C. The cost mapping F is defined more detail in the following examples. It is well-known that if G is ξ-strongly monotone on C and ξ > Q , then g is strongly monotone on C × C with constant ξ − Q (see [23]). As usual, we can say that xk is an ǫ-solution of Problem BKF (g, F, C) if xk+1 − xk ≤ ǫ. Example 3.1 Consider the mapping F : R3 → R3 given in [24] which is paramonotone (not strictly monotone) of the form     2 2 0 6.1 2 0 F (x) := 1 2 0 x, G(x) := P x and P :=  2 5.6 0  . 0 0 0 0 0 4.5 In this example, let C, Q and q be defined by C := x ∈ R3 : xi ≥ 0 ∀i = 1, 2, 3, x1 + x2 + x3 ≤ 10 , and     1.6 1 0 1    Q := 1 1.6 0 , q := −3 . 0 0 1.5 4 Example 3.2 Consider the mapping F : R4 → R4 described in [7] which is paramonotone of the form F (x) = (x1 − 2x2 , −2x1 + 4x2 , x3 − 2x4 , −2x3 + 4x4 )T and   7 2.5 0 0 2.5 5.5 0 0  . G(x) := P x, P :=  0 0 4 1 0 0 1 2.7 Take C, Q and q as the follows: C := x ∈ R4 : xi ≥ 0 ∀i = 1, · · · , 4, x1 + x2 + x3 + x4 ≤ 9 , and    1.2 1 0 0 −1  1 1.5 0 2  0  , q :=   . Q :=  0 3 0 1.2 0  5 0 0 0 3.5  5 From the preliminary numerical results reported in the tables, we observe that (a) Similar to other methods for Ky Fan inequalities such as the proximal point algorithm, or the auxiliary principle for equilibrium problems, the rapidity of the algorithm depends very much on the starting point. (b) The algorithm is quite sensitive to the choice of the parameters λk and αk . References [1] Anh, P.N.: An interior proximal method for solving pseudomonotone nonlipschitzian multivalued variational inequalities. Nonlinear Analysis Forum 14, 27-42 (2009) [2] Anh, P.N., Kim, J.K.: Outer approximation algorithms for pseudomonotone equilibrium problems. Computers and Mathematics with Applications 61, 2588-2595 (2011) [3] Anh, P.N., Kim, J.K., Muu, L.D.: An extragradient method for solving bilevel variational inequalities. Journal of Global Optimization 52, 627-639 (2012) [4] Anh, P.N., Muu, L.D., Strodiot, J.J.: Generalized projection method for nonLipschitz multivalued monotone variational inequalities. ACTA Mathematica Vietnamica 34, 67-79 (2009) [5] Anh, P.N., Muu, L.D., Hien, N.V., Strodiot, J.J.: Using the Banach contraction principle to implement the proximal point method for multivalued monotone variational inequalities. Journal of Optimization Theory and Applications 124, 285-306 (2005) [6] Baiocchi, C., Capelo, A.: Variational and quasivariational inequalities, applications to free boundary problems. NewYork, (1984) [7] Bao, X-B., Liao, L-Z., Qi, L.: Novel Neural Network for Variational inequalities with Linear and Nonlinear Constraints. IEEE Transactions on Neural Network 16, 1305-1317 (2005) [8] Ceng, L.C., Yao, J.C.: Relaxed viscosity approximation methods for fixed point problems and variational inequality problems Nonlinear Analysis 69, 3299-3309 (2008) [9] Daniele, P., Giannessi, F., Maugeri, A.: Equilibrium problems and variational models. Kluwer, (2003) [10] Facchinei, F., Pang, J.S.: Finite-dimensional variational inequalities and complementary problems. Springer-Verlag, NewYork, (2003) [11] Glackin, J., Ecker, J.G., Kupferschmid, M.: Solving bilevel linear programs using multiple objective linear programming. Journal Optimization Theory Applications 140, 197-212 (2009) [12] Iiduka, H.: Strong convergence for an iterative method for the triple-hierarchical constrained optimization problem. Nonlinear Analysis 71, e1292-e1297 (2009) [13] Kalashnikov, V.V., Kalashnikova, N.I.: Solving two-level variational inequality. Journal of Global Optimization 8, 289-294 (1996) 6 [14] Konnov, I.V.: Combined relaxation methods for variational inequalities. SpringerVerlag, Berlin, (2000) [15] Lu, X.W., Xu, H.K., Yin, X.M.: Hybrid methods for a class of monotone variational inequalities. Nonlinear Analysis 71, 1032-1041 (2009) [16] Luo, Z.Q., Pang, J.S., Ralph, D.: Mathematical programs with equilibrum constraints. Cambridge University Press, Cambridge, (1996) [17] Mainge, P.E.: Strong convergence of projected subgradient methods for nonsmooth and nonstrictly convex minimization, Set-Valued Analysis 16, 899-912 (2008) [18] Mainge, P.E.: Projected subgradient techniques and viscosity methods for optimization with variational inequality constraints. European Journal of Operational Research 205, 501-506 (2010) [19] Mainge, P.E., Moudafi, A.: Strong convergence of an iterative method for hierarchical fixed-point problems. Pacific Journal of Optimization 3 (3), 529538 (2007) [20] Moudafi, A.: Proximal methods for a class of bilevel monotone equilibrium problems. Journal of Global Optimization 47, 287-292 (2010) [21] Sabharwal, A., Potter, L.C.: Convexly constrained linear inverse problems: Iterative least-squares and regularization. IEEE Trans. Signal Process 46, 2345-2352 (1998) [22] Solodov, M.: An explicit descent method for bilevel convex optimization. Journal of Convex Analysis 14, 227-237 (2007) [23] Quoc, T.D., Muu, L.D., Nguyen, V.H.: Extragradient algorithms extended to equilibrium problems. Optimization 57, 749-776 (2008) [24] Trujillo-Corteza, R., Zlobecb, S.: Bilevel convex programming models. Optimization 58, 1009-1028 (2009) [25] Xu, M.H.: Viscosity methods for hierarchical fixed point approach to variational inequalities. Taiwanese Journal of Mathematics 14 (2), 463-478 (2010) [26] Xu, M.H., Li, M., Yang, C.C.: Neural networks for a class of bi-level variational inequalities. Journal of Global Optimization 44, 535-552 (2009) [27] Yao, Y., Marino, G., Muglia, L.: A modified Korpelevich´s method convergent to the minimum-norm solution of a variational inequality. Optimization, 1-11, iFirst (2012) 7  ... in the tables, we observe that (a) Similar to other methods for Ky Fan inequalities such as the proximal point algorithm, or the auxiliary principle for equilibrium problems, the rapidity of the. .. analyse the convergence of Algorithm 2.4 Theorem 2.5 Let the above assumptions (A1 )−(A4 ) hold Then, the sequence {xk } generated by Algorithm 2.4 converges strongly to the unique solution of Problem... BKF (g, F, C) Proof We divide the proof into several claims Examples and numerical results In this section, we illustrate the proposed algorithms by a class of the bilevel Ky Fan inequality as defined