Approximate karush kuhn tucker optimality conditions in multiobjective ptimization

HANOI PEDAGOGICAL UNIVERSITY DEPARTMENT OF MATHEMATICS ——————–o0o——————— Dao Ngoc Cao Approximate Karush-Kuhn-Tucker optimality conditions in multiobjective optimization BACHELOR THESIS Major: Analysis Hanoi, May 2019 HANOI PEDAGOGICAL UNIVERSITY DEPARTMENT OF MATHEMATICS ——————–o0o——————— Dao Ngoc Cao Approximate Karush-Kuhn-Tucker optimality conditions in multiobjective optimization BACHELOR THESIS Major: Analysis Supervisor: Dr Nguyen Van Tuyen Hanoi, May 2019 Thesis acknowledgment I would like to express my gratitudes to the teachers of the Department of Mathematics, Hanoi Pedagogical University 2, the teachers in the analysis group as well as the teachers involved The lecturers have imparted valuable knowledge and facilitated for me to complete the course and the thesis In particular, I would like to express my deep respect and gratitude to Dr Nguyen Van Tuyen, who has direct guidance, help me complete this thesis Due to time, capacity and conditions are limited, so the thesis can not avoid errors Then, I look forward to receiving valuable comments from teachers and friends Hanoi, 02 May, 2019 Student Dao Ngoc Cao Thesis assurance I assure that the data and the results of this thesis are true and not identical to other topics I also assure that all the help for this thesis has been acknowledge and that the results presented in the thesis has been identified clearly Hanoi, 02 May, 2019 Student Dao Ngoc Cao Contents Preface Preliminaries 1.1 Convex functions 1.2 Clarke subdifferential 1.2.1 Generalization of Derivatives 1.2.2 Subdifferential Calculus Approximate Karush-Kuhn-Tucker optimality conditions in multiobjective optimization 11 2.1 Approximate KKT Condition for Multiobjective Optimization Problems 11 2.2 Relations of the AKKT Condition with Other Optimality Conditions 21 Bibliography 29 Preface Karush–Kuhn–Tucker (KKT) optimality conditions play an important role in optimization theory, both for scalar optimization and for multiobjective optimization However, KKT optimality conditions not need to be fulfilled at local minimum points unless some constraint qualifications are satisfied In [4], Andreani, Mart´ınez and Svaiter introduced the so-called complementary approximate Karush–Kuhn–Tucker (CAKKT) condition for scalar optimization problems with smooth data Then, the authors proved that this condition is necessary for a point to be a local minimizer without any constraint qualification Moreover, they also showed that the augmented Lagrangian method with lower-level constraints introduced in [2] generates sequences converging to CAKKT points under certain conditions Optimality conditions of CAKKT-type have been recognized to be useful in designing algorithms for finding approximate solutions of optimization problems In this thesis, based on the recent work by Giorgi, Jim´enez and Novo [9], we study approximate Karush–Kuhn–Tucker (AKKT) condition for multiobjective optimization problems We show that the AKKT condition holds for local weak efficient solutions without any additional requirement Under the convexity of the related functions, an AKKT-type sufficient condition for global weak efficient solutions is established Some enhanced KKT-conditions are also examined The thesis is organized as follows In Chapter 1, we recall some basic definitions and preliminaries from nonsmooth analysis, which are widely used in the sequel In Chapter 2, we introduce the approximate KKT condition for a continuously differentiable multiobjective problem in finite-dimensional spaces, whose feasible set is defined by inequality and equality constraints We show that, without any constraint qualification, the AKKT condition is a necessary for a local weak efficient solution of the considered problem For convex problems, we prove that the AKKT condition is a necessary and sufficient optimality condition for a global weak efficient solution We also prove that, under some suitable additional conditions, an AKKT condition is also a KKT one We also introduce the notion of enhanced KKT-condition and study the relations with the above concepts Chapter Preliminaries 1.1 Convex functions Definition 1.1 A set X ⊂ Rn is called convex if for all x1 ∈ X and x2 ∈ X it contains all points α.x1 + (1 − α).x2 , < α < Definition 1.2 Let X be a subset of Rn The convex hull of the set X (denoted by conv X) is the intersection of all convex sets containing X Definition 1.3 A set K ⊂ Rn is called a cone if for every x ∈ K and al α > one has αx ∈ K A convex cone is a cone that is a convex set Let R be the set of extended real numbers and defined by R := R ∪ {±∞} With every function f : Rn → R we can associate two sets: the domain domf := {x | f (x) < +∞} and the epigraph epif := {(x, λ) ∈ Rn × R | f (x) ≤ λ} Definition 1.4 A function f is called convex if epif is a convex set Theorem 1.5 A function f is convex if and only if for all x1 and x2 and for all α ∈ [0; 1] we have f (αx1 + (1 − α)x2 ) ≤ αf (x1 ) + (1 − α)f (x2 ) 1.2 1.2.1 Clarke subdifferential Generalization of Derivatives In this section, we give the generalized directional derivative in the sense of Clarke [7], subdifferentials, ε-subdifferentials We start by generalizing the ordinary directional derivative Note that this generalized derivative always exists for locally Lipschitz continuous functions Definition 1.6 Let f : Rn −→ R be locally Lipschitz continuos function at x ∈ Rn The generalized directional derivative of f at x in the direction of d ∈ Rn is defined by f ◦ (x; d) = lim sup y→x t↓0 f (y + td) − f (y) t The following summarizes some basic properties of the generalized directional derivative Theorem 1.7 Let f be locally Lipschitz continuous at x with constant K Then the function d → f ◦ (x; d) is positively homogeneous and subadditive on Rn with |f ◦ (x; d)| ≤ K d Corollary 1.8 If f : Rn → R is locally Lipschitz continuous at x, then the function d → f ◦ (x; d) is convex, its epigraph epif ◦ (x; ·) is a convex cone and we have f ◦ (x; −d) = (−f )◦ (x; d.) Theorem 1.9 Let f be locally Lipschitz continuos at x with constant K Then the function (x, d) → f ◦ (x; d) is upper semicontinuous We are now ready to generalize the subdifferential to nonconvex locally Lipschitz continuous functions In what follows we sometimes refer to this subdifferential as Clarke subdifferential Definition 1.10 Letf : Rn → R be a locally Lipschitz continuous function at a point x ∈ Rn Then the subdifferential of f at x is the set ∂f (x) of vectors ξ ∈ Rn such that ∂f (x) = {ξ ∈ Rn |f ◦ (x; d) ≥ ξ T d for all d ∈ Rn } Each vector ξ ∈ ∂f (x) is called a subgradient of x at x The subdifferential has the same basic properties than in convex case Theorem 1.11 Let f : Rn → R be a locally Lipschitz continuous function at x ∈ Rn with a Lipschitz constant K Then the subdifferential ∂f (x) is nonempty, convex, and compact set such that ∂f (x) ⊆ B (0; K) Theorem 1.12 Let f : Rn → R be a locally Lipschitz continuous function at x ∈ Rn , then f ◦ (x; d) = max ξ T d | ξ ∈ ∂f (x) for all d ∈ Rn Theorem 1.13 Let f be a locally Lipschitz continuous and differentiable at x Then ∇f (x) ∈ ∂f (x) Theorem 1.14 If f is continuously differentiable at x, then ∂f (x) = {∇f (x)} Theorem 1.15 If the functionf : Rn → R is convex, then (i) f (x; d) = f ◦ (x; d) for all d ∈ Rn and (ii) ∂c f (x) = ∂f (x) Theorem 1.16 Let f : Rn → R be a locally Lipschitz continuous at x ∈ Rn Then ∂f (x) = conv{ξ ∈ Rn | ∃(xi ) ⊂ Rn \Ωf such that xi −→ x and 1.2.2 f (xi ) −→ ξ} Subdifferential Calculus In order to maintain equalities instead of inconclusions in subderivation rules we need the following regularity property Definition 1.17 The function f : Rn → R is said to be subdifferentially regular at x ∈ Rn if it locally Lipschitz continuous at x and for all d ∈ Rn the classical directional derivative f (x; d) exists and we have f (x; d) = f ◦ (x; d) (1.1) erties However sequences satisfying (A0)-(A3) are easily obtained As instance, if x0 is a KKT-point (see Definition 2.24) and xk , λk , µk , τ k p l=1 λl = 1, then every sequence r ⊂ Rn × Rp+ × Rm + ì R converging to (x , , à, τ ) satisfy (A0)-(A3) whenever µkj = for sufficiently large k if gj (x0 ) < The reciprocal of Theorem 2.5 is not true, the example is shown as follows: Example 2.7 Consider problem (MOP) with the following data: f1 = x1 − 2x22 , f2 = −x2 , g = −x1 ≤ 0, x0 = (0, 0) , xk = 1 − , k k , λk1 = 1, λk2 = 0, µk = One has λk1 ∇f1 xk + λk2 ∇f2 xk + µk ∇g xk = 0, −4 k → (0, 0) Therefore condition (A0)-(A3) are fulfilled Moreover, (E1) is satisfied with bk = µk g (xk ) = k Condition (E2) is also fulfilled since + f xk − f x0 + µ k g xk = f xk − f x0 + µ k g xk −2 ∀k, k2 = ∀k However x0 is not a local weak efficient solution, because for the feasible points x (t) = (t2 , t) one has f (x (t)) = (−t2 , −t) < f (x0 ) for all t > In the following remark we study the relationships between conditions (E1) and (E2) and other conditions, some of them already considered in the literature Remark 2.8 The following statement are true: (i) Condition (E1) implies the following condition (sign condition, SGN in short): for every k one has (SGN ) µkj gj xk ≥ (∀j = 1, , m) and τik hi xk ≥ (∀i = 1, , r) , (2.10) and, moreover, µkj > ⇔ gj xk > 0, and τik = ⇔ hi xk = (ii) Condition (A0), (E2) and (SGN) imply the following convergence condition: (CV G) µkj gj xk → (∀j = 1, , m) and τik hi xk → (∀i = 1, , r) 17 (2.11) (iii) (CVG) implies the following condition, that we say that sum tending to zero condition: r m τik hi xk → µkj gj xk + (SCZ) i=1 j=1 (iv) (SGN) and (SCZ) imply (CVG) Indeed, part (i) is immediate since µkj gj xk = bk gj xk if gj xk ≥ and µkj gj xk = if gj xk < So µkj gj xk ≥ Similarly, τik hi xk = ck hi xk ≥ The rest of part (i) is obvious For part (ii), let sk := m j=1 k µkj gj xk + we have ≤ sk ≤ fl (x0 ) − fl x r k i=1 τi hi xk Using (SGN) and (E2), for all k and l From here, sk → follows by the continuity of fl and property (A0) Parts (iii)-(iv) are obvious By taking into account Remark 2.8(i), from (E2) it shows that fl xk ≤ fl (x0 ) , l = 1, , p, i.e., the points xk are better than x0 This condition is equivalent to the property given by (22), which is used further on to define a strong EFJ-point As a consequence, if we wish a sequence (xk ) satisfying (E1)-(E2), then we have to look for it in the set defined by the system fl (x) ≤ fl (x0 ) , l = 1, , p Remark 2.9 In some works, in order to define the AKKT condition for scalar optimization problems, several variants of condition (A3) or some of the above ones are used For instance, in [4] the authors use the CAKKT condition, where (A3) is replaced with m r µkj gj xk j=1 τik hi xk + → 0, i=1 which is clearly similar to (CVG) We are going to prove that the reciprocal implications of Remark 2.8 are not true and the invalidity if any assumption is not satisfied in next remark Remark 2.10 (i) (SGN) + k1 , , λk1 = not since g x (ii) (CVG) k + , λk2 11 = (E1) Consider Example 2.6, with x0 = (1, 1) , xk = , µk 11 = = 0, and so bk g With these data, (SGN) 11 k x + = = µk for all bk > is satisfied but (E1) is (E2) even if (SGN) holds Consider Example 2.6, with x0 = (1, 1) , xk = − k1 , , λk1 = , λk2 11 = , µk 11 = 11 One has µk g xk → and (SGN) is satisfied, 18 but (E2) is not since f xk − f x0 + µ k g xk = (iii) (E2) 6k − > ∀k > 11k (CVG) if (SGN) is not satisfied Consider the following data in problem (MOP): f1 = x1 + x2 , f2 = −x1 + x2 , g1 = x21 − x2 ≤ 0, g2 = −2x21 + x2 ≤ 0, x0 = (0, 0) , xk = 0, − k1 , λk1 = 12 , λk2 = 12 , µk1 = + k, µk2 = k One has that (E2) is satisfied since fl xk − fl x0 + µk1 g1 xk + µk2 g2 xk = ∀k, l = 1, 2, However, (CVG) is not satisfied because µk1 g1 xk = (1 + k) k1 = (iv) (SCZ) k + → (CVG) if (SGN) is not satisfied The same data of part (iii) show this fact since µk1 g1 xk + µk2 g2 xk = (1 + k) −1 +k = → 0, k k k so (SCZ) holds but (CVG) does not Theorem 2.5 is extended to multiobjective optimization (and improved in some cases) Theorem 3.3 in Andreani, Mart´ınez, Svaiter [4], Theorem 2.1 in Haeser and Schuverdt [11] and Theorem 2.1 (with I = ∅) in Andreani, Haeser and Mart´ınez [3] Next we found that the reciprocal of Theorem 2.5 is true for convex programs Theorem 2.11 Assume that fl (l = 1, , p) and gj (j = 1, , m) are convex and hi (i = 1, , r) are affine If x0 ∈ S satisfies the AKKT condition and (SCZ) is fulfilled, then x0 is a (global) weak sfficient solution of (MOP) Proof Suppose x0 is not a weak efficient solution Then, there is x ∈ S satisfying fl (x) < fl x0 l = 1, , p (2.12) Let (xk ) and λk , µk , τ k be the sequences that satisfy (A0)-(A3) Without any loss of generality we may supppose that λk → λ0 , with λ0 ≥ and 19 p l=1 λ0l = As fl , gj are convex and hi are affine, for all k one has fl (x) ≥ fl xk + ∇fl xk x − xk ∀l = 1, , p, (2.13) gj (x) ≥ gj xk + ∇gj xk x − xk ∀j = 1, , m, (2.14) x − xk ∀i = 1, , r hi (x) = hi xk + ∇hi xk (2.15) Multiplying (2.13) by λkl , (2.14) by µkj and (2.15) by τik and adding up, we obtain the following results (the first inequality is valid because x ∈ S) p p λkl fl m λkl fl (x) ≥ p m p l=1 i=1 r µkj gj xk + j=1 l=1 τik hi (x) (x) + j=1 λkl fl xk + ≥ where γk = (x) + l=1 l=1 r µkj gj τik hi xk + γk , i=1 m k k + ri=1 τik ∇hi j=1 µj ∇gj x m k k + ri=1 τik hi xk → j=1 µj gj x λkl ∇fl xk + xk → x0 , γk → by (A1) and (2.16) xk x − xk As by (SCZ), taking the limit in (2.16) we obtain p p λ0l fl l=1 λ0l fl x0 (x) ≥ (2.17) l=1 As λ0 ≥ and λ0 = 0, from (2.12) it follows that p l=1 λ0l fl (x) < p l=1 λ0l fl (x0 ), contradiction to (2.17) This theorem is extended to multiobjective optimization Theorem 4.2 in Andreani, Mart´ınez and Svaiter [4] and Theorem 2.2 Haeser and Schuverdt [11] Theorem 2.11 will be clearly by the following example Example 2.12 Consider problem (MOP) with the following data: f1 = x21 + x22 , f2 = −x1 , g = x1 − ≤ 0, x0 = (2, 1) , xk = (2, 1) , λk1 = 0, λk2 = 1, µk = 20 With these data, x0 satisfies AKKT and moreover (SCZ) holds, so by applying Theorem 2.11 we conclude that x0 is a global weak efficient solution The conclusion of Theorem 2.11 cannot be that x0 is a global efficient solution as this example shows, because x0 is not efficient since f (2, 0) ≤ f (x0 ) and f (2, 0) = f (x0 ) 2.2 Relations of the AKKT Condition with Other Optimality Conditions In the next part, we are going to state necessary optimality conditions, which are alternative to the conditions of Theorem 2.5 and the relations with AKKT conditions is studied These conditions, in scalar optimization, are said to be by Bertsekas and Ozdaglar [5] ”enhanced Fritz John” conditions because the associated multipliers have some additional properties (see also Hestenes [10] (Theorem5.7.1)) Definition 2.13 We say that a point x0 ∈ S satisfies the enhanced Fritz John condir tions (EFJ) (or it is an EFJ-point) iff there is (λ, µ, τ ) ∈ Rp+ × Rm + × R such that (EFJ1) p l=1 λl ∇fl (x0 ) + m j=1 µj ∇gj (x0 ) + r i=1 τi ∇hi (x0 ) = 0, (EFJ2) (λ, µ, τ ) = 0, (EFJ3) if the index set J ∪ I is nonempty, in which J = {j : µj = 0} , I = {i : τi = 0} , then there is a sequence xk ⊂ Rp that converges to x0 and is such that, for all k, µj gj xk > 0, ∀j ∈ J, τi hi xk > 0, ∀i ∈ I, gj xk + = o ω xk , ∀j ∈ / J, hi xk = o ω xk (2.18) , ∀i ∈ / I, (2.19) where ω (x) = min gj (x)+ , {|hi (x)|} j∈J i∈I 21 (2.20) We say that x0 satisfies the strong EFJ-condition (or it is a strong EFJ-point) iff it satisfies (EFJ1)-(EFJ3) and, in addition, the sequence (xk ) in (EFJ3) satisfies f xk < f x (2.21) We say that x0 satisfies the weak EFJ-condition (or it is a weak EFJ-point) iff it satisfies (EFJ1)-(EFJ3), but condition (2.19)-(2.20) are removed Remark 2.14 Condition (EFJ3) implies µj = if gj (x0 ) < 0, i.e., the usual complementary condition, since if µj > 0, then in view of (2.18) one has that gj xk > for xk near x0 , and this is a contradiction We say that x0 ∈ S is an EKKT-point by extending the above definition, iff it satisfies the conditions (EFJ1), (EFJ3) and, moreover, λ = instead of (EFJ2) Similarly it is defined a strong EKKT-point and a weak EKKT-point In scalar optimization, multipliers satisfying (EFJ1), (EFJ2) and (EFJ3) with (2.21) are called by Bertsekas and Ozdaglar [5] informative multipliers See [5] for interesting properties of this class of multipliers Let us observe that (EFJ3) with (2.21) says that, whenever (µ, τ ) = 0, the constraints whose indexes are in the set J ∪ I can be violated by a sequence of (infeasible) points xk converging to x0 , that improve all the objectives fl ; the remaining constraints, whose indices not belong to J ∪ I, may also be violated, but the degree of their violation is arbitrarily small relative to the other constraints according to (2.19)-(2.20) Another consequence of (2.18) is the following: if gj (x) ≤ on some neighborhood of x0 , then µj = If we wish to obtain a multiplier rule with λ = 0, then usually a constraint qualification is utilized The following constraint qualification is one of the weakest and was introduced by Hestenes [10] Definition 2.15 We say that x0 ∈ S satisfies the quasi-normality constraint qualifir cation (QNCQ) iff there is not any multiplier (µ, τ ) ∈ Rm + ì R such that (i) (à, ) = 0, (ii) m j=1 µj ∇gj (x0 ) + r i=1 τi ∇hi (x0 ) = 0, (iii) in every neighborhood of x there is a point x ∈ Rn such that µj gj (x) > for all j having µj = and τi hi (x) > for all i having τi = It is well known that quasi-normality implies regularity (see Hestenes [10] (Theorem5.8.1)), i.e., the Abadie constraint qualification is satisfied, and this property implies the existence of KKT multipliers at a local minimum 22 We continue establishing a point satisfying conditions AKKT and (E1)-(E2) which is an EFJ-point, and if additionally quasi-normality holds, then it is an EKKTpoint Theorem 2.16 Suppose that x0 ∈ S satisfies the AKKT condition with the sequences (xk ) and λk , µk , τ k and condition (E1), with ck = bk for all k Then, (i) x0 is an EFJ-point (ii) If, in addition, condition (E2) is satisfied, then x0 is a strong EFJ-point Proof We simultaneously prove parts (i) and (ii) Let tk := and λk ≥ As that tk k k λ ,µ ,τ λk , µk , τ k k k , where λk , µk , τ k is the l1 -norm Then, tk ≥ since p l=1 λkl = = ∀k, without any loss of generality we may suppose 0 → (λ , µ , τ ) with (λ0 , µ0 , τ ) = Clearly λ0 ≥ and µ0 ≥ Dividing (A1) by tk and taking the limit it results (EFJ1) (with (λ, µ, τ ) = (λ0 , µ0 , τ )) In order to check condition (EFJ3), assume that (µ0 , τ ) = and consider set m sk := j=1 µkj gj xk + r i=1 τik hi xk By Remark 2.8(i), one has that (SGN) holds For all k large enough, if µ0j > 0, as µ0j = lim µkj , tk then it follows that µkj > By Remark (2.8)(i), we derive that gj (xk ) > [the first part of 2.18 holds] and so µ0j gj xk > and µkj gj xk > for all k large enough Therefore, taking into account (2.10), one has sk > If τi0 = 0, as τi0 = lim τik tk = lim ck hi (xk ) tk one has hi (xk ) = and the second part of (2.18) follows To prove (2.21), we analyze two possibilities: (a) If τi0 > 0, then τik > and hi (xk ) > 0, so τik hi xk > and τi0 hi xk > for all k large enough In consequence, according with (2.10), one has sk > (b) If τi0 < 0, then τik < and hi (xk ) < 0, so τik hi xk > and τi0 hi xk > for all k large enough In consequence, view of (2.10), one has sk > Therefore, if (µ0 , τ ) = 0, then one has sk > 0, and consequently, making use of (E2), fl (x0 ) − fl xk < for all l = 1, , p and condition (2.21) is proved In order to prove (2.19), assume that in (E1) ck = bk for all k Then, gj xk |τ k | for j = 1, , m and hi xk = bik for i = 1, , r, and so ω xk = min {µj } , {|τi |} j∈J i∈I bk 23 + = µkj bk Therefore gj0 xk ω + (xk ) gj0 xk = + tk ω (xk ) tk = µj0 /tk {minj∈J {µj /tk } , mini∈I {|τi |/tk }} Now, this quotient clearly tends to zero with j0 ∈ / J because µj0 /tk → µ0j0 = and the denominator is equal to εk ≥ α > for some α and for all k large enough, since µkj tk → µ0j > ∀j ∈ J and τik tk → |τi0 | > ∀i ∈ I Remark 2.17 Note that the multipliers given in Theorem 2.5, which are determined by (2.7), satisfy condition (E1) with bk = ck Proposition 2.18 If x0 ∈ S is an EFJ-point (respectively strong EFJ-point or weak EFJ-point) and (QNCQ) holds, then x0 an EKKT-point (respectively strong EKKTpoint or weak EKKT-point) Proof For λ0 = 0, it infers that (QNCQ) does not hold since in every neighborhood of x0 there exists a point xk that does not satisfy condition (iii) of the definition of (QNCQ) Part (ii) of Theorem 2.16, taking into account Proposition 2.18, expands to multiobjective optimization and improves Theorem 2.3 in Haeser and Schuverdt [11], which is true for scalar problems and where it is not required that the multipliers satisfy (2.19)-(2.21) The following result is an immediate consequence of Theorems 2.5 and 2.16 taking into account Remark 2.17 and Proposition 2.18 Theorem 2.19 If x0 ∈ S is a local weak solution of (MOP), then (i) x0 is a strong EFJ-point (ii) If, in addition, (QNCQ) holds at x0 , then x0 is a strong EKKT-point Part (i) of this theorem is extended to multiobjective optimization and improved Theorem 5.7.1 in Hestenes [10] and Proposition 3.3.5 in Bertsekas [6] (for almost, only it is claimed that x0 is a weak EFJ-point) It also extends (partially) Proposition 2.1 in Bertsekas and Ozdaglar [5], in which a scalar problem is considered with equality and inequality constraints, but also with a set constraint Next a notion of approximate KKT type is presented and required boundedness of the multipliers Then we will claim that this condition is stronger than AKKT, and we will study its relations with weak EKKT-points and KKT-points 24 Definition 2.20 We say that the bounded approximate Karush-Kuhn-Tucker condition (BAKKT, in short) is satisfied at a feasible point x0 ∈ S iff there is sequences r xk ⊂ Rn and k , àk , k Rp+ ì Rm + × R such that conditions (A0)-(A3) hold and the sequence λk , µk , τ k is bounded (2.22) Clearly condition (2.22) can be alternative with the convergence condition of the multipliers: the sequence λk , µk , τ k converges to λ0 , µ0 , τ (2.23) Proposition 2.21 If x0 ∈ S satisfies (BAKKT), then x0 satifies also the AKKT condition and, in addition, (CVG) holds Proof We only have to prove condition (CVG), but this follows immediately from (2.23), the continuity of gj and hi and the fact that xk → x0 The converse implication is not true, as the following example can be shown Example 2.22 Consider problem (MOP) with the data f (x1 , x2 ) = x1 and g (x1 , x2 ) = −x31 + x22 Clearly x0 = (0, 0) ia a minimum of f subject to g So by Theorem 2.5, the AKKT condition is satisfied Let us check that (BAKKT) does not hold Assume that there is sequences xk = xk1 , xk2 ∈ R2 and µk ∈ R+ such that xk → x0 and ∇f xk + µk ∇g xk =: ω1k , ω2k → (0, 0) From here, (1, 0) + µk −3 xk1 , 2xk2 1−ω1k xk2 k → Therefore, every solution (µ ) is not ) (xk1 ) bounded, and consequently (BAKKT) is not satisfied at x0 Note that this can also be xk1 3( → +∞ whenever xk1 = and = ω1k , ω2k Therefore there is solution µk = deduced from Theorem 4.4 since x0 is not a KKT-point Let us note that by Theorem 2.25(i), x0 is an EFJ-point, and also that any constraint qualification does not hold because x0 is not a KKT-point In the next theorem, it proves that the converse of Proposition 2.21 is true if a constraint qualification holds at the point x0 Recall: The Mangasarian-Fromovitz constraint qualification (MFCQ) holds at x ∈ S if the gradients ∇h1 (x0 ) , , ∇hr (x0 ) are linearly independent and there 25 exists a vector d ∈ Rn such that ∇gj (x0 ) d < for all j ∈ J (x0 ) and ∇hi (x0 ) d = for all i = 1, , r It is well known that (MFCQ) is similar to the positive linearly independent constraint qualification [10], i.e., the following implication is true: µj ≥ 0, j ∈ J (x0 ) , τ ∈ Rr , r µj ∇gj x τi ∇hi x0 = ⇒ µj = 0, ∀j ∈ J x0 , τ = + j∈J(x0 ) i=1 It is also well-known that (MFCQ) is stronger than the so-called constant positive linear dependence condition implying (QNCQ) [1] Theorem 2.23 Suppose that (MFCQ) holds at x0 Then, for all pair of sequences r xk ⊂ Rn and λk , µk , τ k ⊂ Rp+ × Rm + × R that satisfy (A0)-(A3), one has that the second sequence is bounded Proof Let (sk ) be the sequence defined by p m λkl ∇fl sk := x k r µkj ∇gj + x k j=1 l=1 τik ∇hi xk + → (2.24) i=1 If the sequence µk , τ k is not bounded, then, without any loss of generality we can suppose that tk := µk , τ k → +∞ and that tk µk , τ k converges to (µ0 , τ ) with (µ0 , τ ) = Dividing (2.24) by tk , one has p l=1 λkl ∇fl xk + tk m j=1 µkj ∇gj xk + tk r i=1 τik ∇hi xk tk = sk tk The sequence (λk ) is bounded since λ ≥ and (A2) holds So λk tk → 0, and as sk /tk → 0, taking the limit it results m r µ0j ∇gj x0 + j=1 τi0 ∇hi x0 = (2.25) i=1 It infers from (A3), µ0j = for all j ∈ / J (x0 ) With this observation, (2.25) contradicts (MFCQ) because (µ0 , τ ) = and µ0 ≥ Therefore, the sequence µk , τ k is bounded and so condition (2.22) holds Definition 2.24 We say that a feasible point x0 ∈ S is a KKT-point iff there exist r λk , µk , τ k ⊂ Rp+ × Rm + × R such that 26 p λl ∇fl (x0 ) + (K1) m j=1 l=1 r µj ∇gj (x0 ) + τi ∇hi (x0 ) = 0, i=1 (K2) λ = 0, (K3) µj = if gj (x0 ) < 0, j = 1, , m Theorem 2.25 If x0 ∈ S satisfies (BAKKT), then x0 is a KKT-point Proof Let xk , λk , µk , τ k be the sequences that satisfy (A0)-(A3) and (2.23) Then, taking the limit in (A1) and (A2), we obtain p x p l=1 µ0j ∇gj + l=1 and r m λ0l ∇fl x τi0 ∇hi x0 = + i=1 j=1 / J(x0 ) λ0l = 1, with λ0 ≥ 0, µ0 ≥ The condition µ0j gj (x0 ) = for all j ∈ follows from (A3) The following result is a direct consequence of Theorem 2.23 and 2.25 Theorem 2.26 If x0 ∈ S satisfies the AKKT condition and (MFCQ) holds at x0 , then x0 is a KKT-point It follows by a notion due to Dutta et al [8], given a sequence of positive numbers εk ↓ and points xk → x0 , we say that (xk ) is a sequence εk -KKT-points if there is r λk , µk , τ k ⊂ Rp+ × Rm + × R such that p m λkl ∇fl xk + l=1 r µkj ∇gj xk + j=1 τik ∇hi xk ≤ εk , i=1 µkj gj xk = for j = 1, , m and (A2) holds Note that we not require xk to be feasible It is obvious that, if (xk ) is a sequence of εk -KKT-points, then x0 satisfies the AKKT condition, and so the following result is an immediate consequence of Theorem 2.26 Proposition 2.27 If (xk ) is a sequence of εk -KKT-points and (MFCQ) holds at x0 , then x0 is a KKT-point Theorem 3.2 in Dutta et al [8] is a particular case of Proposition 2.27 These authors only consider a scalar function f and inequality constraints 27 Theorem 2.28 If x0 ∈ S is a KKT-point, then x0 is a weak EKKT-point In addition, the sequence (xk ) satisfying (2.18) also satisfies p p λl f l x k λ l f l x0 < l=1 l=1 Proof In order that we give a unified treatment to the multipliers, one transform the multipliers associated with equality constraints as follows: if τi ≥ 0, gm+1 := hi (x) and µm+1 := τi ; if τi < 0, then we define gm+1 := −hi (x) and µm+1 := −τi > By similar way, (K1) can be written p s λl ∇fl x µj ∇gj x0 = 0, + (2.26) j=1 l=1 where s = m + r Rearranging, we may assume that µ1 , , µq are positive and µq+1 , , µs are zero Let us observe that the indices j satifying (K3) are in the second group Through the following reduction process, we can assume that the gradients ∇g1 (x0 ) , , ∇gq (x0 ) are positive linearly independent Indeed, assume that there exist nonnegative numbers α1 , , αq , not all zero, such that q j=1 αj ∇gj (x0 ) = Then, the multipliers µj = µj − tαj (j = 1, , q) , µj = µj (∀j > q) satisfy (2.26) for all t ∈ R So we can choose t0 > so that µj ≥ 0∀j = 1, , q and µj0 = for some index j0 q So, rearranging we obtain a proper subset of subgradients ∇g1 (x0 ) , , ∇gq (x0 ) and positive multipliers µ1 , , µq with q < q (the rest of multipliers is zero) If q = (i.e., if for t0 , µj = for all j = 1, , q), then (2.26) is satisfied with µj = 0, j = 1, , s, and there is nothing to in order to prove (EFJ3) If q > 0, repeating the process if necessary, we can assume that the gradients ∇g1 (x0 ) , , ∇gq (x0 ) are positive linearly independent So, by a theorem of the alternative, there exists u ∈ Rn such that ∇gj x0 u > ∀j = 1, , q Since gj (x0 ) = 0, there exists δ > such that gj (x0 + tu) > ∀t ∈ ]0, δ[ and for all j = 1, , q Therefore (2.21) is satisfied for this set of multipliers since µj gj (x (t)) > 0∀t ∈ ]0, δ[ and for all j = 1, , q, where x(t) = x0 + tu Let us observe that, if gj = −hi (and µj = −τi > 0), then one has τi hi (x (t)) > This claims that x0 is a 28 weak EKKT-point Finally, if we apply (2.26) to the vector u, then it results that p s λl ∇fl x0 u = − l=1 and so for a suitable δ > we have that µj ∇gj x0 u < 0, j=1 p l=1 λl (fl (x (t)) − fl (x0 )) < Corollary 2.29 The following statements are equivalent for x0 ∈ S: (a) x0 is a KKT-point (b) x0 is a weak EKKT-point (c) x0 satisfies (BAKKT) Proof (a) ⇒ (b) and (c) ⇒ (a) follow from Theorem 2.28 and 2.25, respectively Let us prove the implication (b) ⇒ (c) Assume that x0 is a weak EKKT-point We may suppose that (EFJ1) is satisfied with p l=1 λl = Then, we can choose xk = x0 , λk = λ, µk = µ, τ k = τ , and it is obvious that (BAKKT) is satisfied 29 Bibliography [1] Andreani R., Mart´ınez, J M., Schuverdt, M L.: On the relation between constant positive linear dependence condition and quasinormality constraint qualification J optim Theory Appl 125, 473-483 (2005) [2] Andreani, R., Birgin, E G., Mart´ınez, J M., Schuverdt, M.L.: On augmented Lagrangian methods with general lower-level constraints SIAM J Optim 18, 1286–1309 (2007) [3] Andreani, R., Haeser, G., Mart´ınez, J M.: On sequential optimality conditions for smooth constrained optimization Optimization 60, 627-641 (2011) [4] Andreani, R., Mart´ınez, J M., Svaiter, B F.: A new sequential optimality condition for constrained optimization and algorithmic consequences SIAM J Optim 20, 3533-3554 (2010) [5] Bertsekas, D P., Ozdaglar, A E.: Pseudonormality and a Lagrange multiplier theory for constrained optimization J Optim Theory Appl 114, 287-343 (2002) [6] Bertsekas, D P.: Nonlinear Programming Athena Scientific, Belmont (1999) [7] Clarke, F H.: Optimization and Nonsmooth Analysis Wiley, New York (1983) [8] Dutta, J., Deb, Tulshyan, R., Arora, R.: Approximate KKT points and a proximity measure for termination, J Glob Optim 56, 1463-1499 (2013) [9] Giorgi, G., Jim´enez, B., Novo, V.: Approximate Karush–Kuhn–Tucker condition in multiobjective optimization J Optim Theory Appl 171 (2016), 70–89 [10] Hestenes, M R.: Optimization Theory: The Finite Dimensional Case Wiley, New York (1975) 30 [11] Haeser, G., Schuverdt, M L.: On approximate KKT condition and its extension to continuous variational inequalities J Optim Theory Appl 149, 528-539 (2011) [12] Tuyen, N V., Yao, J -C., Wen, C -F.: A note on approximate Karush–Kuhn–Tucker conditions in locally Lipschitz multiobjective optimization Optim Lett 13 (2019), 163–174 31 ... Approximate Karush- Kuhn- Tucker optimality conditions in multiobjective optimization 11 2.1 Approximate KKT Condition for Multiobjective Optimization... converging to CAKKT points under certain conditions Optimality conditions of CAKKT-type have been recognized to be useful in designing algorithms for finding approximate solutions of optimization... S We now recall the concept of approximate Karush- Kuhn- Tucker condition for the multiobjective problem (MOP) from [9] Definition 2.2 The approximate Karush? ? ?Kuhn? ? ?Tucker condition (AKKT) is satisfied