DSpace at VNU: Generalized Affine Functions and Generalized Differentials

J Optim Theory Appl (2012) 154:321–338 DOI 10.1007/s10957-012-0051-4 Generalized Affine Functions and Generalized Differentials N.T.H Linh · J.-P Penot Received: December 2010 / Accepted: 20 March 2012 / Published online: April 2012 © Springer Science+Business Media, LLC 2012 Abstract We study some classes of generalized affine functions, using a generalized differential We study some properties and characterizations of these classes and we devise some characterizations of solution sets of optimization problems involving such functions or functions of related classes Keywords Colinvex · Colinfine · Generalized differential · Optimization problem · Protoconvex function · Pseudoconvex function · Pseudolinear function · Quasiconvex function Introduction A generalized affine function is a function which is both generalized convex and generalized concave Such functions have been studied in [1–9] Among them are quasiaffine functions, i.e., those functions which are both quasiconvex and quasiconcave Also, among them are pseudoaffine functions, i.e., those functions which are differentiable, pseudoconvex, and pseudoconcave, also called pseudolinear functions (see [4, 5, 7, 9]) Given a bifunction h, the class of h-colinfine functions also belongs to this category (see [8]) These references provide interesting, nontrivial examples of generalized affine functions; in particular, fractional functions are noticeable pseudoaffine functions and quadratic pseudoaffine functions can be characterized Communicated by Jean-Pierre Crouzeix N.T.H Linh ( ) International University, Vietnam National University at Ho Chi Minh City, Quarter 6, Linh Trung Ward, Thu Duc District, Ho Chi Minh City, Vietnam e-mail: honglinh98t1@yahoo.com J.-P Penot Laboratoire J.-L Lions, Université Pierre et Marie Curie, place Jussieu, 75005 Paris, France e-mail: jean-paul.penot@univ-pau.fr 322 J Optim Theory Appl (2012) 154:321–338 It is the purpose of the present paper to introduce and study new concepts of generalized affine functions, as it has been done for generalized convex functions in [10] Here, to define these classes, we use a generalized differential, i.e., a set-valued map ∂f , as a substitute for the derivative of f This concept allows much flexibility as it encompasses several notions of nonsmooth analysis We give some elementary properties and characterizations for these generalized affine functions We also present applications to the characterization of the solution set of a constrained minimization problem Notation and Definitions Throughout this paper, X is a normed vector space (n.v.s.), X ∗ is the dual space of X, C is a nonempty subset of X, and f : C → R We assume that a set-valued map ∂f : C ⇒ X ∗ be given, which stands for a substitute for the derivative of f ; we call it a generalized differential of f As observed in [10], the choice for ∂f is not limited to the subdifferentials of nonsmooth analysis; one can also take the convexificators of [11], the pseudodifferentials of Jeyakumar and Luc (see [12]), and much more We assume that ∂f (x) = ∅ for all x ∈ C, although such an assumption could be relaxed for several results We denote by P (resp., R+ ) the set of positive numbers (resp., nonnegative numbers) and R (resp., R∞ ) stands for R ∪ {−∞, +∞} (resp., R ∪ {+∞}) We recall that the visibility cone V (C, x) of C at x ∈ C is the cone generated by C − x: V (C, x) := P(C − x) := r(c − x) : r ∈ P, c ∈ C The visibility bundle of C is the set V C := (x, u) ∈ C × X : ∃r ∈ P, w ∈ C, u = r(w − x) = {x} × V (C, x) x∈C It contains the radial tangent bundle of C, which is the set T r C := {x} × T r (C, x) x∈C with T r (C, x) := u ∈ X : ∃(rn ) → 0+ , x + rn u ∈ C ∀n We also use the tangent bundle of C, which is the set {x} × T (C, x) T C := x∈C with T (C, x) := u ∈ X : ∃(rn ) → 0+ , (un ) → u, x + rn un ∈ C ∀n The upper and the lower radial derivatives (or upper and lower Dini derivatives) of f at x ∈ C in the direction u ∈ T r (C, x), are defined by D + f (x, u) := lim sup t 0+, x+tu∈C f (x + tu) − f (x) , t J Optim Theory Appl (2012) 154:321–338 D+ f (x, u) := t 323 lim inf 0+, x+tu∈C f (x + tu) − f (x) t We recall the following definitions (see [9, 10, 13]) Definition 2.1 Let f : C ⊂ X → R A function f is said to be (a) ∂f -pseudoconvex at x iff, for all w ∈ C: f (w) < f (x) ⇒ for all x ∗ ∈ ∂f (x) : x ∗ , w − x < (1) (b) ∂f -quasiconvex at x iff, for all w ∈ C: f (w) < f (x) ⇒ for all x ∗ ∈ ∂f (x) : x ∗ , w − x ≤ (2) (c) ∂f -protoconvex at x iff, for all w ∈ C: f (w) ≤ f (x) ⇒ for all x ∗ ∈ ∂f (x) : x ∗ , w − x ≤ (3) We add the word “eventually” when, in the preceding implications, “for all” is changed into “there exists.” Thus, for instance, f is eventually ∂f -pseudoconvex at x iff, for all w ∈ C satisfying f (w) < f (x), there exists x ∗ ∈ ∂f (x) such that x ∗ , w − x < Clearly, f is (eventually) ∂f -pseudoconvex at x f is (eventually) ∂f -protoconvex at x ⇒ f is (eventually) ∂f -quasiconvex at x, (4) ⇒ f is (eventually) ∂f -quasiconvex at x (5) Now we introduce some definitions related to generalized concavity Definition 2.2 Let f : C → R, g := −f and some ∂f : C ⇒ X ∗ , ∂g : C ⇒ X ∗ be given (a) f is said to be (eventually) ∂(−f )-pseudoconcave at x iff, the function g := −f is (eventually) ∂g-pseudoconvex at x (b) f is said to be (eventually) ∂f -∂(−f )-pseudoaffine iff it is both (eventually) ∂f -pseudoconvex and (eventually) ∂(−f )-pseudoconcave It is said to be ∂f -protoaffine iff it is ∂f -protoconvex and ∂(−f )-protoconcave The concepts of (eventual) ∂(−f )-quasiconcavity and (eventual) ∂(−f )-protoconcavity are defined similarly Remark 2.1 (a) If we assume that ∂(−f )(x) = −∂f (x), then f is ∂(−f )-pseudoconcave (resp., ∂(−f )-quasiconcave, ∂(−f )-protoconcave) at x if and only if −f is −∂f -pseudoconvex (resp −∂f -quasiconvex, −∂f -protoconvex) at x (b) When ∂f (x) := Df (x) ∪ −D−f (x), where Df (x) and D−f (x) are subdifferentials of f and −f at x respectively, one has ∂(−f )(x) = −∂f (x) 324 J Optim Theory Appl (2012) 154:321–338 We recall from [8] that, given a bifunction h : V C → R, a function f : C → R is said to be h-colinfine at x ∈ C iff there exists λ : C × C → P such that, for every w ∈ X, f (w) − f (x) = λ(w, x)h(x, w − x) Here we introduce a concept in which the bifunction h is replaced with a generalized differential ∂f The presence of some collinearity property explains the terminology Definition 2.3 (a) A function f : C → R is ∂f -colinfine at x ∈ C iff there exists λx : C × ∂f (x) → P such that for all w ∈ C, x ∗ ∈ ∂f (x), f (w) − f (x) = λx w, x ∗ x ∗ , w − x iff there exists λ : C × Graph(∂f ) → P such that, for all w ∈ C, x ∗ ∈ ∂f (x), f (w) − f (x) = λ w, x, x ∗ x ∗ , w − x (6) (b) A function f : C → R is eventually ∂f -colinfine at x ∈ C iff there exists λ : C × Graph(∂f ) → P such that, for all w ∈ C, there exists x ∗ ∈ ∂f (x) satisfying (6) A function f is said to be ∂f -colinfine (resp., ∂f -pseudoconvex ) on C iff it is ∂f -colinfine (resp ∂f -pseudoconvex ) at each point of C For that reason in Definition 2.3 one may consider λ as defined on C × Graph(∂f ) rather than on C × ∂f (x), although only the values of λ on C × {x} × ∂f (x) play a role Similarly, we have chosen to write Definition 2.3(b) in that way in order to stress the analogy with (a) Note that this definition can be rephrased as: f is eventually ∂f -colinfine iff there exist g : C × C → X ∗ and μ : C × C → P such that, for every w, x ∈ X, g(w, x) ∈ ∂f (x) and f (w) − f (x) = μ(w, x) g(w, x), w − x When C is open, f is differentiable and ∂f := {Df }, f is ∂f -colinfine iff f is pseudoaffine Although the preceding definitions are quite restrictive, they are satisfied in some cases of significant interest Note that if f is a h-colinfine function, where h : V C → R is linear and continuous in its second variable, with (x) := h(x, ·) for all x ∈ C, then f is a ∂f -colinfine function for ∂f := { } and f is an eventually ∂f -colinfine function when ∂f ⊇ { } Hence, every pseudoaffine, differentiable function is ∂f -colinfine where ∂f := {f }, f being Gâteaux derivative of f, and eventually ∂f -colinfine if ∂f ⊇ {f } Remark that the converse is not true in general, as shown in Example 2.3, where f is eventually ∂f -colinfine but f is not quasiconvex, hence f is not pseudoaffine Example 2.1 Let X = Rn and let a, b ∈ X ∗ , α, β ∈ R For C := {x ∈ X : bx + β > 0}, ax+α let f : C → R be given by f (x) = bx+β for all x ∈ C When ∂f (x) := {Df (x)}, a simple calculation shows that f is ∂f -colinfine at x If ∂f (x) ⊇ {Df (x)}, then f is eventually ∂f -colinfine at x In particular, every affine function is colinfine J Optim Theory Appl (2012) 154:321–338 325 Example 2.2 Let the function f : R → R be given by f (x) := x Then f is not pseudoaffine But if ∂f (x) is any nonempty subset of P, then f is ∂f -colinfine at x If ∂f (x) ∩ P = ∅, then f is eventually ∂f -colinfine at x Example 2.3 Let f : R → R be given by f (x) = for x = and f (0) = Let ∂f (x) = {−1, 0, 1} for x = and ∂f (0) = {−1, 1} Then f is eventually ∂f -colinfine but f is not quasiconvex Example 2.4 Let f : P → R be given by f (x) = x x−1 Then Df (x) = x x+1 If ∂f (x) ⊆ P, ∂f (x) = ∅ for all x ∈ P, in particular if ∂f (x) = {Df (x)}, then f is ∂f -colinfine If ∂f (x) ∩ P = ∅ for all x ∈ P, then f is eventually ∂f -colinfine 2 Example 2.5 Let X be a √ n.v.s and let g : X → R be ∂g-colinfine For C := {x ∈ X : g(x) > 0}, let f (x) = g(x) If ∂f ⊆ P∂g, then f is ∂f -colinfine on C If ∂f ∩ P∂g = ∅, then f is eventually ∂f -colinfine on C Example 2.6 More generally, let X be a n.v.s., g : X → R be ∂g-colinfine on C := {x ∈ X : g(x) > 0} and for some p ∈ N, let f (·) := g p (·) := (g(·))p If ∂f ⊆ P∂g, then f is ∂f -colinfine on C If ∂f ∩ P∂g = ∅, then f is eventually ∂f -colinfine on C The following result is similar to [8], Proposition 4, but instead of using a bifunction as a generalized directional derivative, we use an arbitrary generalized differential We use a terminology similar to the one in [8] because the vector v := λ(w, x, x ∗ )(w − x) in the invexity relation (7) is collinear to w − x, a feature close to what occurs in the convex case Proposition 2.1 For any function f : C → R, x ∈ domf and any multimap ∂f : C ⇒ X ∗ , the following assertions are equivalent (a) f is ∂f -pseudoconvex and ∂f -protoconvex at x ∈ C; (b) f is ∂f -colinvex at x in the sense: there exists λ : C × Graph(∂f ) → P such that, for all w ∈ C, x ∗ ∈ ∂f (x), f (w) ≥ f (x) + λ w, x, x ∗ x ∗ , w − x (7) Proof (b) ⇒ (a) is obvious: given w ∈ C, if f (w) < f (x) then x ∗ , w − x < for all x ∗ ∈ ∂f (x), while if f (w) ≤ f (x), then x ∗ , w − x ≤ for all x ∗ ∈ ∂f (x) Hence, f is ∂f -pseudoconvex and ∂f -protoconvex at x (a) ⇒ (b) Let f be ∂f -pseudoconvex and ∂f -protoconvex at x Let w ∈ C and x ∗ ∈ ∂f (x) If x ∗ , w − x = 0, then one has f (w) ≥ f (x) by ∂f -pseudoconvexity of f and one can take λx (w, x ∗ ) = or any element in P If x ∗ , w − x > 0, then f (w) > f (x) Then one can take λx w, x ∗ = f (w) − f (x) > x∗, w − x 326 J Optim Theory Appl (2012) 154:321–338 If x ∗ , w − x < 0, then one can take λx w, x ∗ = max f (w) − f (x) , + 1, x∗, w − x or any r > such that r≥ f (w) − f (x) x∗, w − x A similar result is valid for functions that are eventually colinvex at x, i.e., functions such that, for every w ∈ C, there exists x ∗ ∈ ∂f (x) and λx (w, x ∗ ) ∈ P such that f (w) − f (x) ≥ λx (w, x ∗ ) x ∗ , w − x Proposition 2.2 For any function f : C → R, x ∈ domf and any ∂f : C ⇒ X ∗ , the following assertions are equivalent (a) f is eventually ∂f -pseudoconvex and eventually ∂f -protoconvex at x; (b) f is eventually ∂f -colinvex at x Characterizations of Generalized Affine Functions In the present section, we study some classes of generalized affine functions, which are still more restrictive Their interests lie in their striking properties In particular, they enjoy nice composition properties with the classes introduced in the preceding section Their behaviors will be studied below First, we get some composition properties Proposition 3.1 Let g : C → R be (eventually) ∂g-colinfine, let ϕ : R → R and let f := ϕ ◦ g, ∂f (x) := ∂ϕ(g(x))∂g(x) for some ∂ϕ : R ⇒ R (a) (b) (c) (d) If ϕ If ϕ If ϕ If ϕ is (eventually) ∂ϕ-pseudoconvex, then f is (eventually) ∂f -pseudoconvex is (eventually) ∂ϕ-quasiconvex, then f is (eventually) ∂f -quasiconvex is (eventually) ∂ϕ-protoconvex, then f is (eventually) ∂f -protoconvex is (eventually) ∂ϕ-colinfine, then f is (eventually) ∂f -colinfine Proof (a) Since g is ∂g-colinfine, for all w, x ∈ C and all x ∗ ∈ ∂g(x), there exists λg (w, x, x ∗ ) ∈ P such that g(w) − g(x) = λg (w, x, x ∗ ) x ∗ , w − x Let u := ϕ(x) and v := ϕ(w) Since ϕ is ∂ϕ-pseudoconvex, if there exists xf∗ ∈ ∂f (x) such that xf∗ , w − x ≥ 0, then there exist xg∗ ∈ ∂g(x) and u∗ ∈ ∂ϕ(u) such that xf∗ = u∗ xg∗ Then, since λg (w, x, x ∗ ) > and since g is ∂ϕ-pseudoconvex one has ≤ xf∗ , w − x = u∗ xg∗ , w − x and f is ∂f -pseudoconvex ⇒ u∗ g(w) − g(x) ≥ ⇒ ϕ g(w) ≥ ϕ g(x) J Optim Theory Appl (2012) 154:321–338 327 Now, when g is eventually ∂g-colinfine, for w, x ∈ C, there exist xg∗ ∈ ∂g(x) and μ(w, x) ∈ P such that g(w) − g(x) = μ(w, x) xg∗ , w − x Let u := ϕ(x) and v := ϕ(w), ϕ being eventually ∂ϕ-pseudoconvex If ϕ(g(w)) ≤ ϕ(g(x)), then there exists u∗ ∈ ∂ϕ(u) such that u∗ , v − u ≤ Then, since μ(w, x) > one has the following implications showing that f is eventually ∂f -pseudoconvex: ϕ g(w) ≤ ϕ g(x) ⇒ u∗ , g(w) − g(x) ≤ ⇒ u∗ xg∗ , w − x ≤ Similar proofs can be given for (b), (c), and (d) Using the preceding proposition and Definition 2.3, the following example shows simple constructions of colinfine functions Example 3.1 Let g, h : C → R, g being ∂g-colinfine and h being ∂h-colinfine and let x ∈ C (a) Let f := g + h or f := gh If P∂g = P∂h and if ∂f ⊆ P∂g, then f is ∂f -colinfine and D := {x ∈ C : g(x) > 0} If ∂f ⊆ −P∂g, then f is (b) Let f (·) := g(·) ∂f -colinfine on D Let us turn to properties of colinfine functions The following result is an easy consequence of Propositions 2.1 and 2.2 Proposition 3.2 (a) A function f is ∂f -colinfine iff f is ∂f -colinvex and −f is −∂f -colinvex (b) Suppose that for all x ∈ C, the set ∂f (x) be convex Then f is eventually ∂f -colinvex and −f is eventually −∂f -colinvex, iff f is eventually ∂f -colinfine (c) Let C be open and let f be radially continuous If f is (eventually) ∂f -protoconvex and −f is (eventually) −∂f -protoconvex and ∈ / ∂f (x) for all x ∈ C, then f is (eventually) ∂f -colinfine Proof (a) Let f be ∂f -colinfine Then there exists λ : C × Graph(∂f ) → P such that, for all w, x ∈ C, x ∗ ∈ ∂f (x), f (w) − f (x) = λ(w, x, x ∗ ) x ∗ , w − x Thus, f is ∂f -colinvex and −f is −∂f -colinvex since this function λ will satisfy relation (7) both for f and −f Conversely, let w, x ∈ C Since f is ∂f -colinvex and −f is −∂f -colinvex, there exist λ1 : C × Graph(∂f ) → P and λ2 : C × Graph(∂f ) → P such that, for all w, x ∈ C, x ∗ ∈ ∂f (x), f (w) − f (x) ≥ λ1 w, x, x ∗ x ∗ , w − x , −f (w) + f (x) ≥ λ2 w, x, x ∗ −x ∗ , w − x Thus, if f (w) − f (x) < 0, then x ∗ , w − x < 0, while if f (w) − f (x) > 0, then (x) x ∗ , w − x > 0; hence, in both cases, we can take λ(w, x, x ∗ ) = f (w)−f x ∗ ,w−x > to satisfy relation (6) Now, when f (w) = f (x), then x ∗ , w − x = and one can take any λ(w, x, x ∗ ) ∈ P 328 J Optim Theory Appl (2012) 154:321–338 (b) If f is eventually ∂f -colinfine, then as in the proof of (a), one gets that f is eventually ∂f -colinvex and −f is eventually −∂f -colinvex Conversely, suppose f be eventually ∂f -colinvex and −f be eventually −∂f colinvex: there exist μ1 : C × C → P, μ2 : C × C → P and g1 : C × C → X ∗ , g2 : C × C → X ∗ such that, for all w, x ∈ C, μ2 (w, x) g2 (w, x), w − x ≥ f (w) − f (x) ≥ μ1 (w, x) g1 (w, x), w − x If f (w) − f (x) < 0, then g1 (w, x), w − x < 0; hence to satisfy the relation f (w) − f (x) = μ(w, x) g(w, x), w − x we can take g(w, x) := g1 (w, x), (w)−f (x) > If f (w) − f (x) > 0, then g2 (w, x), w − x > and μ(w, x) = gf1 (w,x),w−x (w)−f (x) > If f (w) = f (x), we can take g(w, x) := g2 (w, x), μ(w, x) = gf2 (w,x),w−x then g1 (w, x), w − x ≤ and g2 (w, x), w − x ≥ and, since ∂f (x) is convex, there exists g3 (w, x) ∈ X ∗ such that g3 (w, x), w − x = and we can take g(w, x) := g3 (w, x) and any μ(w, x) ∈ P (c) Under the assumptions of (c), f is ∂f -pseudoconvex and −f is −∂f pseudoconvex By Proposition 2.1, f is ∂f -colinvex and −f is −∂f -colinvex Hence, from (a), f is ∂f -colinfine Similarly, if f is eventually ∂f -protoconvex and −f is eventually −∂f -protoconvex and ∈ / ∂f (x) for all x ∈ C, then f is (eventually) ∂f -colinfine It may happen that f is eventually ∂f -colinvex and −f is eventually −∂f colinvex, but f is not eventually ∂f -colinfine when ∂f is not convex Example 3.2 Let f : R → R be given by f (x) = for x ≤ and f (x) = x for x ≥ 0, and let ∂f be given by ∂f (x) = {−1, 1} for all x ∈ C := X Then f is eventually ∂f -colinvex and −f is eventually −∂f -colinvex but f is not eventually ∂f -colinfine since f (−1) = f (0) = 0, and 1, −1 − = 0, −1, −1 − = The next assertions follow from Remark 2.1 and Proposition 2.1 Proposition 3.3 (a) If f is ∂f -colinfine on C, then f is ∂f -quasiaffine on C (b) If f is ∂f -colinfine at x ∈ C, then setting ∂(−f )(x) := −∂f (x), f is ∂f protoaffine and ∂f -pseudoaffine at x Now, let us give a partial converse of the preceding result It uses the following assumptions in which x ∈ C: ∀u ∈ T r (C, x), D + f (x, u) > 0, ∗ ∀u ∈ T (C, x), ∀x ∈ ∂f (x) r ∗ ⇒ + x , u ≤ D f (x, u), ∀u ∈ T r (C, x), ∀x ∗ ∈ ∂(−f )(x) ∀u ∈ T r (C, x), ∀x ∗ ∈ ∂f (x) D+ f (x, u) ≥ x ∗ , u ≤ D + (−f )(x, u), D+ f (x, u) ≤ x ∗ , u D+ f (x, u) > 0, (8) (9) (10) (11) J Optim Theory Appl (2012) 154:321–338 329 Obviously, Condition (8) is satisfied when f has a radial derivative at x in all directions u ∈ T r (C, x) Moreover, if C is open, f is differentiable on C and ∂f (x) = {f (x)}, ∂(−f )(x) = {−f (x)} the other conditions are satisfied However, these conditions allow x to be a boundary point of C (which is not assumed to be open in the next results) Let us recall that a function f is radially nonconstant iff one cannot find any proper line segment on which f is constant Proposition 3.4 Let f be ∂f -pseudoaffine on a convex set C Then f is quasiaffine on C Let x ∈ C (a) If f is radially nonconstant and if ∂(−f ) = −∂f , then f is ∂f -colinfine (b) If relation (9) is satisfied, then f is ∂f -colinvex at x In addition to (9), if one of the following two conditions holds, then f is ∂f -colinfine at x (i) ∂(−f )(x) = −∂f (x) and relation (10) is satisfied (ii) Conditions (8), (11) are satisfied at x and, for all u ∈ V (C, x), there exists y ∗ ∈ ∂(−f )(x) such that D+ (−f )(x, u) ≤ y ∗ , u Proof Since f is ∂f -pseudoconvex and −f is ∂(−f )-pseudoconvex, f and −f are quasiconvex by [10], Proposition Thus, f is quasiaffine on C (a) Let us first prove that f is ∂f -protoconvex Suppose that f be not ∂f protoconvex Then there exist some w, x ∈ C such that f (w) ≤ f (x) and some x ∗ ∈ ∂f (x) with x ∗ , w − x > Then f (w) = f (x) On the other hand, since f is quasiconvex and radially nonconstant, there exists z ∈ (w, x) such that f (z) < f (x) and then x ∗ , z − x < 0, a contradition Similarly, we get −f is ∂(−f )-protoconvex On the other hand, since ∂(−f ) = −∂f , −f is −∂f -protoconvex and −f is −∂f -pseudoconvex Hence, f is ∂f -colinvex and −f is −∂f -colinvex By Proposition 2.2(a), f is ∂f -colinfine at x (b) When relation (9) holds, we see that f is ∂f -protoconvex at x, since if f (w) ≤ f (x) for some w ∈ C, then D + f (x, w − x) ≤ by quasiconvexity and, for all x ∗ ∈ ∂f (x), we have x ∗ , w − x ≤ D + f (x, w − x) ≤ Thus, since f is ∂f -pseudoconvex, it is ∂f -colinvex at x by Proposition 2.1 Let us first consider the case (i): ∂(−f )(x) = −∂f (x) and relation (10) is satisfied Since f is ∂f -pseudoaffine, f is ∂(−f )-pseudoconcave, hence by Definition 2.2, −f is −∂f -pseudoconvex Now, let us prove that f is ∂(−f )-protoconcave at x If −f (w) ≤ −f (x) for some w ∈ C, then, for all x ∗ ∈ ∂(−f )(x), one has x ∗ , w − x ≤ D + (−f )(x, w − x) ≤ by relation (10), and the fact that −f is quasiconvex Thus, −f is −∂f -protoconvex at x and −f is −∂f -colinvex at x By Proposition 2.2(a), f is ∂f -colinfine at x Finally, let us suppose Assumption (ii) be satisfied Let us prove that −f is −∂f -protoconvex if relation (11) is satisfied If −f (w) ≤ −f (x) for some w ∈ C, then f (w) ≥ f (x) and D+ f (x, w − x) ≥ since f is quasiaffine For any x ∗ ∈ ∂f (x), relation (11) implies ≤ D+ f (x, w − x) ≤ x ∗ , w − x Thus, one has −x ∗ , w − x ≤ and −f is −∂f -protoconvex at x 330 J Optim Theory Appl (2012) 154:321–338 Now, let us prove that −f is −∂f -pseudoconvex Since f is ∂f -pseudoaffine, −f is ∂(−f )-pseudoconvex Let w ∈ C be such that −f (w) < −f (x) and let u := w −x Then we have y ∗ , w − x < for all y ∗ ∈ ∂(−f )(x) Let us pick y ∗ ∈ ∂(−f )(x) such that D+ (−f )(x, u) ≤ y ∗ , u Thus, we have D + f (x, w − x) > Since f is quasiaffine and f (w) > f (x), we also have D+ f (x, w − x) ≥ Then Condition (8) entails < D+ f (x, w − x) Since relation (11) is satisfied, for all x ∗ ∈ ∂f (x) we have < D+ f (x, w − x) ≤ x ∗ , w − x Thus, we get −x ∗ , w − x < for all −x ∗ ∈ −∂f (x), and then −f is −∂f -pseudoconvex Thus, −f is −∂f -colinvex at x and then f is colinfine at x Let us give a similar result for an eventually colinfine function at some x ∈ C It uses the following assumptions, which are weaker than the preceding assumptions: ∀u ∈ T r (C, x) ∃x ∗ ∈ ∂f (x), x ∗ , u ≤ D + f (x, u), (12) ∀u ∈ T r (C, x) ∃x ∗ ∈ ∂(−f )(x), x ∗ , u ≤ D + (−f )(x, u), (13) ∗ ∀u ∈ T (C, x) ∃x ∈ ∂f (x), r ∗ D+ f (x, u) ≤ x , u (14) Proposition 3.5 Let f be ∂f -pseudoaffine on a convex set C and let x ∈ C (a) If f is radially nonconstant and if ∂(−f )(x) ∩ −∂f (x) = ∅ for all x ∈ C then f is eventually ∂f -colinfine (b) If relation to (12) is satisfied, then f is eventually ∂f -colinvex In addition, if ∂f (x) is convex and one of the following conditions holds, then f is eventually ∂f -colinfine at x (i) ∂(−f )(x) ∩ −∂f (x) = ∅ and relation (10) is satisfied (ii) ∂(−f )(x) = −∂f (x) and relation (13) is satisfied (iii) Conditions (8), (14) are satisfied at x and, for all u ∈ T r (C, x) there exists y ∗ ∈ ∂(−f )(x) such that D+ (−f )(x, u) ≤ y ∗ , u Proof of this proposition is similar to the previous one Remark 3.1 (a) Note that, if we change “≤” into “ 0, w ∗ , w − x = 0, w ∗ , w − x < 0, w ∗ , w − x ≤ 0), then there exists x ∗ ∈ ∂f (x) such that x ∗ , w − x ≥ (resp., x ∗ , w − x > 0, x ∗ , w − x = 0, x ∗ , w − x < 0, x ∗ , w − x ≤ 0) (b) ∂f is upper sign-continuous on C The preceding proposition can easily be adapted to the case f is ∂f -colinfine by interchanging “for all” and “there exists.” Moreover, in such a case, one sees ∂f and −∂f are pseudomonotone (in the sense of [13]) Thus, we get a multivalued generalization of the concept of PPM map which has been used in [15, 16] to study variational inequalities The following result is similar to [10], Proposition 10 Proposition 3.7 If f is ∂f -colinfine, then the function t → f (x + t (w − x)) is either increasing or decreasing or constant on the interval on which it is defined Proof Since f is ∂f -colinfine, there exists λ : C × Graph(∂f ) → P such that for every w, x ∈ C, for any x ∗ ∈ ∂f (x), f (w) − f (x) = λ(w, x, x ∗ ) x ∗ , w − x If f (w) = f (x) then, one has f (x + t (w − x)) = f (x) for all t ∈ R such that xt := x + t (w − x) ∈ C If f (w) > f (x) then, for all x ∗ ∈ ∂f (x), one has x ∗ , w − x > For all t > such that xt ∈ C, by homogeneity, one gets x ∗ , xt − x > Thus f (xt ) > f (x) Also, given u := x + r(w − x) ∈ C and v := x + s(w − x) ∈ C with s > r, one has f (v) > f (u) Otherwise, one would have either f (v) = f (u) and then f (w) = f (x) by what precedes, or f (v) < f (u) and then, for all v ∗ ∈ ∂f (v), v ∗ , u − v > 0, hence v ∗ , x − v > since x − v = q(u − v) for some q > 0, and then f (x) > f (v), a contradiction with f (v) = f (xs ) > f (x) Similarly, for every t > 0, we have f (w + t (x − w)) < f (w) and for s > r, f (w + s(x − w)) < f (w + r(x − w)) when the involved points are in C If f (x) > f (w), then we also have the conclusion by interchanging the roles of w and x in what precedes In the sequel, we will consider the following hypothesis: (H+ ) If x ∗ , w − x > for some w, x ∈ C, x ∗ ∈ ∂f (x), then there exists z ∈]w, x[ such that f (z) > f (x) (H− ) If x ∗ , w − x < for some w, x ∈ C, x ∗ ∈ ∂f (x), then there exists z ∈]w, x[ such that f (z) < f (x) (H0 ) If x ∗ , w − x = for some w, x ∈ C, x ∗ ∈ ∂f (x), then one has f (w) = f (x) 332 J Optim Theory Appl (2012) 154:321–338 Hypothesis (H+ ) (resp (H− )) is weaker than ∂f -protoconvexity of f (resp −∂f protoconvexity of −f ) Hypothesis (H+ ) (resp (H− )) is clearly satisfied when ∂f (x) (resp −∂f (x)) is contained in the Dini–Hadamard (or contingent) subdifferential (resp ∂ D (−f )(x) of −f at x) The following theorem has been given in [2, 3, 5, 6], in the case the function f is differentiable and ∂f := {f } Here, the function f is nonsmooth In [8], a generalized directional derivative is used instead of a generalized differential We start with a preparatory lemma dealing with the one-dimensional case Lemma 3.1 Let C be an open interval of R and let ϕ : C → R be a continuous function satisfying (H+ ) and (H− ) with respect to ∂ϕ Then ϕ is ∂ϕ-colinfine iff either (H0 ) is satisfied and there exists some t0 ∈ C such that ∂ϕ(t0 ) 0, or for all t ∈ C, the sign of t ∗ ∈ ∂ϕ(t) is constant on the level set Lϕ (t) := ϕ −1 (ϕ(t)) Proof Let ϕ be ∂ϕ-colinfine If there exists t0 ∈ C such that ∈ ∂ϕ(t0 ), then, by the proof of the preceding proposition, ϕ is constant on C Thus, (H0 ) is satisfied, and since ϕ is ∂ϕ-colinfine, one has t ∗ = for any t ∗ ∈ ∂ϕ(t) and any t ∈ C Otherwise, for all t ∈ C, t ∗ ∈ ∂ϕ(t), one has t ∗ = Thus, by Proposition 3.7, ϕ is increasing or decreasing on C and then the level set Lϕ (t) of ϕ at t ∈ C is the singleton {t} Now, let t1∗ , t2∗ ∈ ∂ϕ(t) and let s ∈ C, s > t Since ϕ is ∂ϕ-colinfine, there exist λ1 (s, t, t1∗ ) ∈ P and λ2 (s, t, t2∗ ) ∈ P such that ϕ(s) − ϕ(t) = λ1 s, t, t1∗ t1∗ , s − t = λ2 s, t, t2∗ t2∗ , s − t Thus, t1∗ t2∗ > For the converse, let us first suppose (H0 ) be satisfied and for some t0 ∈ C, we have ∂ϕ(t0 ) {0} Then, for all s ∈ C, we have t0∗ , s − t0 = for t0∗ := so that, by (H0 ), ϕ(s) = ϕ(t0 ) Hence, ϕ is constant and, in view of conditions (H+ ) and (H− ), for all r ∈ C, r ∗ ∈ ∂ϕ(r), we must have r ∗ = 0: then ϕ is trivially ∂ϕ-colinfine Now suppose that, for all t ∈ C, t ∗ ∈ ∂ϕ(t), the sign of t ∗ be constant on Lϕ (t) Given r ∈ C such that for all r ∗ ∈ ∂ϕ(r), r ∗ > 0, we will prove that ϕ(t) > ϕ(r) for all t > r Let us first prove that we cannot have ϕ(t) < ϕ(r) If ϕ(t) < ϕ(r), let s := sup{p ∈ [r, t] : ϕ(p) = ϕ(r)} Since ϕ is continuous, we have s < t and ϕ(s) = ϕ(r) Then our assumption ensures that, for all s ∗ ∈ ∂ϕ(s), we have s ∗ > 0, so that by condition (H+ ), there exists some p ∈]s, t[ with ϕ(p) > ϕ(s) and the intermediate value theorem yields some q ∈ [p, t] with ϕ(q) = ϕ(s) = ϕ(r), a contradiction with the definition of s Now let us suppose that ϕ(t) = ϕ(r) Then t ∗ > for any t ∗ ∈ ∂ϕ(t) by our assumption, so that t ∗ , r − t < Then condition (H− ) yields some t ∈]r, t[ such that ϕ(t ) < ϕ(t) = ϕ(r), and replacing t by t in what precedes, we get again a contradiction A similar proof shows that ϕ(q) < ϕ(r) for all q < r We can see in a similar way that when for some r ∈ C, and all r ∗ ∈ ∂ϕ(r), we have r ∗ < 0, the function ϕ is decreasing on C Thus, for all r, s ∈ C, s ∗ ∈ ∂ϕ(s), there exists some λ(r, s, s ∗ ) ∈ P such that ϕ(r) − ϕ(s) = λ(r, s, s ∗ ) s ∗ , r − s J Optim Theory Appl (2012) 154:321–338 333 Theorem 3.1 Let C be an open, convex subset of X, let f : C → R be radially continuous and let ∂f : C ⇒ X ∗ with nonempty values (a) Let f be ∂f -colinfine Then, for all w ∈ C, the set Kw := u ∈ X : ∀w ∗ ∈ ∂f (w), w ∗ , u = is a linear subspace of X and ∂f (w) is contained in a half-line (b) If f is ∂f -colinfine on C then, for all w, x ∈ C such that f (w) = f (x), one has Kx = Kw and for any w ∗ ∈ ∂f (w),x ∗ ∈ ∂f (x), there exists some r > such that w ∗ = rx ∗ (c) Conversely, suppose that f and ∂f satisfy (H0 ), (H+ ), and (H− ) If for any w, x ∈ C such that f (w) = f (x) and for any w ∗ ∈ ∂f (w), x ∗ ∈ ∂f (x), there exists some r > such that w ∗ = rx ∗ , then f is ∂f -colinfine (d) Assume that the dimension of X is greater than 1, that C = X, and that f and ∂f satisfy (H0 ), (H+ ), and (H− ) Then f is ∂f -colinfine iff, for all w, x ∈ C and all w ∗ ∈ ∂f (w), x ∗ ∈ ∂f (x) there exists some r > such that w ∗ = rx ∗ Proof (a) It is obvious that Kw is a linear subspace Let w ∗ , x ∗ ∈ ∂f (w) For any u ∈ X such that w ∗ , u = 0, taking t > small enough, we have w + tu ∈ C and f (w + tu) = f (w) Since f is ∂f -colinfine, we obtain x ∗ , u = Thus, ker w ∗ = ∗ w∗ x∗ w∗ ∗ ∗ Kw and xx ∗ = w ∗ or x ∗ = − w ∗ if w and x are both nonnull, while if one of them is 0, the other one is 0, also In other terms, there exists some r ∈ R\{0} such that w ∗ = rx ∗ Let us prove that r is positive when x ∗ = Let y ∈ C be such that x ∗ , y − x = Since f is ∂f -colinfine, one has f (y) − f (x) = λ y, x, x ∗ x ∗ , y − x = λ y, x, w ∗ w ∗ , y − x , hence we get ≤ x ∗ , y − x w ∗ , y − x = x ∗ , y − x rx ∗ , y − x = r x ∗ , y − x and r > (b) Now let w, x ∈ C be such that f (w) = f (x) and let u ∈ Kw Since f is ∂f colinfine, we have x − w ∈ Kw , hence x − w + tu ∈ Kw for all t ∈ R since Kw is a linear subspace Thus, for |t| small enough, we have x + tu ∈ C and f (x + tu) = f (w) = f (x) Therefore, x ∗ , u = for all x ∗ ∈ ∂f (x) and u ∈ Kx So, Kw ⊂ Kx The symmetry of the roles of w and x yields Kw = Kx Let w ∗ ∈ ∂f (w), x ∗ ∈ ∂f (x) By (a) and the preceding case, we have ker w ∗ = Kw = Kx = ker x ∗ and so there exists some r ∈ R\{0} such that w ∗ = rx ∗ Let us prove that r is positive Suppose that r < Pick u ∈ X such that x ∗ , u = and set z1 = w + tu and z2 = x + tu, with t > small enough to ensure that z1 , z2 ∈ C Then, since f is ∂f -colinfine, one has f (z1 ) < f (w) = f (x) < f (z2 ) and x ∗ , w − x = Since f is radially continuous, there exists some s ∈]0, 1[ such that, for z := sz1 + (1 − s)z2 , one has f (z) = f (x) Then z − x = sw − sx + tu and = x ∗ , z − x = x ∗ , sw − sx + tu = t > 0, a contradiction Hence, the case r < is excluded (c) If there exists x ∈ C such that ∂f (x) 0, then one has f (y) = f (x) for all y ∈ C, by condition (H0 ) since x ∗ , y − x = Thus, the level set of f at x is C and our assumption ensures that for all y ∗ ∈ ∂f (y), for some r > 0, we have y ∗ = rx ∗ = Thus, f is ∂f -colinfine 334 J Optim Theory Appl (2012) 154:321–338 Thus, we can assume that ∈ / ∂f (x) for all x ∈ C Now, given w, x ∈ C, for t ∈ Cw,x := {t ∈ R : xt ∈ C}, where xt := (1 − t)x + tw, let us set ϕ(t) := f (xt ) For t ∈ Cw,x , let us take ∂ϕ(t) := t ∗ ∈ R : ∃xt∗ ∈ ∂f (xt ), t ∗ = xt∗ , w − x Let us show that (ϕ, ∂ϕ) satisfies (H+ ), (H− ) and (H0 ) Suppose that there exist s, t ∈ Cw,x , t ∗ ∈ ∂ϕ(t) such that t ∗ , s − t > By definition of ∂ϕ(t), there exists xt∗ ∈ ∂f (xt ) such that xt∗ , xs − xt = (s − t)t ∗ > Since f satisfies (H+ ), there exists q between s and t such that ϕ(q) := f (xq ) > f (xt ) := ϕ(t) Thus, ϕ satisfies (H+ ) With a similar proof, we can see that ϕ satisfies (H− ) Now suppose that t ∗ , s − t = for t, s ∈ Cw,x , with s = t and t ∗ ∈ ∂ϕ(t) Then there exists xt∗ ∈ ∂f (xt ) such that xt∗ , xs − xt = (s − t)t ∗ = Since f satisfies (H0 ), one has ϕ(s) := f (xs ) = f (xt ) := ϕ(t), hence ϕ satisfies (H0 ) / ∂ϕ(t) Let t, s ∈ Cw,x be such that ϕ(t) = ϕ(s) Suppose that, for all t ∈ Cw,x , ∈ and let t ∗ ∈ ∂ϕ(t), s ∗ ∈ ∂ϕ(s) We shall prove that t ∗ s ∗ > Let xt∗ ∈ ∂f (xt ) and xs∗ ∈ ∂f (xs ) be such that t ∗ = xt∗ , w − x and s ∗ = xs∗ , w − x Since f (xt ) = ϕ(t) = ϕ(s) = f (xs ), there exists some r > such that xs∗ = rxt∗ , and we have s ∗ = xs∗ , w − x = r xt∗ , w − x = rt ∗ Hence, we get t ∗ s ∗ > Thus, either there exists some t ∈ Cw,x such that ∈ ∂ϕ(t) or, for all t ∈ C, the sign of t ∗ ∈ ∂ϕ(t) is constant on the level set Lϕ (t) := ϕ −1 (ϕ(t)), hence ϕ is ∂ϕcolinfine, by the preceding lemma Moreover, for any w, x ∈ C, for any x ∗ ∈ ∂f (x), we have s ∗ := x ∗ , w − x ∈ ∂ϕ(0) and f (w) − f (x) = ϕ(1) − ϕ(0) = λϕ 1, 0, s ∗ s ∗ , − 0) = λ w, x, x ∗ x ∗ , w − x , where λ(w, x, x ∗ ) := λϕ (1, 0, s ∗ ) > 0; thus f is a ∂f -colinfine function (d) Suppose C = X The sufficiency is clear by (c) Conversely, suppose that f be ∂f -colinfine Given w, x ∈ C, w ∗ ∈ ∂f (w) and x ∗ ∈ ∂f (x), let prove there exists some r > such that w ∗ = rx ∗ If x ∗ = 0, since f is ∂f -colinfine, then f is constant and so w ∗ = 0: thus the conclusion holds with r = in this case or in the case w ∗ = When w ∗ = 0, x ∗ = 0, the subspaces Kw and Kx are the hyperplanes ker w ∗ and ker x ∗ , respectively, so that the dimension of X being greater than 1, there exists z ∈ (w + Kw ) ∩ (x + Kx ) One has z − w ∈ Kw , z − x ∈ Kx , hence w ∗ , z − w = and x ∗ , z − x = Since f is ∂f -colinfine, we have f (x) = f (z) = f (w) Then (b) yields the existence of some r > satisfying w ∗ = rx ∗ Note that Lemma 17 and Theorem 18 in [8] follow from the preceding lemma and the preceding theorem by taking ∂f (x) := { (x)} When ∂f (x) := {Df (x)}, [2], Theorems 4.13, 4.14 is a consequence of the preceding theorem We devote the rest of this section to a study of the structure of continuous colinfine functions on a finite dimensional n.v.s X When X is one-dimensional, it is easy to see that f is quasiaffine iff, it is either nondecreasing or nonincreasing Thus, we suppose that the dimension of X is greater than The differentiable case is considered in [2, 5, 6] as a Df -colinfine function f is pseudoaffine (or pseudolinear in the J Optim Theory Appl (2012) 154:321–338 335 terminology of these papers) The nonsmooth case is dealt with in [8] with the help of a bifunction Here, we use a generalized differential ∂f of f and we assume that some natural composition rules are satisfied We suppose that, when f = g ◦ for some nonnull continuous linear form on X, and some continuous function g : R → R, one the following conditions is satisfied: (C1) ∂g( (x)) ◦ ⊆ ∂(g ◦ )(x); (C2) ∂(g ◦ )(x) ⊆ ∂g( (x)) ◦ Conditions (C1) and (C2) are satisfied for the Fréchet and the Hadamard subdifferentials We deduce from [8, 9] a characterization of continuous ∂f -colinfine functions Proposition 3.8 Let f : X → R be a lower semicontinuous (resp., continuous), nonconstant function on a finite dimensional n.v.s If Condition (C1) is satisfied, then assertion (a) below implies assertion (b); if Condition (C2) is satisfied, the reverse implication holds: (a) f is ∂f -colinfine; (b) there exist a continuous linear form on X, = 0, and a lower semicontinuous (resp., continuous) increasing, ∂g-colinfine function g : R → R such that f = g◦ Proof (a) ⇒ (b) Let Condition (C1) be satisfied and let f : X → R be l.s.c (resp., continuous) and ∂f -colinfine Hence, f is quasiaffine by Proposition 3.3 According to [9], there exist a continuous linear form on X and a lower semicontinuous (resp., continuous) nondecreasing function g : R → R such that f = g ◦ Since f is nonconstant, = Let us prove that g is increasing Suppose that there exist r1 < r2 in R such that, for all r ∈ [r1 , r2 ], g(r) = g(r1 ) = g(r2 ) Since f is nonconstant, g is nonconstant and there exists some t0 > 1, such that, for t ≥ t0 , one has either g(r1 + t (r2 − r1 )) > g(r2 ) or g(r2 + t (r1 − r2 )) < g(r1 ) We consider the first case, the second one being similar Since = 0, there exist w, x ∈ X such that (w) = r1 < (x) = r2 Then, for t ≥ t0 , we have (w + t (x − w)) > (x) = r2 and then f w +t (x −w) = g w +t (x −w) = g r1 +t (r2 −r1 ) > g(r2 ) = f (w) = f (x) By Proposition 3.7, f is not ∂f -colinfine Therefore, g is increasing Now, we shall prove that g is ∂g-colinfine For any u, v ∈ R, and v ∗ ∈ ∂g(v), by Condition (C1), there exists w, x ∈ X and x ∗ ∈ ∂f (x) such that (w) = u, (x) = v and v ∗ ◦ = x ∗ Hence, there exists λf (w, x, x ∗ ) ∈ P such that g( (w)) − g( (x)) g(u) − g(v) = = x ∗ , w − x = v ∗ , (w) − (x) = v ∗ , u − v λf (w, x, x ∗ ) λf (w, x, x ∗ ) so that g is ∂g-colinfine (b) ⇒ (a) Let Condition (C2) be satisfied and let w, x ∈ X Then, for any x ∗ ∈ ∂f (x), there exists u∗ ∈ ∂g( (x)) such that x ∗ = u∗ ◦ Since g is ∂g-colinfine, there 336 J Optim Theory Appl (2012) 154:321–338 exists λg ( (w), (x), u∗ ) ∈ P such that f (w) − f (x) g( (w)) − g( (x)) = ∗ λg ( (w), (x), u ) λg ( (w), (x), u∗ ) = u∗ , (w) − (x) = u∗ ◦ , w − x = x ∗ , w − x , hence, f is ∂f -colinfine Characterizations of Solution Sets In the present section, we consider the minimization problem (C) f (x) subject to x ∈ C, where C is a subset of a n.v.s X and f : C → R Our purpose here is limited: since optimality conditions for (C) and mathematical programming problems using the concepts of the previous sections are dealt with in [8, 10, 13, 17–19], we are just concerned with characterizations of the solution set S of (C) Let ∂f : C ⇒ X ∗ be a generalized differential of f The following results are direct consequences of Definition 2.3 Here, we set Sa := x ∈ C : ∃a ∗ ∈ ∂f (a), a ∗ , x − a = , Sa := x ∈ C : ∀a ∗ ∈ ∂f (a), a ∗ , x − a = Proposition 4.1 Let f be ∂f -colinfine on C (a) If a ∈ S, then for all x ∈ C, a ∗ ∈ ∂f (a), x ∗ ∈ ∂f (x), one has a ∗ , x − a ≥ 0, and x ∗ , a − x ≤0 Moreover, S = Sa = Sa (b) Conversely, if for all x ∈C, either there exists a ∗ ∈∂f (a) such that a ∗ , x − a ≥0, or there exists x ∗ ∈ ∂f (x) such that x ∗ , a − x ≤ 0, then a ∈ S Proposition 4.2 Let f be an eventually ∂f -colinfine function on C (a) If a ∈S, then for all x ∈C, there exists some a ∗ ∈∂f (a) such that a ∗ , x − a ≥0, and there exists some x ∗ ∈ ∂f (x) such that x ∗ , a − x ≤ Moreover, Sa ⊂ S ⊂ Sa (b) Conversely, if for all x ∈ C, either for all a ∗ ∈ ∂f (a) one has a ∗ , x − a ≥ 0, or for all x ∗ ∈ ∂f (x) one has x ∗ , a − x ≤ then a ∈ S Let us consider now the case in which the constraint set C is defined by a finite family of inequalities, so that problem (C) turns into the mathematical programming problem (M) f (x) subject to x ∈ C := x ∈ X : g1 (x) ≤ 0, , gn (x) ≤ , where f : W → R, gi : W → R and W is a subset of X J Optim Theory Appl (2012) 154:321–338 337 Let a ∈ C, let I := {1, , n} and let I (a) := {i ∈ I : gi (a) = 0} The following result follows from [8] Note that here instead of colinvexity of the objective function f , we only need that f be ∂f -protoconvex Proposition 4.3 Let a ∈ S be such that there exists y = (yi ) ∈ Rn+ , a ∗ ∈ ∂f (a), ai∗ ∈ ∂gi (a) for i ∈ I (a) satisfying a ∗ + i∈I (a) yi ai∗ = 0, and yi gi (a) = for i ∈ I (a) If either f is ∂f -colinfine at a or f is ∂f -protoconvex at a and gi is ∂gi protoconvex at a (i ∈ I (a)), then i∈I (a) yi ai∗ , b − a = for every b ∈ S (b) Let f be ∂f -protoconvex at a and gi be ∂gi -colinvex at a (i ∈ I (a)) Let L be the Lagrangian given by L(x, u) := f (x) + ug(x) for (x, u) ∈ C × Rn+ and let b ∈ C Then b ∈ S iff yg(b) = and L(a, y) = L(b, y) Proof (a) Let a ∗ ∈ ∂f (a), ai∗ ∈ ∂gi (a) for i ∈ I (a), y ∈ Rn+ and b ∈ S be as in the statement If f is ∂f -colinfine at a, then for μ := λ(b, a, a ∗ ) > one has = a∗, b − a + yi ai∗ , b − a i∈I (a) = f (b) − f (a) + μ yi ai∗ , b − a i∈I (a) yi ai∗ , b − a = i∈I (a) Now, assume that f be ∂f -protoconvex at a and gi be ∂gi -protoconvex at a (i ∈ I (a)) Since f (b) = f (a), one has a ∗ , b − a ≤ Thus, i∈I (a) yi ai∗ , b − a ≥ On the other hand, since gi (b) ≤ gi (a) = for i ∈ I (a), one has ai∗ , b − a ≤ Thus, we get i∈I (a) yi ai∗ , b − a = (b) If gi is ∂gi -colinvex at a (i ∈ I (a)), then ≥ gi (b) = gi (b) − gi (a) ≥ λi ai∗ , b − a for some λi > Set λ := max{λi : i ∈ I (a)} Then, for all i ∈ I (a), one has gi (b) ≥ λ ai∗ , b − a By (a), since yi ∈ R+ , for b ∈ S, one gets ≥ yg(b) ≥ λ i∈I (a) yi ai∗ , b −a = Moreover, we have L(a, y) = L(b, y) since f (a) + yg(a) = f (a) = f (b) = f (b) + yg(b) Now, for the converse, let b ∈ C be such that yg(b) = and L(a, y) = L(b, y) Then we have b ∈ S since f (a) = L(a, y) = L(b, y) = f (b) + yg(b) = f (b) Conclusions The main result of this paper is the presentation of characterizations of some new classes of generalized affine functions, the class of ∂f -colinfine functions, and the class of eventually ∂f -colinfine functions using a generalized differential ∂f Further, characterizations of solution sets of a constrained minimization problem using these concepts are given 338 J Optim Theory Appl (2012) 154:321–338 References Aggarwal, S., Bhatia, S.: Pseudolinearity and efficiency via Dini derivatives Indian J Pure Appl Math 20(12), 1173–1183 (1989) Cambini, A., Martein, L.: Generalized convexity and optimality conditions in scalar and vector optimization In: Hadjisavvas, N., Komlosi, S., Schaible, S (eds.) Handbook of Generalized Convexity and Generalized Monotonicity, pp 151–194 Kluwer Academic, Amsterdam (2005) Chew, K.L., Choo, E.V.: Pseudolinearity and efficiency Math Program 28, 226–239 (1984) Jeyakumar, V., Yang, X.Q.: On characterizing the solution sets of pseudolinear programs J Optim Theory Appl 87(3), 747–755 (1995) Komlosi, S.: First and second characterizations of pseudolinear functions Eur J Oper Res 67, 278– 286 (1993) Kortanek, K.O., Evans, J.P.: Pseudoconcave programming and lagrange regularity Oper Res 15(5), 891–892 (1967) Kruk, S., Wolkowicz, H.: Pseudolinear programming SIAM Rev 41(4), 795–805 (1999) Linh, N.T.H., Penot, J.-P.: Generalized affine maps and generalized convex functions Pac J Optim 4(2), 353–380 (2008) Penot, J.-P.: Glimpses upon quasiconvex analysis ESIAM: Proc 20, 170–194 (2007) 10 Linh, N.T.H., Penot, J.-P.: Generalized convex functions and generalized differentials Optim., J Math Program Oper (2011) doi:10.1080/02331934.2011.611882 11 Demyanov, V.F., Rubinov, A.M.: Contructive Nonsmooth Analysis Verlag Peter Lang, Frankfurt am Main (1995) 12 Jeyakumar, V., Luc, D.T.: Nonsmooth Vector Functions and Continuous Optimization Springer, Berlin (2007) 13 Penot, J.-P.: Are generalized derivatives useful for generalized convex function In: Crouzeix, J.-P., Volle, M., Martinez-Legaz, J.-E (eds.) Generalized Convexity, Generalized Monotonicity, pp 3–39 Kluwer Academic, Amsterdam (1998) 14 Aussel, D., Hadjisavvas, N.: On quasimonotone variational inequalities J Optim Theory Appl 121(2), 445–450 (2004) 15 Bianchi, M., Schaible, S.: An extension of pseudolinear functions and variational inequalities problems J Optim Theory Appl 104(1), 59–71 (2000) 16 Bianchi, M., Hadjisavvas, N., Schaible, S.: On pseudomonotone maps T for which −T is also pseudomonotne J Convex Anal 10(1), 149–168 (2003) 17 Linh, N.T.H., Penot, J.-P.: Optimality conditions for quasiconvex programming SIAM J Optim 17(2), 500–510 (2006) 18 Penot, J.-P.: Characterization of solution sets of quasiconvex programs J Optim Theory Appl 117(3), 627–636 (2003) 19 Penot, J.-P.: A Lagrangian approach to quasiconvex analysis J Optim Theory Appl 117(3), 637–647 (2003) ... -pseudoconvex and eventually ∂f -protoconvex at x; (b) f is eventually ∂f -colinvex at x Characterizations of Generalized Affine Functions In the present section, we study some classes of generalized affine. .. exists some r > such that w ∗ = rx ∗ , then f is ∂f -colinfine (d) Assume that the dimension of X is greater than 1, that C = X, and that f and ∂f satisfy (H0 ), (H+ ), and (H− ) Then f is ∂f... convex and one of the following conditions holds, then f is eventually ∂f -colinfine at x (i) ∂(−f )(x) ∩ −∂f (x) = ∅ and relation (10) is satisfied (ii) ∂(−f )(x) = −∂f (x) and relation (13) is satisfied