Hindawi Publishing Corporation EURASIP Journal on Advances in Signal Processing Volume 2009, Article ID 730968, 10 pages doi:10.1155/2009/730968 Research Article Extended LaSalle’s Invariance Principle for Full-Range Cellular Neural Networks Mauro Di Marco, Mauro Forti, Massimo Grazzini, and Luca Pancioni Depar tment of Information Engineering, University of Siena, 53100 - Siena, Italy Correspondence should be addressed to Mauro Di Marco, dimarco@dii.unisi.it Received 15 September 2008; Accepted 20 February 2009 Recommended by Diego Cabello Ferrer In several relevant applications to the solution of signal processing tasks in real time, a cellular neural network (CNN) is required to be convergent, that is, each solution should tend toward some equilibrium point. The paper develops a Lyapunov method, which is based on a generalized version of LaSalle’s invariance principle, for studying convergence and stability of the differential inclusions modeling the dynamics of the full-range (FR) model of CNNs. The applicability of the method is demonstrated by obtaining a rigorous proof of convergence for symmetric FR-CNNs. The proof, which is a direct consequence of the fact that a symmetric FR-CNN admits a strict Lyapunov function, is much more simple than the corresponding proof of convergence for symmetric standard CNNs. Copyright © 2009 Mauro Di Marco et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. 1. Introduction The Full-Range (FR) model of cellular neural networks (CNNs)hasbeenintroducedin[1]inordertoobtain advantages in the VLSI implementation of CNN chips with a large number of neurons. One main feature is the use of hard-limiter nonlinearities that constrain the evolution of the FR-CNN trajectories within a closed hypercube of the state space. This improved range of the trajectories has enabled us to reduce the power consumption and obtain higher cell densities and increased processing speed [1–4] compared to the original standard (S)CNNmodelbyChua and Yang [5]. In several applications for solving signal processing tasks in real time it is needed that a FR-CNN is convergent (or completely stable), that is, each solution is required to approach some equilibrium point in the long-run behavior [5–7]. For example, given a two-dimensional image, a CNN is able to perform contour extraction and morphological operations, noise filtering, or motion detection, during the transient motion toward an equilibrium point [8]. Other rel- evant applications of convergent FR-CNN dynamics concern the solution of optimization or identification problems or the implementation of nonlinear electronic devices for pattern formation [9, 10]. An FR-CNN is characterized by ideal hard-limiter non- linearities with vertical segments in the i-v characteristic, hence its dynamics is mathematically described by a differen- tial inclusion, where a set-valued vector field models the set of feasible velocities for each state of the FR-CNN. A recent paper [11] has been devoted to the rigorous mathematical foundation of the FR model within the framework of the theory of differential inclusions [12].Thegoalofthispaper is to extend the results in [11] by developing a generalized Lyapunov approach for addressing stability and convergence of FR-CNNs. The approach is based on a suitable notion of derivative of a (candidate) Lyapunov function and a generalized version of LaSalle’s invariance principle for the differential inclusions modeling the FR-CNNs. The Lyapunov method developed in the paper is for- mulated in a general fashion, which makes it suitable to check if a continuously differentiable (candidate) Lyapunov function is decreasing along the solutions of a FR-CNNs, and to verify if this property in turn implies convergence of each FR-CNN solution. The applicability of the method is demonstrated by obtaining a rigorous convergence proof for the important and widely used class of symmetric FR- CNNs. It is shown that the proof is more simple than the proofofananalogousconvergenceresultin[11], which 2 EURASIP Journal on Advances in Signal Processing is not based on an invariance principle for FR-CNNs. The same proof is also much more simple than the proof of convergence for symmetric S-CNNs. We refer the reader to [13] for other applications of the method to classes of FR- CNNs with nonsymmetric interconnection matrices used in the real-time solution of some classes of global optimization problems. The structure of the paper is briefly outlined as follows. Section 2 introduces the FR-CNN model studied in the paper, whereas Section 3 gives some fundamental properties of the solutions of FR-CNNs. The extended LaSalle’s invari- ance principle for FR-CNNs and the convergence results for FR-CNNs are described in Sections 4 and 5,respectively. Section 6 discusses the significance of the convergence results and, finally, Section 7 draws the main conclusions of the paper. Notation. Let R n be the real n-space. Given matrix A ∈ R n×n , by A we mean the transpose of A.Inparticular,byE n we denote the n × n identity matrix. Given the column vectors x, y ∈ R n ,wedenotebyx, y= n i =1 x i y i the scalar product of x and y, while x= x, x is the Euclidean norm of x. Sometimes, use is made of the norm x ∞ = max i=1,2, ,n |x i |. Given a set D ⊂ R n ,bycl(D) we denote the closure of D, while dist(x, D) = inf y∈D x − y is the distance of vector x ∈ R n from D.ByB(z, r) ={y ∈ R n : y − z <r} we mean an n-dimensional open ball with center z ∈ R n and radius r. 1.1. Preliminaries 1.1.1. Tangent and Normal Cones. This section reports the definitions of tangent and normal comes to a closed convex set and some related properties that are used throughout the paper. The reader is referred to [12, 14, 15] for a more thorough treatment. Let Q ⊂ R n be a nonempty closed convex set. The tangent cone to Q at x ∈ Q is given by [14, 15] T Q (x) = v ∈ R n : lim inf ρ →0 + dist(x + ρv, Q) ρ = 0 ,(1) while the normal cone to Q at x ∈ Q is defined as N Q (x) = p ∈ R n : p, v≤0, ∀v ∈ T Q (x) . (2) The orthogonal set to N Q (x)isgivenby N ⊥ Q (x) = v ∈ R n : p, v=0, ∀p ∈ N Q (x) . (3) From a geometrical point of view, the tangent cone is a generalization of the notion of the tangent space to a set, which can be applied when the boundary is not necessarily smooth. In particular, T Q (x) is the closure of the cone formed by all half lines originating at x and intersecting Q in at least one point y distinct from x. The normal cone is the dual cone of the tangent cone, that is, it is formed by all directions with an angle of at least ninety degrees with any direction belonging to the tangent cone. It is known that T Q (x)andN Q (x) are nonempty closed convex cones in R n , which possibly reduce to the singleton {0}.Moreover,N ⊥ Q (x) is a vector subspace of R n ,andwehaveN ⊥ Q (x) ⊂ T Q (x). The next property holds [11]. Property 1. If Q coincides with the hypercube K = [−1, 1] n , then N K (x), T K (x), and N ⊥ K (x) have the following analytical expressions. For any x ∈ K we have N K (x) = H(x) = (h(x 1 ), h(x 2 ), , h(x n )) ,(4) where h(ρ) = ⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ (−∞,0], ρ =−1, 0, ρ ∈ (−1, 1), [0, + ∞), ρ = 1, (5) whereas T K (x) = H T (x) = (h T (x 1 ), h T (x 2 ), , h T (x n )) ,(6) with h T (ρ) = ⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ [0, +∞), ρ =−1, ( −∞,+∞), ρ ∈ (−1, 1), ( −∞,0], ρ = 1. (7) Finally, for any x ∈ K we have N ⊥ K (x) = H ⊥ (x) = (h ⊥ (x 1 ), h ⊥ (x 2 ), , h ⊥ (x n )) ,(8) where h ⊥ (ρ) = ⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ 0, ρ =−1, ( −∞,+∞), ρ ∈ (−1, 1), 0, ρ = 1. (9) The above cones, evaluated at some points of the set K = [−1, 1] 2 , are reported in Figure 1. Let Q ⊂ R n be a nonempty closed convex set. The orthogonal projector onto Q is a mathematical operator which associates to any x ∈ R n the set P Q (x), composed by the points of Q that are closest to x,namely, x −P Q (x)=dist(x, Q) = min y∈Q y −x. (10) Under the considered assumptions, P Q (x) always contains exactly one point. The name derives from the fact that x − P Q (x) ∈ N Q (x). 2. CNN Models and Motivating Results The dynamics of the S-CNNs, introduced by Chua and Yang in the fundamental paper [5], can be described by the differential equations: ˙ x(t) =−x(t)+AG(x(t)) + I,(S) EURASIP Journal on Advances in Signal Processing 3 x 2 1 N ⊥ K (b) = 0 N K (b) b T K (b)K −1 T K (a) = N ⊥ K (a) a −1 T K (c) 1 x 1 N ⊥ K (c) N K (c) c Figure 1: Set K = [−1, 1] 2 and cones T K , N K ,andN ⊥ K at points a, b, and c of K (the cones are shown translated into the corresponding points of K). Point a belongs to the interior of K, and hence T K (a) is the whole space R 2 , while N Q (a)reducesto{0}. Point b coincides with a vertex of K,andsoT K (b) corresponds to the third quadrant of R 2 , while N K (b) corresponds to the first quadrant of R 2 . Finally, point c belongs to the right edge of the square and, consequently, T K (c) coincides to the left half plane of R 2 , while N K (c) coincides with the nonnegative part of x 1 axis. where x ∈ R n is the vector of neuron state variables; A ∈ R n×n is the neuron interconnection matrix; I ∈ R n is the constant input; G(x) = (g(x 1 ), g(x 2 ), , g(x n )) : R n → R n , where the piecewise-linear neuron activation g is given by g(ρ) = 1 2 ( |ρ +1|−|ρ −1|) = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ 1, ρ>1, ρ, −1 ≤ ρ ≤ 1, −1, ρ<−1, (11) see Figure 2(a). It is convenient to define A = A −E n , (12) which is the matrix of the affine system satisfied by (S) in the linear region |x i | < 1, i = 1, 2, , n. The improved signal range (ISR) model of CNNs has been introduced in [1, 2] with the goal to obtain advan- tages in the electronic implementation of CNN chips. The dynamics of an ISR-CNN can be described by the differential equations: ˙ x(t) =−x(t)+AG(x(t)) + I − mL(x(t)), (I) where m ≥ 0, L(x) = ((x 1 ), (x 2 ), , (x n )) : R n → R n and (ρ) = ⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ ρ −1, ρ ≥ 1, 0, −1 <ρ<1, ρ +1, ρ ≤−1, (13) see Figure 2(b). When the slope m of the nonlinearity m( ·) is large, m( ·) plays the role of a limiter device that prevents the state variables x i of (I) from exceedingly enter the saturation regions where |x i (t)| > 1. The larger m, the smaller the neighborhood of the hypercube: K = [−1, 1] n , (14) where the state variables x i areconstrainedtoevolveforall large t. A particularly interesting limiting situation is that where m → +∞, in which case m(·) approaches the ideal hard- limiter nonlinearity h( ·)givenin(5); see Figure 2(c).The hard-limiter h( ·) now constrains the state variables of (F)to evolve within K, that is, we have |x i (t)|≤1forallt and for all i = 1, 2, , n. Since for x ∈ K we have x = G(x), (I) becomes the FR model of CNNs [1, 2, 11]: ˙ x(t) ∈−x(t)+Ax(t)+I −H(x(t)), (F) where H(x) = (h(x 1 ), h(x 2 ), , h(x n )) ,andh isgivenin(5). From a mathematical viewpoint, h(ρ) is a set-valued map assuming the entire interval of values [0, + ∞)(resp., ( −∞,0]) at ρ = 1(resp.,ρ =−1). As a consequence, the vector field defining the dynamics of (F), −x+Ax+I−H(x), is a set-valued map assuming multiple values when some state variable x i is saturated at x i =±1, which represent the set of feasible velocities for (F) at point x.AnFR-CNNisthus described by a differential inclusion as in (F)[11, 12]and not by an ordinary differential equation. In [16], Corinto and Gilli have compared the dynamical behavior of (S)(m = 0), with that of (I)(m 0) and (F)(m → +∞), under the assumption that the three models are characterized by the same set of parameters (interconnections and inputs). It is shown in [16] that there are cases where the global behavior of (S)and(I)isnot qualitatively similar for the same set of parameters, due to bifurcations in model (I) occurring for some positive values of m. In particular, a class of completely stable, second-order S-CNNs (S) has been considered, and it has been shown that, for the same parameters, (I) displays a heteroclinic bifurcation at some m = m β > 0, which leads to the birth of a stable limit cycle for any m>m β . In other words, (I)is not completely stable for m>m β , and the same holds for (F), which is the limit of (I)asm → +∞. The result in [16] has the important consequence that in the general case the stability of model (F) cannot be deduced from existing results on stability of (S). Hence, it is needed to develop suitable tools, which are based on the theory of differential inclusions, for studying in a rigorous way the stability and convergence of FR-CNNs. The goal of this paper is to develop an extended Lyapunov approach for addressing stability and convergence of FR-CNNs. The approach is based on a suitable notion of derivative and an extended version of LaSalle’s invariance principle for the differential inclusion (F)modelingaFR- CNN. 4 EURASIP Journal on Advances in Signal Processing g(ρ) 1 −1 ρ1 −1 (a) m(ρ) −1 ρ 1 (b) h(ρ) −1 ρ 1 (c) Figure 2: Nonlinearities used in the CNN models (S), (I), and (F). 3. Solutions of FR-CNNs To the authors knowledge, [11] has been the first paper giving a foundation with the theory of differential inclusions of the FR model of CNNs. One main property noted in [11] is that we have H(x) = N K (x), (15) for all x ∈ K, that is, H(x) coincides with the normal cone to K at point x (cf. Property 1). Therefore, (F)canbewrittenas ˙ x(t) ∈−x(t)+Ax(t)+I −N K (x(t)), (16) which represents a class of differential inclusions termed differential variational inequalities (DVIs) [12, Chapter 5]. Let x 0 ∈ K. A solution of (F)in[0, t ], with initial condition x 0 ,isafunctionx satisfying [12]: (a) x(t) ∈ K for t ∈ [0, t ]andx(0) = x 0 ;(b)x is absolutely continuous on [0, t ], and for almost all (a.a.) t ∈ [0, t ]wehave ˙ x(t) ∈ − x(t)+Ax(t)+I − N K (x(t)). By an equilibrium point (EP) we mean a constant solution x(t) = ξ ∈ K, t ≥ 0, of (F). Note that ξ ∈ K is an EP of (F) if and only if there exists γ ξ ∈ N K (ξ) such that 0 =−ξ + Aξ + I − γ ξ ,orequivalently, we have (A −E n )ξ + I ∈ N K (ξ). By exploiting the theory of DVIs, the next result has been proved in [11]. Property 2. For any x 0 ∈ K, there exists a unique solution x of (F) with initial condition x(0) = x 0 , which is defined for all t ≥ 0. Moreover, there exists at least an EP ξ ∈ K of (F). We w il l deno te by E / =∅ the set of EPs of (F). It can be shown that E is a compact subset of K. It is both of theoretic and practical interest to compare the solutions of the ideal model (F) with those of model (I). The next result shows that the solutions of (F) are the uniform limit, as the slope m → +∞, of the solutions of model (I). Property 3. Let x(t), t ≥ 0, be the solution of (F) with initial condition x(0) = x 0 ∈ K.Moreover,foranym = k = 1, 2, 3, ,letx k (t), t ≥ 0, be the solution of model (I)such that x k (t) = x 0 .Then,x k (·) converges uniformly to x(·)on any compact interval [0, T] ⊂ [0, +∞), as k → +∞. Proof. See Appendix A. 4. LaSalle’s Invariance Pr i nciple for FR-CNNs Consider the system of ordinary differential equations: ˙ x = f (x), (17) where x ∈ R n ,and f : R n → R n is continuously differ- entiable. Let φ : R n → R be a continuously differentiable (candidate) Lyapunov function, and consider the vector field: δ(x) =f (x), ∇φ(x), (18) for all x ∈ R n . From the standard Lyapunov method for ordinary differential equations [17], it is known that for all times t the derivative of φ along a solution x of (17)canbe evaluated from δ as follows: d dt φ(x(t)) = δ(x(t)). (19) Such a treatment cannot be directly applied to the differential inclusion (16) modeling the dynamics of a FR- CNN, since the vector field at the right-hand side of (16) assumes multiple values when some component x i of x assumes the values ±1. In what follows our goal is to introduce a suitable concept of derivative, which generalizes the definition of δ, for evaluating the time evolution of a candidate Lyapunov function along the solutions of the differential inclusion (16).Then,weproveaversionof LaSalle’s invariance principle generalizing to the differential inclusions (16) the classic version for ordinary differential equations [17]. In doing so, we need to take into account that the limiting sets for the solutions of (16)enjoyaweaker invariance property with respect to the solutions of the standard differential equations defined by a continuously differentiable vector field. We begin by introducing the following definition of derivative. EURASIP Journal on Advances in Signal Processing 5 f (c) P T K (c) f (c) c b f (b) x 2 1 K f (d) d −1 a −1 f (a) 1 x 1 P T K (e) f (e) f (e) e Figure 3: Vector fields involved in the definition of the derivative Dφ forasecond-orderFR-CNN.Let f (x) = Ax + I.Wehave P T K (x) f (x) ∈ N ⊥ K (x), hence Dφ(x) is a singleton, when x is one of the points a, d, e ∈ K. On the other hand, P T K (x) f (x) / ∈N ⊥ K (x)and then Dφ(x) = ∅, when x is one of the points b, c ∈ K. Definition 1. Let φ : R n → R be a continuously differen- tiable function in R n .ThederivativeDφ(x)offunctionφ at a point x ∈ K is given by Dφ(x) = P T K (x) (Ax + I), ∇φ(x) , (20) if P T K (x) (Ax + I) ∈ N ⊥ Q (x), while Dφ(x) = ∅, (21) if P T K (x) (Ax + I) / ∈N ⊥ K (x). We stress that, for any x ∈ K, Dφ(x) is either the empty set or a singleton. These two different cases are illustrated in Figure 3 for a second-order FR-CNN. Moreover, if ξ ∈ E, then we have Dφ(ξ) = 0. Indeed, we have Aξ + I ∈ N K (ξ), and then P T Q (ξ) (Aξ + I) = 0 ∈ N ⊥ Q (ξ). Moreover, P T Q (ξ) (Aξ +I), ∇φ(ξ)=0, ∇φ(ξ)=0andsoDφ(ξ) = 0. Definition 2. Let φ : R n → R be a continuously differ- entiable function in R n . We say that φ is a Lyapunov function for (F), if we have Dφ(x) = ∅ or Dφ(x) ≤ 0, for any x ∈ K. If, in addition, we have Dφ(x) = 0 if and only if x is an EP of (F), then φ is said to be a strict Lyapunov function for (F). The next fundamental property can be proved. Property 4. Let φ : R n → R be a continuously differentiable function in R n , and let x(t), t ≥ 0, be a solution of (F). Then, for a.a. t ≥ 0wehave d dt φ(x(t)) = Dφ(x(t)). (22) If φ is a Lyapunov function for (F), then for a.a. t ≥ 0we have d dt φ(x(t)) = Dφ(x(t)) ≤ 0, (23) hence φ(x(t)) is a nonincreasing function for t ≥ 0, and there exists the lim t →+∞ φ(x(t)) = φ ∞ > −∞. Proof. The function φ(x(t)), t ≥ 0, is absolutely continuous on any compact interval in [0, + ∞), since it is the compo- sition of a continuously differentiable function φ and an absolutely continuous function x. Then, for a.a. t ≥ 0we have that x( ·)andφ(x(·)) are differentiable at t.By[12,page 266, Proposition 2] we have that for a.a. t ≥ 0 ˙ x(t) ∈ P T K (x(t)) (Ax(t)+I). (24) Let t>0 be such that x is differentiable at t. Let us show that ˙ x(t) ∈ N ⊥ K (x(t)). Let h>0, and note that since x(t)and x(t + h)belongtoK,wehave dist(x(t)+h ˙ x(t), K) ≤x(t)+h ˙ x(t) −x(t + h). (25) Dividing by h, and accounting for the differentiability of x at time t,weobtain lim h →0 + dist(x(t)+h ˙ x(t), K) h = 0, (26) and hence we have ˙ x(t) ∈ T K (x(t)). Now, suppose that h ∈ (−t, 0). Since once more x(t)and x(t + h)belongtoK,wehave 0 ≤ dist(x(t)+(−h)(− ˙ x(t)), K) −h ≤ (x(t)+h ˙ x(t) −x(t + h) −h . (27) Let ρ =−h.Then, lim ρ →0 + dist(x(t)+ρ(− ˙ x(t)), K) ρ = 0, (28) and hence, by definition, − ˙ x(t) ∈ T K (x(t)). Now, it suffices to observe that T K (x) ∩ (−T K (x)) = N ⊥ K (x)foranyx ∈ K. In fact, if v ∈ T K (x) ∩ (−T K (x)) and p ∈ N K (x), then v, p≤0and−v, p≤0. This means that v, p=0, that is, v ∈ N ⊥ K (x). Conversely, if v ∈ N ⊥ K (x)andp ∈ N K (x), then we have v, p=0and−v, p=0. Hence v ∈ T K (x) ∩(−T K (x)). For a.a. t ≥ 0wehave d dt φ(x(t)) = ˙ x(t), ∇φ(x(t)) = P T K (x(t)) (Ax(t)+I), ∇φ(x(t)) , (29) and hence, by Definition 1, d dt φ(x(t)) = Dφ(x(t)). (30) 6 EURASIP Journal on Advances in Signal Processing Now, suppose that φ is a Lyapunov function for (F). Then, for a.a. t ≥ 0wehave d dt φ(x(t)) = Dφ(x(t)) ≤ 0, (31) and hence φ(x(t)), t ≥ 0, is a monotone nonincreasing function. Moreover, φbeing a continuous function, it attains a minimum over the compact set K. Since we have x(t) ∈ K for all t ≥ 0, the function φ(x(t)), t ≥ 0, is bounded from below, and there exists the lim t →+∞ φ(x(t)) = φ ∞ > −∞. It is important to stress that, as in the standard Lyapunov approach for differential equations, Dφ permits to evaluate dφ(x(t))/dt for a.a. t ≥ 0 directly from the vector field Ax+I, without involving integrations of (F) (see Property 4). We are now in a position to prove the next extended version of LaSalle’s invariance principle for FR-CNNs. Theorem 1. Let φ : R n → R be a continuously differentiable function in R n ,whichisaLyapunovfunctionfor(F).Let Z ={x ∈ K : Dφ(x) = 0},andletM be the largest positively invariant subset of (F) in cl(Z). Then, any s olution x(t), t ≥ 0,of (F) converges to M as t → +∞,thatis, lim t →+∞ dist(x(t), M) = 0. Proof. Consider the differential inclusion ˙ x ∈ F r (x) = Ax + I −[N K (x) ∩cl(B(0, r))], (32) where + ∞ >r>sup K Ax + I and F r from K into R n is an upper-semicontinuous set-valued map with nonempty compact convex values. By [11, Proposition 5] we have that if x(t), t ≥ 0, is a solution of (F), then x is also a solution of (32)fort ≥ 0. Denote by ω x the ω-limit set of the solution x(t), t ≥ 0, that is, the set of points y ∈ R n such that there exists a sequence {t k },witht k → +∞ as k → +∞, such that lim k →+∞ x(t k ) = y. It is known that ω x is a nonempty compact connected subset of K,andx(t) → ω x as t → +∞ [18, pages 129, 130]. Furthermore, due to the uniqueness of the solution with respect to the initial conditions (Property 2), ω x is positively invariant for the solutions of (F)[18, pages 129, 130]. Now, it suffices to show that ω x ⊆ M. It is known from Property 4 that φ(x(t)), t ≥ 0, is a nonincreasing function on [0, + ∞)andφ(x(t)) → φ(∞) > −∞ as t → +∞.For any y ∈ ω x , there exists a sequence {t k },witht k → +∞ as k → +∞, such that x(t k ) → y as k → +∞. From the continuity of φ,wehaveφ(y) = lim t k →+∞ φ(x(t k )) = φ(∞), hence φ is constant on ω x . Let y 0 ∈ ω x and let y(t), t ≥ 0, be the solution of (F) such that y(0) = y 0 . Since ω x is positively invariant, we have y(t) ⊆ ω x for t ≥ 0. It follows that φ(y(t)) = φ(∞) for t ≥ 0 and hence, by Property 4, for a.a. t ≥ 0wehave 0 = dφ(y(t))/dt = Dφ(y(t)). This means that y(t) ∈ Z for a.a. t ≥ 0. Hence, y(t) ∈ cl(Z)forallt ≥ 0. In fact, if we had y(t ∗ ) / ∈cl(Z)forsomet ∗ ≥ 0, then we could find δ>0such that y([t ∗ , t ∗ + δ)) ∩Z = ∅,whichisacontradiction.Now, note that in particular we have y 0 = y(0) ∈ cl(Z). y 0 being an arbitrary point of ω x , we conclude that ω x ⊂ cl(Z). Finally, since ω x is positively invariant, it follows that ω x ⊆ M. 5. Convergence of Symmetric FR-CNNs In this section, we exploit the extended LaSalle’s invariance principle in Theorem 1 in order to prove convergence of FR- CNNs with a symmetric neuron interconnection matrix. Definition 3. The FR-CNN (F)issaidtobequasiconvergent if we have lim t →+∞ dist(x(t), E) = 0foranysolutionx(t), t ≥ 0, of (F). Moreover, (F)issaidtobeconvergentifforany solution x(t), t ≥ 0, of (F) there exists an EP ξ such that lim t →+∞ x(t) = ξ. Suppose that A = A is a symmetric matrix, and consider for (F) the (candidate) quadratic Lyapunov function φ(x) =− 1 2 x Ax −x I, (33) where x ∈ R n . Property 5. If A = A , then for function φ as in (33)wehave Dφ(x) =− P T K (x) (Ax + I) 2 ≤ 0, (34) if P T K (x) (Ax + I) ∈ N ⊥ K (x), while Dφ(x) = ∅, (35) if P T K (x) (Ax + I) / ∈N ⊥ K (x). Furthermore, Dφ(x) = 0ifand only if x is an EP of (F), that is, φ is a strict Lyapunov function for (F). Proof. Let x ∈ K and suppose that P T K (x) (Ax + I) ∈ N ⊥ K (x). Observe that ∇φ(x) =−(Ax + I). Moreover, since N K (x)is the negative polar cone of T K (x)[12, page 220, Proposition 2], we have [12, page 26, Proposition 3] Ax + I = P T K (x) (Ax + I)+P N K (x) (Ax + I), (36) with P T K (x) (Ax + I), P N K (x) (Ax + I)=0. Accounting for Definition 1,wehave Dφ(x) = P T K (x) (Ax + I), ∇φ(x) = P T K (x) (Ax + I), −P T K (x) (Ax + I) + P T K (x) (Ax + I), −P N K (x) (Ax + I) =− P T K (x) (Ax + I) 2 ≤ 0. (37) Hence, φ is a Lyapunov function for (F). It remains to show that it is strict. If x is an EP of (F), then we have P T K (x) (Ax + I) = 0 and hence Dφ(x) = 0. Conversely, if Dφ(x) = 0, then we have P T K (x) (Ax + I)=0. Thus, x is an EP for (F). Property 5 and Theorem 1 yield the following. Theorem 2. Suppose that A = A .Then,(F) is quasiconver- gent, and it is convergent if the EPs of (F) are isolated. Proof. Since φ is a strict Lyapunov function for (F), we have Z = E.LetM be the largest positively invariant set of (F) contained in Z. Due to the uniqueness of the solutions for (F)(Property 2), it follows that E ⊆ M. On the other hand, EURASIP Journal on Advances in Signal Processing 7 E is a closed set and hence E = cl(E) = cl(Z) ⊇ M. In conclusion, M = E.Then,Theorem 1 implies that any solution x(t), t ≥ 0, of (F)convergestoE as t → +∞. Hence (F) is quasiconvergent. Suppose in addition that the equilibrium points of (F) are isolated. Observe that ω x is a connected subset of M = E. This implies that there exists ξ ∈ E such that ω x = ξ. Since x(t) → ω x ,wehavex(t) → ξ as t → +∞. 6. Remarks and Discussion Here, we discuss the significance of the result in Theorem 2 by comparing it with existing results in the literature on convergence of FR-CNNs and S-CNNs. Furthermore, we briefly discuss the possible extensions of the proposed Lyapunov approach to neural network models described by more general classes of differential inclusions. (1) Theorem 2 coincides with the result on convergence obtained in [11, Theorem 1]. In what follows we point out some advantages with respect to that paper. It is stressed that the proof of Theorem 2 is a direct consequence of the extended version of LaSalle’s invariance principle in this paper. The proof of [11, Theorem 1], which is not based on an invariance principle, is comparatively more complex, and in particular it requires an elaborate analysis of the behavior of the solutions of (F) close to the set of equilibrium points of (F). Also the mathematical machinery employed in [11]is more complex than that in the present paper. In fact, in [11] use is made of extended Lyapunov functions assuming the value + ∞ outside K and a generalized version of the chain- rule for computing the derivative of the extended-valued functions along the solutions of (F). Here, instead, we have analyzed convergence of (F) by means of a simple quadratic Lyapunov function as in (33). (2) Consider the S-CNN model (S) and suppose that the neuron interconnection matrix A = A is symmetric. It has been shown in [5] that (S) admits the Lyapunov function: ψ(x) =− 1 2 G (x)(A −E n )G(x) −G (x)I, (38) where x ∈ R n . One key problem is that ψ is not a str ict Lyapunov function for the symmetric S-CNN (S), since in partial and total saturation regions of (S) the time derivative of ψ along solutions of (S) may vanish in sets of points that are larger than the sets of equilibrium points of (S). Then, in order to prove quasiconvergence or convergence of (S), it is needed to investigate the geometry of the largest invariant sets of (S) where the time derivative of ψ along solutions of (S) vanishes [7]. Such an analysis is quite elaborate and complex (see [19] for the details). It is worth to remark once more that, according to Theorem 2, φ as in (33) is a strict Lyapunov function for a symmetric FR-CNN, hence the proofofquasiconvergenceorconvergenceof(F) is a direct consequence of the generalized version of LaSalle’s invariance principle in this paper. (3) The derivative Dφ in Definition 1 and the extended version of LaSalle’s invariance principle in Theorem 1 have been inspired by analogous concepts previously developed by Shevitz and Paden [20] and later improved by Bacciotti and Ceragioli [21]. Next, we briefly compare the derivative Dφ with the derivative Dφ proposed in [21]. If we consider that φ is continuously differentiable in R n , then we have Dφ(x) = v, ∇φ(x), v ∈ Ax + I − N K (x) , (39) for any x ∈ K. Note that Dφ is in general set valued, that is, it may assume an entire interval of values. Since P T K (x) (Ax + I) ∩N ⊥ K (x) ⊆ P T K (x) (Ax + I) ⊆ Ax + I −N K (x), we have Dφ(x) ⊆ Dφ(x), (40) for any x ∈ K. An analogous inclusion holds when comparing Dφ with the derivative in [20]. Consider now the following second-order symmetric FR- CNN: ˙ x =−x + Ax + I − N K (x) = f (x) −N K (x), (41) where x = (x 1 , x 2 ) ∈ R 2 , A = ⎛ ⎜ ⎜ ⎝ 2 − 1 2 − 1 2 1 ⎞ ⎟ ⎟ ⎠ , I = ⎛ ⎜ ⎝ 0 2 3 ⎞ ⎟ ⎠ , (42) whose solutions evolve in the square K = [−1, 1] 2 . Also consider the candidate Lyapunov function φ given in (33), namely, φ(x) =− 1 2 x Ax −I x =− 1 2 x 1 (x 1 −x 2 ) − 2 3 x 2 . (43) Simple computations show that, for any x = (x 1 , x 2 ) ∈ K such that x 2 = 1, it holds P T K (x) f (x) ∈ N ⊥ K (x). As a consequence, if a solution of the FR-CNN (41) passes through a point belonging to the upper edge of K, then the solution will slide along that edge during some time interval. Now, consider the point x ∗ = (0,1) , lying on the upper edge of K.Wehave f (x ∗ ) = (−1/2, 2/3) , ∇φ(x ∗ ) = − f (x ∗ ) = (1/2, −2/3) and, from Definition 1, Dφ(x ∗ ) = P T K (x ∗ ) ( f (x ∗ )), ∇V(x ∗ ) = − 1 2 ,0 , 1 2 , 2 2 =− 1 4 < 0. (44) On the other hand, we obtain Dφ(x ∗ ) = v, ∇φ(x ∗ ), v ∈ f (x ∗ ) −N K (x ∗ ) = − 25 36 ,+ ∞ . (45) It is seen that Dφ(x ∗ ) assume both positive and negative values; see Figure 4 for a geometric interpretation. Therefore, by means of the derivative Dφ we can conclude that φ as in (33) is a Lyapunov function for the FR-CNN, while it cannot be concluded that φ is a Lyapunov function for the FR-CNN using the derivative Dφ. 8 EURASIP Journal on Advances in Signal Processing f (x ∗ ) P T K (x ∗ ) ( f (x ∗ )) x 2 x ∗ ∇φ(x ∗ ) f (x ∗ ) −γ 0 f (x ∗ ) −γ + K x 1 N ⊥ K (x ∗ ) Figure 4: Comparison between the derivative Dφ in Definition 1, and the derivative Dφ in [21], for the second-order FR-CNN (41). The point x ∗ = (0, 1) lies on an edge of K such that T K (x ∗ ) = { (x 1 , x 2 ) ∈ R 2 : −∞ <x 1 < +∞, x 2 ≤ 0}, N K (x ∗ ) ={(x 1 , x 2 ) ∈ R 2 : x 1 = 0, x 2 ≥ 0} and N ⊥ K (x ∗ ) ={(x 1 , x 2 ) ∈ R 2 : −∞ < x 1 < +∞, x 2 = 0}.WehaveP T K (x ∗ ) f (x ∗ ) ∈ N ⊥ K (x)andDφ(x ∗ ) = P T K (x ∗ ) f (x ∗ ), ∇φ(x ∗ )=−1/4 < 0. The derivative Dφ(x ∗ )is given by Dφ(x ∗ ) ={v, ∇φ(x ∗ ), v ∈ f (x ∗ ) − N K (x ∗ )}= [−25/36, +∞), hence it assumes both positive and negative values. For example, the figure shows a vector γ 0 ∈ N K (x ∗ ) such that we have Dφ(x ∗ ) 0 =f (x ∗ ) − γ 0 , ∇φ(x ∗ ), and a vector γ + ∈ N K (x ∗ ) for which we have Dφ(x ∗ ) f (x ∗ ) − γ + , ∇φ(x ∗ ) > 0. (4) The Lyapunov approach in this paper has been developed in relation to the differential inclusion modeling the FR model of CNNs, that is, a class of DVIs (16)where the dynamics defined by an affine vector field Ax + I are constrained to evolve within the hypercube K = [−1, 1] n . The approach can be generalized to a wider class of DVIs, by substituting K with an arbitrary compact convex set Q ⊂ R n , or by substituting the affine vector field with a more general (possibly nonsmooth) vector field. In the latter case, it is needed to use nondifferentiable Lyapunov functions and a generalized nonsmooth version of the derivative given in Definition 1. The details on these extensions can be found in the recent paper [13]. 7. Conclusion The paper has developed a generalized Lyapunov approach, which is based on an extended version of LaSalle’s invariance principle, for studying stability and convergence of the FR model of CNNs. The approach has been applied to give a rigorous proof of convergence for symmetric FR-CNNs. The results obtained have shown that, by means of the developed Lyapunov approach, the analysis of convergence of symmetric FR-CNNs is much more simple than that of the symmetric S-CNNs. In fact, one basic result proved here is that a symmetric FR-CNN admits a strict Lyapunov function, and thus it is convergent as a direct consequence of the extended version of LaSalle’s invariance principle. Future work will be devoted to investigate the possibility to apply the proposed methodology for addressing stability of other classes of FR-CNNs that are used in the solution of signal processing tasks in real time. Particular attention will be devoted to certain classes of FR-CNNs with nonsymmetric interconnection matrices. Another interesting issue is the possibility to extend the approach in order to consider the presence of delays in the FR-CNN neuron interconnections. Appendices A. Proof of Property 3 Let M i = n j=1 (|A ij | + |I i |), i = 1,2, , n,andM = max{M 1 , M 2 , , M n }≥0. We have Ax + I ∞ ≤ M +1 for all x ∈ K. We need to define the following maps. For k = 1, 2,3, , let H k (x) = (h k (x 1 ), h k (x 2 ), , h k (x n )) , x ∈ R n ,where h k (ρ) = ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ − M −1, if ρ< −1 −(M +1) k , k(ρ), if |ρ|≤1+(M +1) k , M +1, if ρ>1+(M +1) k , (A.1) and ( ·)isdefinedin(13). Then, let H ∞ (x) = (h ∞ (x 1 ), h ∞ (x 2 ), , h ∞ (x n )) , x ∈ R n ,where h ∞ (ρ) = ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ − (M +1), if ρ<−1, [−M −1,0], if ρ =−1, 0, if |ρ| < 1, [0, M +1], if ρ = 1, M +1, if ρ>1. (A.2) Finally, let B M (x) = (b m (x 1 ), b m (x 2 ), , b m (x n )) , x ∈ R n , where b m (ρ) = ⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩ − (M +1), ifρ<−(1 + M), ρ,if |ρ|≤1+M, M +1, if ρ>1+M. (A.3) The three maps h m , h ∞ and b m are represented in Figure 5. The proof of Property 3 consists of the three main steps detailed below. Step 1. Let x(t), t ≥ 0, be the solution of (F) such that x(0) = x 0 ∈ K. We want to verify that x is also a solution of ˙ x(t) ∈−B M (x(t)) + AG(x(t)) + I − H ∞ (x(t)), (A.4) EURASIP Journal on Advances in Signal Processing 9 h k (ρ) M +1 −1 − M +1 k −1 ρ 1 1+ M +1 k −M −1 (a) h ∞ (ρ) M +1 −1 ρ 1 −M −1 (b) b m (ρ) M +1 −1 −M ρ 1+M −M −1 (c) Figure 5: Auxiliary maps (a) h m ,(b)h ∞ , and (c) b m employed in the proof of Property 3. for t ≥ 0, where G(x) = (g(x 1 ), g(x 2 ), , g(x n )) , x ∈ R n , and g( ·)isgivenin(11). On the basis of [12, page 266, Proposition 2], for a.a. t ≥ 0wehave ˙ x(t) = P T K (x(t)) (Ax(t)+I)) =−x(t)+Ax(t)+I −P N K (x(t)) (Ax(t)+I), (A.5) where P N K (x(t)) (Ax(t)+I) ∈ N K (x(t)) [12, page 24, Proposition 2; page 26, Proposition 3]. Since for any t ≥ 0 we have Ax(t)+I ∞ ≤ M + 1, by applying the result in Lemma 1 in Appendix B,weobtainP N K (x(t)) (Ax(t)+I) ∈ H ∞ (x(t)). Furthermore, considering that for any t ≥ 0we have x(t) ∈ K, it follows that B M (x(t)) = x(t) = G(x(t)). In conclusion, for a.a. t ≥ 0wehave ˙ x(t) ∈−B M (x(t)) + AG(x(t)) + I − H ∞ (x(t)). (A.6) Step 2. For any k = 1, 2, 3, ,letx k (t), t ≥ 0, be the solution of (I) such that x k (0) = x 0 ∈ K. We want to show that x k is also a solution of ˙ x(t) ∈−B M (x(t)) + AG(x(t)) + I − H k (x(t)), (A.7) for t ≥ 0. For any i ∈{1, 2, , n}and t ≥ 0wehavefrom[2, equation 12] |x k i (t)|≤ M + k k +1 + 1 − M + k k +1 exp(−(k +1)t) = 1+ M −1 k +1 (1 −exp(−(k +1)t)) ≤ 1+ |M −1| k +1 ≤ 1+min M, M +1 k . (A.8) Then, B M (x(t)) = x(t) = G(x(t)) and H k (x(t)) = kL(x(t)), for t ≥ 0. Step 3. Consider the map Φ ∞ (x) =−B M (x)+AG(x)+ I − H ∞ (x), x ∈ R n ,andfork = 1, 2, 3, , the maps Φ k (x) =−B M (x)+AG(x)+I − H k (x), x ∈ R n , which are upper semicontinuous in R n with nonempty compact convex values. Let graph(H ∞ ) ={(x, y) ∈ R n × R n : y = H ∞ (x)} and graph(H k ) ={(x, y) ∈ R n × R n : y = H k (x)}.Givenany δ>0, for sufficiently large k,sayk>k δ ,wehave graph(H k ) ⊆ graph(H ∞ )+B(0, δ). (A.9) By applying [12, page 105, Proposition 1] it follows that for any > 0, T>0, and for any k>k δ , there exists a solution x k (t), t ∈ [0, T], of (A.4), such that max [0,T] x k (t)−x k (t) < . Choose = exp(−A 2 T/2)/2, where > 0, A 2 = (λ M (A A)) 1/2 and λ M (A A) denotes the maximum eigenvalue of the symmetric matrix A A. Then, we obtain x k (0) −x(0)=x k (0) −x k (0) ≤ max [0,T] x k (t) −x k (t) < 2 exp( −A 2 T). (A.10) By Property 6 in Appendix C we have max [0,T] x k (t) − x(t) < /2. Then, max [0,T] x k (t) −x(t)≤max [0,T] x k (t) −x(t) +max [0,T] x k (t) −x k (t) < 2 + 2 = , (A.11) for all t ∈ [0, T]. B. Lemma 1 and Its Proof Lemma 1. For any x ∈ K,andanyv ∈ R n such that v ∞ ≤ M +1,wehaveP N K (x) (v) ∈ H ∞ (x). Proof. For any i ∈{1, 2, ,n} we have P N K (x) (v) i = ⎧ ⎨ ⎩ v i ,if|x i |=1, x i v i > 0, 0, otherwise. (B.1) 10 EURASIP Journal on Advances in Signal Processing If P N K (x) (v) i = 0, we immediately obtain [P N K (x) (v)] i ∈ h ∞ (x i ). If x i = 1andx i v i > 0, we may proceed as follows. We hav e h ∞ (x i ) = h ∞ (1) = [0, M + 1]. On the other hand, 0 <v i ≤ M+1 and so [P N K (x) (v)] i = v i ∈ [0, M+1] = h ∞ (x i ). We can proceed in a similar way in the case x i =−1and x i v i > 0. C. Property 6 and Its Proof Property 6. Let > 0. For any y 0 , z 0 ∈ R n such that z 0 − y 0 < exp − A 2 T 2 ,(C.1) we have max [0,T] z(t) − y(t) < ,wherey and z are the solutions of (A.4) such that y(0) = y 0 and z(0) = z 0 , respectively. Proof. Let ϕ(t) =z(t) − y(t) 2 /2, t ∈ [0, T]. Due to (C.1), for a.a. t ∈ [0, T]wehave ˙ ϕ(t) =z(t) − y(t), ˙ z(t) − ˙ y(t) =− z(t) − y(t), B M (z(t)) −B M (y(t)) + z(t) − y(t), A(G(z(t)) −G(y(t))) − z(t) − y(t), γ y (t) −γ z (t), (C.2) where γ y (t) ∈ H ∞ (y(t)) and γ z (t) ∈ H ∞ (z(t)). It is seen that B M is a monotone map in R n [12, page 159, Proposition 1], that is, for any x, y ∈ R n and any γ x ∈ B M (x), γ y ∈ B M (y), we have x − y, γ x −γ y ≥0. Also H ∞ is a monotone map in R n . Then, we obtain ˙ ϕ(t) ≤z(t) − y(t), A(G(z(t)) −G(y(t))) ≤ Az(t) − y(t) 2 = 2Aϕ(t). (C.3) Gronwall’s lemma yields ϕ(t) ≤ ϕ(0)e AT ,andso z(t) − y(t)= 2ϕ(t) ≤ 2ϕ(0)e AT < ,(C.4) for t ∈ [0, T]. Acknowledgment The authors wish to thank the anonymous Reviewers and Associate Editor for the insightful and constructive comments. References [1] A. Rodr ´ ıguez-V ´ azquez, S. Espejo, R. Dom ´ ınguez-Castro, J. L. Huertas, and E. S ´ anchez-Sinencio, “Current-mode techniques for the implementation of continuous- and discrete-time cellular neural networks,” IEEE Transactions on Circuits and Systems II, vol. 40, no. 3, pp. 132–146, 1993. [2] S. Espejo, R. Carmona, R. Dom ´ ınguez-Castro, and A. Rodr ´ ıguez-V ´ azquez, “A VLSI-oriented continuous-time CNN model,” International Journal of Circuit Theory and Applica- tions, vol. 24, no. 3, pp. 341–356, 1996. [3] G. L. Cembrano, A. Rodr ´ ıguez-V ´ azquez, S. E. Meana, and R. Dom ´ ınguez-Castro, “ACE16k: a 128 × 128 focal plane analog processor with digital I/O,” International Journal of Neural Systems, vol. 13, no. 6, pp. 427–434, 2003. [4] A. Rodr ´ ıguez-V ´ azquez, G. Li ˜ n ´ an-Cembrano, L. Carranza, et al., “ACE16k: the third generation of mixed-signal SIMD- CNN ACE chips toward VSoCs,” IEEE Transactions on Circuits and Systems I, vol. 51, no. 5, pp. 851–863, 2004. [5] L. O. Chua and L. Yang, “Cellular neural networks: theory,” IEEE Transactions on Circuits and Systems, vol. 35, no. 10, pp. 1257–1272, 1988. [6] L.O.Chua,CNN: A Paradigm for Complexity, World Scientific, Singapore, 1998. [7] M. W. Hirsch, “Convergent activation dynamics in continuous time networks,” Neural Networks, vol. 2, no. 5, pp. 331–349, 1989. [8]L.O.ChuaandT.Roska,Cellular Neural Networks and Visual Computing: Foundations and Applications, Cambridge University Press, Cambridge, UK, 2005. [9] M. Forti, P. Nistri, and M. Quincampoix, “Convergence of neural networks for programming problems via a nonsmooth Łojasiewicz inequality,” IEEE Transactions on Neural Networks, vol. 17, no. 6, pp. 1471–1486, 2006. [10] L. O. Chua, Ed., “Special issue on nonlinear waves, patterns and spatio-temporal chaos in dynamic arrays,” IEEE Transac- tions on Circuits and Systems I, vol. 42, no. 10, pp. 557–823, 1995. [11] G. De Sandre, M. Forti, P. Nistri, and A. Premoli, “Dynamical analysis of full-range cellular neural networks by exploiting differential variational inequalities,” IEEE Transactions on Circuits a nd Systems I, vol. 54, no. 8, pp. 1736–1749, 2007. [12] J. P. Aubin and A. Cellina, Differential Inclusions, Springer, Berlin, Germany, 1984. [13] M. Di Marco, M. Forti, M. Grazzini, P. Nistri, and L. Pancioni, “Lyapunov method and convergence of the full-range model of CNNs,” IEEE Transactions on Circuits and Systems I, vol. 55, no. 11, pp. 3528–3541, 2008. [14] J. P. Aubin and H. Frankowska, Set-Valued Analysis, Birkh ¨ auser, Boston, Mass, USA, 1990. [15] T. Rockafellar and R. Wets, Variational Analysis, Springer, Berlin, Germany, 1997. [16] F. Corinto and M. Gilli, “Comparison between the dynamic behaviour of Chua-Yang and full-range cellular neural net- works,” International Journal of Circuit Theory and Applica- tions, vol. 31, no. 5, pp. 423–441, 2003. [17] J. K. Hale, Ordinary Differential Equations, Wiley Interscience, New York, NY, USA, 1969. [18] A. F. Filippov, Differential Equations with Discontinuous Right- Hand Side, Mathematics and Its Applications (Soviet Series), Kluwer Academic Publishers, Boston, Mass, USA, 1988. [19] S S. Lin and C W. Shih, “Complete stability for standard cellular neural networks,” International Journal of Bifurcation and Chaos, vol. 9, no. 5, pp. 909–918, 1999. [20] D. Shevitz and B. Paden, “Lyapunov stability theory of nonsmooth systems,” IEEE Transactions on Automatic Control, vol. 39, no. 9, pp. 1910–1914, 1994. [21] A. Bacciotti and F. Ceragioli, “Stability and stabilization of discontinuous systems and nonsmooth Lyapunov functions,” ESAIM: Control, Optimisat ion and Calculus of Variations,no. 4, pp. 361–376, 1999. . Processing Volume 2009, Article ID 730968, 10 pages doi:10.1155/2009/730968 Research Article Extended LaSalle’s Invariance Principle for Full-Range Cellular Neural Networks Mauro Di Marco, Mauro Forti, Massimo. of the generalized version of LaSalle’s invariance principle in this paper. (3) The derivative Dφ in Definition 1 and the extended version of LaSalle’s invariance principle in Theorem 1 have been. is a direct consequence of the extended version of LaSalle’s invariance principle in this paper. The proof of [11, Theorem 1], which is not based on an invariance principle, is comparatively more