Hindawi Publishing Corporation Boundary Value Problems Volume 2008, Article ID 217636, 10 pages doi:10.1155/2008/217636 ResearchArticleExistenceResultforaClassofEllipticSystemswithIndefiniteWeightsin R 2 Guoqing Zhang 1 and Sanyang Liu 2 1 College of Sciences, University of Shanghai for Science and Technology, Shanghai 200093, China 2 Department of Applied Mathematics, Xidian University, Xi’an 710071, China Correspondence should be addressed to Guoqing Zhang, zgqw2001@sina.com.cn Received 31 October 2007; Accepted 4 March 2008 Recommended by Zhitao Zhang We obtain the existenceofa nontrivial solution foraclassof subcritical ellipticsystemswithindefiniteweightsin R 2 . The proofs base on Trudinger-Moser inequality and a generalized linking theorem introduced by Kryszewski and Szulkin. Copyright q 2008 G. Zhang and S. Liu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. 1. Introduction In this paper, we study the existenceofa nontrivial solution for the following systemsof two semilinear coupled Poisson equations P −Δu u gx, v,x∈ R 2 , −Δv v fx, u,x∈ R 2 , 1.1 where fx, t and gx, t are continuous functions on R 2 × R and have the maximal growth on t which allows to treat problem P variationally, Δ is the Laplace operator. Recently, there exists an extensive bibliography in the study ofelliptic problem in R N 1–6. As dimensions N ≥ 3, in 1998, de Figueiredo and Yang 5 considered the following coupled elliptic systems: −Δu u gx, v,x∈ R N , −Δv v fx, u,x∈ R N , 1.2 2 Boundary Value Problems where f, g are radially symmetric in x and satisfied the following Ambrosetti-Rabinowitz condition: t 0 fx, sds ≥ c|t| 2δ 1 , t 0 gx, sds ≥ c|t| 2δ 2 , ∀t ∈ R, 1.3 and for some δ 1 > 0,δ 2 > 0. They obtained the decay, symmetry, and existenceof solutions for problem 1.2. In 2004, Li and Yang 6 proved that problem 1.2 possesses at least a positive solution when the nonlinearities fx, t and gx, t are “asymptotically linear” at infinity and “superlinear” at zero, that is, 1 lim t→∞ fx, t/tl>1, lim t→∞ gx, t/tm>1, uniformly in x ∈ R N ; 2 lim t→0 fx, t/tlim t→0 gx, t/t0, uniformly with respect to x ∈ R N . In 2006, Colin and Frigon 1 studied the following systemsof coupled Poission equations with critical growth in unbounded domains: −Δu |v| 2 ∗ −2 v, −Δv |u| 2 ∗ −2 u, 1.4 where 2 ∗ 2N/N − 2 is critical Sobolev exponent, u, v ∈ D 1,2 0 Ω ∗ and Ω ∗ R N \ E with E a∈Z N a ω ∗ fora domain containing the origin ω ∗ ⊂ ω ∗ ⊂ B0, 1/2. Here, B0, 1/2 denotes the open ball centered at the origin of radius 1/2. The existenceofa nontrivial solution was obtained by using a generalized linking theorem. As it is well known in dimensions N ≥ 3, the nonlinearities are required to have polynomial growth at infinity, so that one can define associated functionals in Sobolev spaces. Coming to dimension N 2, much faster growth is allowed for the nonlinearity. In fact, the Trudinger-Moser estimates in N 2 replace the Sobolev embedding theorem used in N ≥ 3. In dimension N 2, Adimurth and Yadava 7, de Figueiredo et al. 8 discussed the solvability of problems of the type −Δu fx, u,x∈ Ω, u 0,x∈ ∂Ω, 1.5 where Ω is some bounded domain in R 2 . Shen et al. 9 considered the following nonlinear elliptic problems with critical potential: Δu − μ u |x| log R/|x| 2 fx, u,x∈ Ω u 0,x∈ ∂Ω, 1.6 and obtained some existence results. In the whole space R 2 , some authors considered the following single semilinear elliptic equations: −Δu V xu fx, u,x∈ R 2 . 1.7 G. Zhang and S. Liu 3 As the potential V x and the nonlinearity fx, t are asymptotic to a constant function, Cao 10 obtained the existenceofa nontrivial solution. As the potential V x and the nonlinearity fx, t are asymptotically periodic at infinity, Alves et al. 11 proved the existenceof at least one positive weak solution. Our aim in this paper is to establish the existenceofa nontrivial solution for problem P in subcritical case. To our knowledge, there are no results in the literature establishing the existenceof solutions to these problems in the whole space. However, it contains a basic difficulty. Namely, the energy functional associated with problem P has strong indefinite quadratic part, so there is not any more mountain pass structure but linking one. Therefore, the proofs of our main results cannot rely on classical min-max results. Combining a generalized linking theorem introduced by Kryszewski and Szulkin 12 and Trudinger-Moser inequality, we prove an existenceresultfor problem P. The paper is organized as follows. In Section 2, we recall some results and state our main results. In Section 3, main result is proved. 2. Preliminaries and main results Consider the Hilbert space 13 H 1 R 2 u ∈ L 2 R 2 , ∇u ∈ L 2 R 2 , 2.1 and denote the product space Z H 1 R 2 × H 1 R 2 endowed with the inner product: u, v, φ, ψ R 2 ∇u∇φ uφdx R 2 ∇v∇ψ vψdx, ∀φ, ψ ∈ Z. 2.2 If we define Z {u, u ∈ Z},Z − {v,−v ∈ Z}. 2.3 It is easy to check that Z Z ⊕ Z − , since u, v 1 2 u v, u v 1 2 u − v, v − u. 2.4 Let us denote by P resp., Q the projection of Z on to Z resp., Z − ,wehave 1 2 Pu, v 2 − Qu, v 2 1 2 1 2 u v, u v 2 − 1 2 1 2 u − v, v − u 2 1 4 R 2 |∇u| 2 |∇v| 2 2∇u∇v dx R 2 |u| 2 |v| 2 2uv dx − R 2 |∇u| 2 |∇v| 2 − 2∇u∇v dx − R 2 |u| 2 |v| 2 − 2uv dx R 2 ∇u∇v uvdx. 2.5 4 Boundary Value Problems Now, we define the functional Iu, v R 2 ∇u∇v uvdx − R 2 Fx, uGx, v dx Pu, v 2 2 − Qu, v 2 2 − ϕu, v, ∀u, v ∈ Z, 2.6 where ϕu, v R 2 Fx, uGx, v dx. 2.7 Let z 0 ∈ Z \{0} and let R>r>0, we define M z z − λz 0 : z − ∈ Z − , z≤R, λ ≥ 0 , M 0 z z − λz 0 : z − ∈ Z − , z R and λ ≥ 0orz≤R and λ ≥ 0 , N z ∈ Z : z r . 2.8 Here, we assume the following condition: H1 f,g ∈ CR 2 × R, R; H2 lim t→0 fx, t/tlim t→0 gx, t/t0 uniformly with respect to x ∈ R 2 ; H3 there exist μ>2andη>0 such that 0 <μFx, t ≤ tfx, t, 0 <μGx, t ≤ tgx, t, ∀|t|≥η. 2.9 Lemma 2.1 see 12, 14. Assume (H1), (H2), and (H3), and suppose 1 Iz1/2Pz 2 −Qz 2 −ϕz, where ϕ ∈ C 1 Z, R is sequentially lower semicontinu- ous, bounded below, and ∇ϕ is weakly sequentially continuous; 2 there exist z 0 ∈ Z \{0},α>0,andR>r>0, such that inf N Iz ≥ α>0, sup M 0 Iz ≤ 0. 2.10 Then, there exist c>0 and a sequence z n ⊂ Z such that Iz n −→ c, I z n −→ 0, as n −→ ∞ . 2.11 Moreover, c ≥ α. Theorem 2.2. Under the assumptions (H1), (H2), and (H3), if f and g has subcritical growth (see definition below), problem (P) possesses a nontrivial weak solution. G. Zhang and S. Liu 5 In the whole space R 2 , do ´ O and Souto 15 proved a version of Trudinger-Moser inequality, that is, i if u ∈ H 1 R 2 ,β>0, we have R 2 exp β|u| 2 − 1 dx < ∞; 2.12 ii if 0 <β<4π and |u| L 2 R 2 ≤ c, then there exists a constant c 2 c 1 c, β such that sup |∇u| L 2 R 2 ≤1 R 2 exp β|u| 2 − 1 dx < c 2 . 2.13 Definition 2.3. We say fx, t has subcritical growth at ∞, if for all β>0, there exists a positive constant c 3 such that fx, t ≤ c 3 exp βt 2 , ∀x, t ∈ R 2 × 0, ∞. 2.14 3. Proof of Theorem 2.2 In this section, we will prove Theorem 2.2. under our assumptions and 2.14, there exist c ε > 0,β > 0 such that Fx, t , Gx, t ≤ t 2 2 ε c ε exp βt 2 − 1 , ∀ε>0, ∀t ∈ R. 3.1 Then, we obtain Fx, u,Gx, v ∈ L 2 R 2 , ∀u, v ∈ H 1 R 2 . 3.2 Therefore, the functional Iu, v is well defined. Furthermore, using standard arguments, we obtain the functional Iu, v is C 1 functional in Z and I u, vφ, ψ R 2 ∇u∇ψ uψdx R 2 ∇v∇φ vφ dx − R 2 fx, uφ gx, vψ dx, ∀φ, ψ ∈ Z. 3.3 Consequently, the weak solutions of problem P are exactly the critical points of Iu, v in Z. Now, we prove that the functional Iu, v satisfied the geometry of Lemma 2.1. Lemma 3.1. There exist r>0 and α>0 such that inf N Iu, u ≥ α>0. Proof. By 2.14 and assumption H2, there exists c ε > 0 such that Fx, t,Gx, t ≤ t 2 2 ε c ε t 3 exp βt 2 − 1 , ∀t ∈ R, 3.4 6 Boundary Value Problems and thus on N, we have Iu, u ≥ R 2 |∇u| 2 u 2 dx − R 2 εu 2 c ε u 3 exp βu 2 − 1 dx ≥ R 2 |∇u| 2 u 2 dx − ε R 2 u 2 dx − c ε R 2 u 6 dx 1/2 R 2 exp βu 2 − 1 2 dx 1/2 ≥ R 2 |∇u| 2 u 2 dx − ε R 2 u 2 dx − c ε u 3 R 2 exp βu 2 − 1 dx 1/2 . 3.5 So, by the Sobolev embedding theorem and 2.12, we can choose r>0sufficiently small, such that Iu, u ≥ α>0, whenever u r. 3.6 Lemma 3.2. There exist u 0 ,u 0 ∈ Z \{0} and R>r>0 such that sup M 0 I ≤ 0. Proof. 1 By assumption H3,wehaveonZ − Iu, u R 2 |∇u| 2 u 2 dx − R 2 Fx, uGx, −u dx ≤ 0 3.7 because Fx, t ≥ 0,Gx, t ≥ 0 for any x, t ∈ R 2 × R. 2 Assumption H3 implies that there exist c 4 > 0,c 5 > 0 such that Fx, t,Gx, t ≥ c 4 t μ − c 5 , ∀t ∈ R. 3.8 Now, we choose u 0 ,u 0 ∈ Z \{0} such that u 0 ,u 0 r,then I −v, vλ u 0 ,u 0 λ 2 R 2 |∇u 0 | 2 u 2 0 dx − R 2 |∇v| 2 v 2 dx − R 2 Fλu 0 v G λu 0 − v dx ≤− R 2 |∇u| 2 u 2 dx c λ 2 − λ μ . 3.9 Because μ>2, it follows that for w ∈ M 0 Iw −→ − ∞ , whenever w−→∞, 3.10 and so, taking R>rlarge, we get sup M 0 I ≤ 0. G. Zhang and S. Liu 7 Proof of Theorem 2.2. By Lemma 3.1, there exist r>0andα>0 such that inf N Iu, u ≥ α>0. By Lemma 3.2, there exist u 0 ,u 0 ∈ Z \{0} and R>r>0 such that sup M 0 I ≤ 0. Since Z Z ⊕Z − , we have Iu, v R 2 ∇u∇v uvdx − R 2 Fx, uGx, v dx Pu, v 2 2 − Qu, v 2 2 − ϕu, v, ∀u, v ∈ Z. 3.11 From 2.14, 3.1, and assumption H3, ϕu, v ∈ C 1 ,ϕu, v ≥ 0andϕu, v is sequentially lower semicontinuous by Z ⊂ L 2 loc R 2 ×L 2 loc R 2 and Fatou’s lemma; ∇ϕ is weakly sequentially continuous. Thus, by Lemma 2.1 there exists a sequence u n ,v n ⊂ Z such that Iu n ,v n −→ c ≥ α, I u n ,v n −→ 0. 3.12 Claim 3.3. There is c<∞, such that u n ,v n ≤c for any n. Indeed, from 3.12,weobtain that the sequence u n ,v n ⊂ Z satisfies I u n ,v n c δ n ,I u n ,v n φ, ψε n u n ,v n , as n −→ ∞ , 3.13 where φ, ψ ∈{u n ,v n },δ n → 0,ε n → 0asn →∞. Taking φ, ψ{u n ,v n } in 3.13 and assumption H3,wehave R 2 f x, u n u n g x, v n v n dx ≤ 2 R 2 F x, u n G x, v n dx 2c 2δ n ε n u n ,v n ≤ 2 μ R 2 fx, u n u n g x, v n v n dx C 2δ n ε n u n ,v n , 3.14 where C depends only on c and η in assumption H3. Since μ>2, we have 1 − 2/μ > 0, and thus 1 − 2 μ R 2 f x, u n u n g x, v n v n dx ≤ C 2δ n ε n u n ,v n , ∀n ∈ N. 3.15 On the other hand, let φ, ψv n , 0, φ, ψ0,u n in 3.13,weobtain v n 2 − ε n v n ≤ R 2 f x, u n v n dx, u n 2 − ε n u n ≤ R 2 g x, v n u n dx. 3.16 that is, v n ≤ R 2 f x, u n v n v n dx ε n , u n ≤ R 2 g x, v n u n u n dx ε n . 3.17 8 Boundary Value Problems Now, we recall the following inequality see 7, Lemma 2.4 : mn ≤ ⎧ ⎪ ⎨ ⎪ ⎩ e n 2 − 1 mlog m 1/2 ,n≥ 0,m≥ e 1/4 , e n 2 − 1 1 2 m 2 ,n≥ 0, 0 ≤ m ≤ e 1/4 . 3.18 Let n v n /v n and m fx, u n /c 3 , where c 3 is defined in 2.14,wehave c 3 R 2 f x, u n c 3 v n v n dx ≤ c 3 R 2 exp v n v n 2 − 1 dx c 3 {x∈R 2 ,fx,u n /c 3 ≥e 1/4 } f x, u n c 3 log f x, u n c 3 1/2 dx c 3 {x∈R 2 ,fx,u n /c 3 ≤e 1/4 } f x, u n c 3 2 dx. 3.19 By 2.12,wehave R 2 exp v n /v n 2 − 1dx < ∞. By 2.14,wehave log fx, t c 3 1/2 ≤ β 1/2 t. 3.20 Hence, we have c 3 R 2 f x, u n c 3 v n v n dx ≤ c 6 β 1/2 R 2 f x, u n u n dx 3.21 for some positive constant c 6 . So we have v n ≤ c 6 β 1/2 R 2 f x, u n u n dx ε n . 3.22 Using a similar argument, we obtain u n ≤ c 7 β 1/2 R 2 g x, v n v n dx ε n 3.23 for some positive constant c 7 . Combining 3.22 and 3.23,wehave u n ,v n ≤ c 8 1 δ n ε n u n ,v n ε n 3.24 for some positive constant c 8 , which implies that u n ,v n ≤c. Thus, fora subsequence still denoted by u n ,v n , there is u 0 ,v 0 ∈ Z such that u n ,v n −→ u 0 ,v 0 weakly in Z, as n −→ ∞ , u n ,v n −→ u 0 ,v 0 in L s loc R 2 × L s loc R 2 for s ≥ 1, as n −→ ∞ , u n x,v n x −→ u 0 x,v 0 x , almost every, in R 2 , as n −→ ∞ . 3.25 G. Zhang and S. Liu 9 Then, there exists hx ∈ H 1 R 2 such that |u n x|≤h, ∀x ∈ R 2 , ∀n ∈ N. From 2.12 and 2.14,wehave R 2 expβh 2 x − 1dx < c, this implies R 2 f x, u n φdx −→ R 2 f x, u 0 φdx, as n −→ ∞ . 3.26 Similarly, we can obtain R 2 g x, v n ψdx −→ R 2 g x, v 0 ψdx, as n −→ ∞ . 3.27 From these, we have I u n ,v n φ, ψ0, so u 0 ,v 0 is weak solution of problem P. Claim 3.4. u 0 ,v 0 is nontrivial. By contradiction, since fx, t has subcritical growth, from 2.14 and H ¨ older inequality, we have R 2 f x, u n u n dx ≤ c R 2 u n exp βu 2 n − 1 dx ≤ c R 2 |u n | q dx 1/q R 2 exp βqu 2 n − 1 dx 1/q , 3.28 where 1/q 1/q 1. Choosing suitable β and q, we have R 2 exp βqu 2 n − 1 dx ≤ c. 3.29 Then, we obtain R 2 f x, u n u n dx ≤ c R 2 u n q dx 1/q . 3.30 Since u n → 0inL q R 2 , as n →∞, this will lead to R 2 f x, u n u n dx −→ 0, as n −→ ∞ . 3.31 Similarly, we have R 2 g x, v n v n dx −→ 0, as n −→ ∞ . 3.32 Using assumption H3,weobtain R 2 F x, u n dx −→ 0, R 2 G x, v n dx −→ 0, as n −→ ∞ . 3.33 This together with I u n ,v n u n ,v n → 0, we have R 2 ∇u n ∇v n u n v n dx −→ 0, as n −→ ∞ . 3.34 Thus, we see that I u n ,v n −→ 0, as n −→ ∞ . 3.35 which is a contradiction to Iu n ,v n → c ≥ α>0, as n →∞. Consequently, we have a nontrivial critical point of the functional Iu, v and conclude the proof of Theorem 2.2. 10 Boundary Value Problems Acknowledgment This work is supported by Innovation Program of Shanghai Municipal Education Commission under Grant no. 08 YZ93. References 1 F. Colin and M. Frigon, “Systems of coupled Poisson equations with critical growth in unbounded domains,” Nonlinear Differential Equations and Applications, vol. 13, no. 3, pp. 369–384, 2006. 2 Y. Ding and S. Li, “Existence of entire solutions for some elliptic systems,” Bulletin of the Australian Mathematical Society, vol. 50, no. 3, pp. 501–519, 1994. 3 D. G. de Figueiredo, “Nonlinear elliptic systems,” Anais da Academia Brasileira de Ci ˆ encias,vol.72,no.4, pp. 453–469, 2000. 4 D. G. de Figueiredo, J. M. do ´ O, and B. Ruf, “Critical and subcritical ellipticsystemsin dimension two,” Indiana University Mathematics Journal, vol. 53, no. 4, pp. 1037–1054, 2004. 5 D. G. de Figueiredo and J. Yang, “Decay, symmetry and existenceof solutions of semilinear elliptic systems,” Nonlinear Analysis: Theory, Methods & Applications, vol. 33, no. 3, pp. 211–234, 1998. 6 G. Li and J. Yang, “Asymptotically linear elliptic systems,” Communications in Partial Differential Equations, vol. 29, no. 5-6, pp. 925–954, 2004. 7 Adimurthi and S. L. Yadava, “Multiplicity results for semilinear elliptic equations ina bounded domain of R 2 involving critical exponents,” Annali della Scuola Normale Superiore di Pisa, vol. 17, no. 4, pp. 481–504, 1990. 8 D. G. de Figueiredo, O. H. Miyagaki, and B. Ruf, “Elliptic equations in R 2 with nonlinearities in the critical growth range,” Calculus of Variations and Partial Differential Equations, vol. 3, no. 2, pp. 139–153, 1995. 9 Y. Shen, Y. Yao, and Z. Chen, “On aclassof nonlinear elliptic problem with critical potential in R 2 ,” Science in China Series A, vol. 34, pp. 610–624, 2004. 10 D. M. Cao, “Nontrivial solution of semilinear elliptic equation with critical exponent in R 2 ,” Communications in Partial Differential Equations, vol. 17, no. 3-4, pp. 407–435, 1992. 11 C. O. Alves, J. M. do ´ O, and O. H. Miyagaki, “On nonlinear perturbations ofa periodic elliptic problem in R 2 involving critical growth,” Nonlinear Analysis: Theory, Methods & Applications, vol. 56, no. 5, pp. 781–791, 2004. 12 W. Kryszewski and A. Szulkin, “Generalized linking theorem with an application to a semilinear Schr ¨ odinger equation,” Advances in Differential Equations, vol. 3, no. 3, pp. 441–472, 1998. 13 M. Willem, Minimax Theorems,vol.24ofProgress in Nonlinear Differential Equations and Their Applications, Birkh ¨ auser, Boston, Mass, USA, 1996. 14 G. Li and A. Szulkin, “An asymptotically periodic Schr ¨ odinger equation withindefinite linear part,” Communications in Contemporary Mathematics, vol. 4, no. 4, pp. 763–776, 2002. 15 J. M. do ´ O and M. A. S. Souto, “On aclassof nonlinear Schr ¨ odinger equations in R 2 involving critical growth,” Journal of Differential Equations, vol. 174, no. 2, pp. 289–311, 2001. . existence of a nontrivial solution for a class of subcritical elliptic systems with indefinite weights in R 2 . The proofs base on Trudinger-Moser inequality and a generalized linking theorem introduced. the proofs of our main results cannot rely on classical min-max results. Combining a generalized linking theorem introduced by Kryszewski and Szulkin 12 and Trudinger-Moser inequality, we prove an. Hindawi Publishing Corporation Boundary Value Problems Volume 2008, Article ID 217636, 10 pages doi:10.1155/2008/217636 Research Article Existence Result for a Class of Elliptic Systems with Indefinite