Some topics in geometric invariant theory over non algebraically closed fields

In this paper, we review some problems related with the study of geometric and relative orbits for the actions of algebraic groups on affine varieties defined over nonalgebraically closed fields. Mathematics Subject Classification (AMS 2000) : Primary : 14L24. Secondary: 14L30, 20G15

Some topics in geometric invariant theory over non-algebraically closed fields Dao Phuong Bac∗ and Nguyˆen ˜ Quˆo´c Thˇan ´ g† Abstract In this paper, we review some problems related with the study of geometric and relative orbits for the actions of algebraic groups on affine varieties defined over non-algebraically closed fields. Mathematics Subject Classification (AMS 2000) : Primary : 14L24. Secondary: 14L30, 20G15. Plan. I. Introduction. II. An overview of geometric invariant theory. Observability and related notions. III. Stability in geometric invariant theory over non-algebraically closed fields. IV. Topology of relative orbits for actions of algebraic groups over completely valued fields. 1 Introduction Let G be a smooth affine algebraic group acting morphically on an affine variety X, all defined over a field k. Many results of (geometric) invariant theory related to the orbits of the action of G are obtained in the geometric ∗ Department of Mathematics, VNU of Science, 334 Nguyen Trai, Hanoi, Vietnam. E-mail : daophuongbac@math.harvard.edu, daophuongbac@yahoo.com. † Institute of Mathematics, 18-Hoang Quoc Viet, Hanoi, Vietnam. E-mail : nqthang@math.ac.vn. Support in part by NAFOSTED and VIASM. 1 case, i.e., when k is an algebraically closed field. However, since the very beginning of modern geometric invariant theory, as presented in [Mu], [MFK], there has been a need to consider the relative case of the theory. For example, Mumford has considered many aspects of the theory already over sufficiently general base schemes, with arithmetical aim (say, to construct arithmetic moduli of abelian varieties, as in Chap. 3 of [Mu], [MFK]). Also some questions or conjectures due to Borel ([Bo1]), Tits ([Mu]) ... ask for extensions of results obtained (in the case of algebraically closed fields) to the case of non-algebraically closed fields. As typical examples, we cite the results by Birkes [Bi], Kempf [Ke], Raghunathan [Ra], which gave solutions to some of the above mentioned questions or conjectures. This article has its aim to overview some of the recent results in this direction, while trying to put them in a coherent form. In fact, since the results in the field are quite diverse, to give a reasonable account of (the majority of) all existing results would require a whole book. Therefore, the reades will find that we are concerned mostly with some basic topics such as the geometry and the topology of the orbits. Throughout, we consider only smooth affine (i.e. linear) algebraic groups defined over some field k, which are called also shortly as k-groups. For basic theory of smooth affine (linear) algebraic groups over non-algebraically closed field we refer to [Bo2], and for a k-group G, the notion of a rational k-module V for G is as in [Gr1], [Gr2]. However, in few places we give some account of the recent development, which is directly related to our discussion here. 2 2.1 An overview of geometric invariant theory. Observability and related notions Some basic definitions and facts To keep things simple, we describe the basic notions in their simplest form (thus not in most general form), with the hope that the readers will either be able themselves either to extend to a more general setting later on, or find the corresponding ones in the literature. 2 Action and Orbit. Let k be a field, k¯ an algebraic closure of k. Let G be an affine algebraic group, V an affine variety all are defined over k. For ¯ Assume that there exists simplicity, we identify G, V with their points in k. a regular k-morphism ϕ : G × V → V , (g, v) → g.v such that the following holds: 1) g.(h.v) = (gh).v, ∀ g, h ∈ G, v ∈ V ; 2) e.v = v, ∀ v ∈ V , where e denotes the identity element of G. Then one says that we are given an action of G on V defined over k, or that G acts k-regularly on V. For a fixed v ∈ V , the image of G × {v} via ϕ is called the orbit of v under G and denoted by G.v. Consider a smooth affine algebraic group G acting morphically on an affine variety V , all are defined over a field k. One of the basic subjects in our study is the orbit G.v, v ∈ V , under the action of G. From the geometric point of view, the most important objects are the closed orbits and open orbits (with respect to Zariski topology). The first natural question is the following: Is there any closed (open) orbit in V ? One of the first, though elementary but very basic, results is the following well-known statement, which assures the existence of closed orbits. On should also note that the open orbits may not exist in general. 2.1.1. Theorem. (Cf. [Bo1], [Hum]) With above assumption, a) Each orbit G.v is a smooth locally closed subvariety of V; b) Each orbit G.v contains an open dense subset of its closure; c) The boundary of G.v is the union of orbits of lower dimension; d) There exists G-orbits which are closed (in Zariski topology) in V. Linear action. Linearization. Among those affine varieties which are the most important, one can single out the class of varieties V with linear action of G, i.e., V are vector spaces and G acts on V via a rational (i.e. linear) representation ρ : G → GL(V ). We call such V also G-modules (or k-G-modules if they are defined over k). It is no doubt that the study of representation theory of G by using (or from the point of view of) geometric invariant theory should play some definite role. 3 Fortunately, the general action of affine groups on affine varieties is not far from this linear action, as the following statement shows. 2.1.2. Theorem. (Cf. [Bi], [Bre], [Ke], [KSS]) Let G be an affine algebraic group acting regularly on an affine variety V, all defined over a field k. Then there exists a closed embedding ψ : V → W and a rational representation ρ : G → GL(W ), all defined over k, such that ρ(g)(ψ(x)) = ψ(g.x) for all x ∈ V . One should also note that a closely related and somewhat more general result also holds. Namely we have 2.1.3. Theorem. (Cf. [Su1, Theorem 1], [Su2, Theorem 2.5]) Let G be a connected affine algebraic group acting regularly on a normal quasi-projective variety V . Then there exists a projective embedding ψ : V → W := Pn and a projective group representation ρ : G → PGLn = Aut(W ) such that ρ(g)(ψ(x)) = ψ(g.x) for all x ∈ V . Therefore it is no harm to consider any affine (resp. quasi-projective) Gvariety V just as a G-stable closed k-subvariety of some k-G-module (resp. some projective G-space) W . The proofs of all these facts give also very explicit way to construct such linear (resp. projective spaces), where the action is linearized. The setting in [Su2] is scheme-theoretic, thus it may be applied to a more general situation. Quotients. To study the actions of algebraic groups on algebraic varieties, it is convenient to consider the set of orbits (the quotient sets) for such actions. However, such sets usually do not give much information and the first question comes to mind, when we are considering the action of algebraic groups on algebraic varieties, is how to recognize if there is any other structure on such sets. Naturally, if an algebraic group G acts on a variety V , all defined over a field k, it also acts on the affine algebra k[V ]. To use the correspondence Geometry ↔ Algebra, the quotient set V /G should have the algebra of functions which, being considered as functions on V , are constant on the G-orbits, thus the G-invariant functions. In order that the quotient of V under the action of G exist, it is necessary that the subalgebra k[V ]G of all G-invariant functions of k[V ] be finitely generated. Assuming this, we 4 say that a variety W , together with a morphism p : V → W is an algebraic (or categorical) quotient of V under the action of G, if the comorphism p∗ : k[W ] k[V ]G is a k-isomorphism of k-algebras from k[W ] onto the subalgebra of all G-invariant functions in k[G], and we denote W = V //G. Further, a categorical quotient is called a geometric quotient, if all the fibers of p are closed, in other words, all G-orbits are closed. If it is the case, we denote V /G the corresponding geometric quotient. One should mention also that in order that the geometric quotient exist, it is necessary that all the G-orbits be closed. Meanwhile, the following holds 2.1.4. Theorem. (Rosenlicht) (Cf. [Do, Thm. 6.2], [Sp, Satz 2.2]) Given any action of an affine algebraic group G on an affine variety V, there exists an open G-stable subvariety U → V such that with the induced action of G on U, the geometric quotient U/G exists. One should mention the following notions. An affine algebraic group G is called linearly reductive if any linear representation ρ : G → GL(V ) is completely reducible. Equivalently, G is linearly reductive, if for any such ρ and non-zero G-invariant vector v ∈ V , there exists a linear G-invariant form F on V such that F (v) = 0. G is called geometrically reductive if for any linear representation ρ : G → GL(V ) and non-zero G-invariant vector v ∈ V , there exists a homogeneous G-invariant polynomial F on V such that F (v) = 0. It is tautollogical that linearly reductive ⇒ geometrically reductive. The converse is also true in characteristic 0 (and this is also equivalent to the property of being reductive (the solvable radical is a torus)) but fails in general in characteristic p > 0. Finally, G is called reductive if its unipotent radical Ru (G) is trivial. Then we have the following criteria for linearly reductivity due to Kemper [K]. 2.1.5. Theorem. [K] Let k be a field, G a linear algebraic group defined over k. Then G is linearly reductive if and only if the following conditions hold: a) For every affine G-scheme X, the invariant ring k[X]G is finitely generated over k, b) For every G-module V, the invariant ring k[V ]G is a Cohen-Macaulay ring. Quite recently, Alper (et al.) in a series of papers [Al1], [Al2], [AE] have 5 extended several classical results in the case of smooth affine groups over fields to group schemes. Let S be a scheme, G → S a f ppf group scheme, BG the classifying stack for G. G is called S-linearly reductive if the morphism BG → S is cohomological affine, i.e., if f is quasi-compact and it induces an exact functor f∗ : QCoh(BG) → QCoh(S) between the categories of quasi-coherent modules over BG and that over S. Then it was proved in [Al1] the following characterization of the linear reductivitiy. 2.1.6. Proposition. ([Al1, Prop. 12.6]) Let k be a field, G a separated k-group scheme of finite type. Then the following statements are equivalent. i) G is k-linearly reductive (in newly introduced sense); ii) The functor V → V G from the category Rep(G) of G − representations to the category V ec of vector spaces is exact; iii) The functor V → V G from the category Repf in (G) of finite dimensional G − representations to the category V ec of vector spaces is exact; iv) Every G-representation is completely reducible. v) Every finite dimensional G-representation is completely reducible. vi) For every finite dimensional G-representation V and 0 = v ∈ V G , there exists a G-invariant functional F ∈ (V ∗ )G such that F (v) = 0. Regarding the geometric reductivity, we have the following well-known result, basically due to Mumford and Haboush 2.1.7. Theorem. (Cf. [Gros2, Thm. A, p.3], [MFK, Thm. A 1.0]) Let G be an affine algebraic group defined over a field k. The the following are equivalent. 1) G is reductive; 2) G is geometrically reductive; 3) For any affine k-algebra A with a G-action, the subalgebra of G-invariants AG is also an affine k-algebra. 2.1.8. Corollary. If G is a reductive group acting on an affine variety V , then the categorical quotient V //G always exists. Remarks. 1) Besides the original proof of the Mumford conjecture (that in 2.1.7, (1) ⇔ (2), given by Haboush, recently there was given a second proof by Sastry and Sesahdri [SaSe]. 2) Theorem 2.1.7 about the geometric reductivity can be stated in a more 6 general setting of schemes over very general bases, and we refer to [Se] as a standard reference for this. For yet in a more general stack framework, we mention the works by Alper (et al.) [Al1], [Al2], [AE], where there were given a fragments of a program of extending the major results of Geometric Invariant Theory to this new setting, by employing systematically the stack approach. 3) There is a vast literature on the Hilbert’s 14th Problem. We just indicate a few sources and refer the readers to these and the reference there: [Do], [Gr1], [Kr], [MFK]. Below we consider some examples related with the finiteness of the rings of invariants. Other examples, in particular those due to Nagata and Roberts, are given, say, in [Gr2, Sec. 8]). Examples. 1) ([G-P]) Let k be any commutative ring with 1, the dual number (i.e., 2 = 0, X a variable, and A := k[ , X]. Let Ga act on A via λ.f := f + λ(∂f /∂X). Then one checks that the ring of invariants AGa = k[ X, X 2 , ...], which is not finitely generated over k. 2) (Weitzenb¨ock) Let Ga act regularly on a vector space V over a field of characteristic 0. Then k[V ]Ga is finitely generated. 2.2 Observable and other related subgroups Isotropy subgroups (stabilizers). For v ∈ V , we denote by Gv the isotropy (or the same stabilizer) subgroup of v in G. If V is an affine (resp. quasi-projective) variety, then by 2.1.2 (resp. 2.1.3), the stabilizers can be realized also as stabilizers arising in a linear (resp. projective) representations. Due to the fact that there exists a natural bijection G/Gv G.v, which in fact, is an isomorphism of varieties in many cases, the study of the isotropy subgroups of a given group G also plays an important role. This lead to the study of the so-called observable subgroups. Let G be an affine algebraic group defined over an algebraically closed field k. Then G acts naturally on its regular function ring k[G] by right translation (rg .f )(x) = f (x.g), for all x, g ∈ G, f ∈ k[G]. For H a closed k-subgroup of G, we consider k[G]H := {f ∈ k[G] : rh .f = f, for all h ∈ H}, the k-subalgebra of H-invariant functions of k[G]. By convention, we identify the (smooth) affine algebraic groups considered 7 with their points in a fixed algebraically closed field. For a k-subalgebra R of k[G], we put R = {g ∈ G : rg .f = f, for all f ∈ R}. Then for any closed subgroup H ⊂ G we have H ⊆ (k[G]H ) ⊆ G. With a motivation from representation theory, Bialynicki-Birula, Hochschild and Mostow (see [BHM, p. 134]) introduced the concept of ”observable subgroup”. A closed subgroup H of G is called an observable subgroup of G if any finite dimensional rational representation of H can be extended to a finite dimensional rational representation on the whole group G (or, equivalently, if every finite dimensional rational H-module is a H-submodule of a finite dimensional rational G-module). In loc. cit. some equivalent conditions for a subgroup to be observable were given. Then Grosshans (see [Gr1], [Gr2, Chap. 1] and reference therein) has added several other conditions. It turned out later that for closed subgroups the property of being observable for a subgroup H is equivalent to the equality H = (k[G]H ) . Up to now there are known several equivalent conditions for a subgroup to be observable, which are more or less simple to verify and they are gathered in Theorem 2.2.1 below. Epimorphic subgroups. On the opposite side, a closed subgroup H ⊆ G may satisfy the equality (k[G]H ) = G. If it is so, H is called an epimorphic subgroup of G. In fact, under an equivalent condition, this notion was first introduced and studied by Bien and Borel in [BB1] [BB2] (see also [Gr2, Sec. 23, p. 132] for recent treatment), which in turn, is based on similar notion for Lie algebras, given by Bergman (unpublished). There were given several equivalent conditions for a closed subgroup to be epimorphic (see Theorem 2.2.10 below). Grosshans subgroups. In the connection with the solution of the 14th Hilbert problem, the following well-known problem is of great interest. Assume that X is an affine variety, G is a reductive group acting upon X morphically, H is a closed subgroup of G and consider the G-action on the regular function ring k[X] by left translation: (lg .f )(x) = f (g −1 .x). It is natural to ask when k[X]H is a finitely generated k-algebra. H For a closed subgroup H ⊂ G, we have k[X]H = k[X](k[G] ) (see [Gr1], [Gr2, Chap. 1]). On the other hand, it is well-known (loc. cit) that (k[G]H ) 8 is the smallest observable subgroup of G containing H. So the problem is reduced to the case when H is an observable subgroup. To solve this problem, Grosshans ([Gr1], [Gr2, Chap. 1, Sec. 4, 5]) introduced the codimension 2 condition for observable subgroups, and the subgroups satisfying this condition are called subsequently Grosshans subgroups of G (see below). Quasi-parabolic and subparabolic subgroups. Closely related to observable subgroups is the following class of subgroups. Recall that ([Gr2, p. 17]) a closed subgroup H of an affine algberaic group G is called quasiparabolic if H ⊂ G◦ and H is the isotropy subgroup of a highest weight vector for some (finite-dimensional) absolutely irreducible representation of G◦ . A connected closed subgroup H is called subparabolic if H ⊂ Q, Ru (H) ⊂ Ru (Q) for some quasi-parabolic subgroup Q of G, and in general a closed subgroup H is subparabolic if H ◦ is so. In this section we are interested in some questions of rationality related to observable, epimorphic, quasi-parabolic, subprabolic and Grosshans subgroups. The first rationality results regarding observable (resp. epimorphic) subgroups were already given in [BHM], and then in [Gr2], [W] (resp. [BB1], [BB2] and [W]), where some arithmetical applications to ergodic actions were also given. We give some other new results related to rationality properties of observable, epimorphic and Grosshans subgroups (which were stated initially for algebraically closed fields). Some arithmetic and geometric applications will be considered in another paper under preparation. Some rationality properties for observable groups. First we recall well-known results over algebraically closed fields. For an algebraic group G we denote by G◦ the identity connected component subgroup of G. 2.2.1. Theorem ([BHM], [Gr2, Theorems 2.1 and 1.12]) Let G be a linear algebraic group defined over an algebraically closed field k and let H be a closed k-subgroup of G. Then the following conditions are equivalent. a) H = (k[G]H ) . b) There exists a finite dimensional rational representation ρ : G → GL(V ) and a vector v ∈ V , all defined over k such that H = Gv = {g ∈ G : ρ(g).v = v}. c) There are finitely many functions f ∈ k[G/H] which separate the points in G/H. 9 d) G/H is a quasi-affine k-variety. e) Any finite dimensional rational k-representation ρ : H → GL(V ), can be extended to a finite dimensional rational k-representation ρ : G → GL(V ), where V → V , i. e., every finite dimensional rational H-module is a Hsubmodule of a finite dimensional rational G-module. f ) There is a finite dimensional rational k-representation ρ : G → GL(V ) and a vector v ∈ V such that H = Gv , the isotropy group of v, and G/H ∼ = G.v = {ρ(g).v : g ∈ G} (as algebraic varieties). g) The quotient field of the ring of G◦ ∩ H-invariants in k[G◦ ] is equal to the field of G◦ ∩ H-invariants in k(G◦ ). h) If 1-dimensional rational H-module M is a H-submodule of a finite dimensional rational G-module then the H-dual module M ∗ of M is also a H-submodule of a finite dimensional rational G-module. Examples. 1) If H is a normal closed subgroup of G, then H is observable in G. 2) Let χ be a character of G and let Hχ := Ker(χ). Then H is observable in G. In particular, if H has no character, then H is observable in G. 3) If H1 , H2 are observable subgroups of G, then so is H1 ∩ H2 . 4) If H is observable in K and K is observable in G then so is H in G. 5) Let B be a Borel subgroup of SL2 . Then the quotient SL2 /B is isomorphic to the projective line P1 , so B is not observable in SL2 . Now let k be any field. If a closed k-subgroup H of a linear algebraic k-group G satisfies the condition b) (resp. e)) in Theorem 1 where v ∈ V (k) and the corresponding representation ρ is defined over k, then we say that H is an isotropy k-subgroup of G (resp. has extension property over k). First we recall the following rationality results proved in [BHM, Theorems 5, 8]. 2.2.2. Theorem. ([BHM, Theorem 5]) Let G be a linear algebraic k-group, H a closed k-subgroup of G, K an algebraic field extension of k. Then H has extension property over k if and only if it has one over K. 2.2.3. Theorem. ([BHM, Theorem 8]) If H is a closed k-subgroup of a linear algebraic k-group G with extension property over k, then H is an isotropy 10 k-subgroup of G. Conversely, if k is algebraically closed and H is a isotropy k-subgroup then it has extension property over k. From Theorem 2.2.2 and Theorem 2.2.3, we derive the following. 2.2.4. Proposition. ([TB]) Let k be an arbitrary field and let H be a closed k-subgroup of a k-group G. The following two conditions are equivalent. a) H is an isotropy subgroup of G over k. b) H is an isotropy subgroup of G over k, i.e., there exists a finite dimensional k-rational representation ρ : G → GL(V ) and a vector v ∈ V (k) such that H = Gv . Remark. In [W], another proof of Proposition 2.2.4 was given, which is based on some ideas of Grosshans [Gr1], under the condition (which is not essential) that k = Q and H is connected. We consider k[G]H(k) = {f ∈ k[G] : rh .f = f, ∀h ∈ H(k)}, and (k[G]H(k) ) = {g ∈ G : rg .f = f, ∀f ∈ k[G]H(k) }. Then k[G]H(k) and k[G]H := {f ∈ k[G] : rh .f = f, ∀h ∈ H} are ksubalgebras of k[G]. In general we have the following diagram k[G]H(k) ¯ H(k) ⊆ k[G] | k[G]H | ⊆ ¯ H k[G] so we have (k[G]H(k) ) ¯ H(k) ) ⊇ (k[G] | (k[G]H ) | ⊇ 11 ¯ H) . (k[G] If, moreover, H(k) is Zariski dense in H then we have k[G]H(k) = k[G]H = k[G]H ∩ k[G]. We say that H is relatively observable over k if H = (k[G]H(k) ) , and H is k-observable, if (k[G]H ) = H. It is clear that if k is algebraically closed, then these notions coincide with the observability. We have the following obvious implication H is k-observable ⇒ H is observable. 2.2.5. Proposition. ([TB]) Let k be a field, and let H be a closed k-subgroup of a k-group G. Then a) H = k[G]H = k ⊗k k[G]H ; b) H is observable if and only if H is k-observable; c) Assume that H(k) is Zariski dense in H. Then H is observable ⇔ H is k-observable ⇔ H is relatively observable over k. 2.2.6. Proposition. ([TB]) Let H be a k-subgroup of a k-group G. The following are equivalent: a) There exist finitely many functions in k[G/H] which separate the points in G/H. b) There exist finitely many functions in k[G/H] which separate the points in G/H. 2.2.7. Proposition. ([TB]) Let G be a k-group, H a closed k-subgroup of G. Assume that, there exists finite dimensional k-rational representation ρ : G → GL(V ), and v ∈ V (k) such that H = Gv . Then there is a finite dimensional k-rational representation ρ : G → GL(W ) and w ∈ W (k) such that H = Gw and G/H ∼ =k G.w. The following fact is also very useful, which allows one to reduce to the case of connected groups. 2.2.8. Lemma. With above assumption, H is k-observable in G if and only if H ∩ G◦ is k-observable in G◦ . From results proved above, we have the following theorem, which is an analog of Theorem 2.2.1 for arbitrary fields. 12 2.2.9. Theorem. ([TB]) Let G be a linear algebraic group defined over a field k and let H be a closed k-subgroup of G. Then the following are equivalent. ¯ H ) , i. e., H is observable. a) H = (k[H] a ) H = (k[G]H ) , i.e., H is k-observable. b ) There exists a k-rational representation ρ : G −→ GL(V ) and a vector v ∈ V (k) such that H = Gv = {g ∈ G : g.v = v}. c ) There are finitely many functions f ∈ k[G/H] which separate the points in G/H. d ) G/H is a quasiaffine variety defined over k. e ) Any k-rational representation ρ : H −→ GL(V ), can be extended to a k-rational representation ρ : G −→ GL(V ). f ) There is a k-rational representation ρ : G −→ GL(V ) and a vector v ∈ V (k) such that H = Gv and G/H ∼ =k G.v = {ρ(g)v : g ∈ G}. g ) The quotient field of the ring of G◦ ∩ H-invariants in k[G◦ ] is equal to the field of G◦ ∩ H-invariants in k(G◦ ). If, moreover, H(k) is Zariski dense in H, then the above conditions are equivalent to the relative observability of H over k. Some further extensions. Recently, in a series of papers, J. Alper et al. (see e.g. [AE]) have generalized some of the results considered above for observable subgroup schemes basing on a very general framework of stacks. The following passage is based on [Al1, Al2, AE]. Let S be any scheme, G → S a flat finitely presented quasi-affine group scheme. Denote by OX the structure sheaf of the scheme X, BX the classifying stack of X. Then a flat, finitely presented quasi-affine subgroup scheme H ⊆ G is called observable if every quasi-coherent OS [H]-module is a quotient of a quasi-coherent OS [G]-module. In the case S = Spec(k), k is a field, this definition is equivalent to ours, namely every finite dimensional H-representation is a subH-representation of a finite dimensional Grepresentation. We have the following 13 2.2.9. Theorem (bis) [AE, Thm. 1.3] Let S be any scheme, G → S a flat finitely presented quasi-affine group scheme. Consider a flat, finitely presented quasi-affine subgroup scheme H ⊆ G. Then the following are equivalent. a) H is observable; b) For every quasi-coherent OS [H]-module F, the induced map IndG HF → F is a surjection of OS [H]-modules; c) BH → BG is quasi-affine; d) The quotient G/H → S is quasi-affine. If, moreover, S is a noetherian scheme, then all the above are equivalent to the following e) Every coherent OS [H]-module F is a quotient of a coherent OS [G]-modules; f ) For every coherent OS [H]-module F, the induced map IndG H F → F is a surjection of OS [H]-modules. Some rationality properties for epimorphic subgroups. We recall ¯ (after Grosshans [Gr2, p. 132]) that (k-)epimorphic subgroups H ⊆ G are ¯ H ) = G. We have those closed subgroups of G satisfying the condition (k[G] the following characterizations of epimorphic subgroups over an algebraically closed fields. 2.2.10. Theorem ([BB1, Th´eor`eme 1], [Gr2, Lemma 23.7]) Let H be a closed subgroup of G, all defined over an algebraically closed field k. Then the following are equivalent. a) H is epimorphic, i. e., (k[G]H ) = G. b) k[G/H] = k. c) k[G/H] is finite dimensional over k. d) If V is any rational G-module then the spaces of fixed points of G and H in V coincide. e) If V is a rational G-module such that V = X ⊕ Y , where X, Y are Hinvariant, then X, Y are also G-invariant. f ) Morphisms of algebraic groups from G to another L are defined by their values on H. Remark. The initial definition of epimorphic subgroups was given in [BB1], by only requiring that the condition f ) above hold. 14 Examples. 1) Let H be a closed subgroup of an affine algebraic group G. Then H is epimorphic in (k[G]H ) (k is algebraically closed). 2) If H is a closed subgroup epimoprhic in G then so is H ∩ G◦ in G◦ . Let notation be as above and let k be an arbitrary field. Then for a ksubgroup H of a k-group G we say that H is relatively epimorphic over k if (k[G]H(k) ) = G, and that H is k−epimorphic if (k[G]H ) = G. Recall that we have the following inclusions (k[G]H(k) ) ¯ H(k) ) ⊇ (k[G] | (k[G]H ) | ⊇ ¯ H) , (k[G] therefore, the following implications holds H is epimorphic ⇒ H is k-epimorphic, H is k-epimorphic ⇐ H is relatively epimorphic over k. In fact we have 2.2.11. Proposition. ([TB]) With above notation, if H is either (a) relatively epimorphic over k or (b) k-epimorphic, then it is also epimorphic. We have the following analog of Theorem 2.2.10 over an arbitrary field. 2.2.12. Theorem. Let k be any field and let H be a closed k-subgroup of a k-group G. Then the following are equivalent. a ) H is k-epimorphic, i.e., (k[G]H ) = G. b ) k[G/H] = k. c ) k[G/H] is finite dimensional over k. d ) For any rational G-module V defined over k, the spaces of fixed points of G and H in V coincide. e ) For any rational G-module V defined over k, if V = X ⊕ Y , where X, Y are H-invariant, then X, Y are also G-invariant. f ) Morphisms defined over k of algebraic k-groups from G to another one are defined by their values on H. 15 Remark. It was mentioned in [W, p. 195], that Bien and Borel (unpublished) have also proved that if G is connected, then d) ⇔ d ). (Here d) from Theorem 2.2.10 and d’) from 2.2.12.) Some rationality properties for Grosshans subgroups. One of the main results related with the finite generation problem (hence also with the Hilbert’s 14-th problem) mentioned in the Introduction is the following result of Grosshans (Theorem 2.2.14). First we recall the following very useful result which reduces to the case of connected groups. 2.2.13. Theorem. ([Gr2, Theorem 4.1]) Let k be an algebraically closed field. For any closed subgroup H of G, if one of the following k-algebras ◦ ◦ ◦ k[G]H , k[G]H , k[G◦ ]H∩G , k[G◦ ]H is a finitely generated k-algebra, then the same holds for the other. 2.2.14. Theorem. ([Gr2, Theorem 4.3]) For an observable subgroup H of a linear algebraic group G, all defined over an algebraically closed field k, the following are equivalent. a) There is a finite dimensional rational representation ϕ : G → GL(V ), an element v ∈ V , such that H = Gv and each irreducible component of G.v − G.v has codimension ≥ 2 in G.v. b) The k-algebra k[G]H is a finitely generated k-algebra. If b) holds, let X be an affine variety with k[X] = k[G]H , and with G-action via left translations of G on G/H. There is a point x ∈ X such that G.x is open in X and G.x G/H via gH → g.x and each irreducible component of X \ G.x has codimension ≥ 2 in X. The observable subgroups which satisfy one of the equivalent conditions in Theorem 2.2.14 are called Grosshans subgroups (see [Gr2, Chap. 1, Sec. 4]). There are some nice geometrical characterizations and examples of Grosshans subgroups presented in there and the reference therein. For a field k, a k-group G and an observable k-subgroup H ⊂ G, we say that H satisfies the codimension 2 condition over k if H satisfies condition a) above where V, ϕ are all defined over k and v ∈ V (k). We call H a Grosshans subgroup relatively over k (resp. k-Grosshans subgroup) of G if k[G]H(k) (resp. k[G]H ) is a finitely generated k-algebra. 16 Examples. 1) Let H be a closed subgroup of G. Then the following are equivalent: a) H is a Grosshans subgroup of G; b) H ∩ G◦ is a Grosshans subgroup of G◦ ; c) H ◦ is a Grosshans subgroup of G◦ . 2) If K < H are observable subgroups of G and K is a Grosshans subgroup of G, then so is K in H. 3) If H is a reductive subgroup of G, then H is a Grosshans subgroup in G. Now let k be any field We have a result similar to Theorem 2.2.14 for k-Grosshans subgroups. 2.2.15. Theorem. ([TB]) Let k be an infinite perfect field, G a connected k-group. Assume that H is a observable k-subgroup of G. Consider the following conditions. a ) H satisfies codimension 2 condition over k. ◦ ◦ ◦ b ) One of the k-algebras k[G]H , k[G]H , k[G◦ ]H∩G , k[G◦ ]H is a finitely generated k-algebra. c ) H is a Grosshans subgroup relatively over k of G (i.e k[G]H(k) is finitely generated k-algebra). Then, together with conditions in Theorem 15, we have the following implications a) ⇔ a ) ⇔ b) ⇔ b ) ⇒ c ). If, moreover, H(k) is Zariski dense in H, then all these conditions are equivalent. Remark. It is of interest to find examples where the condition c ) holds but the other conditions do not. It will, perhaps, ultimately lead to another counter-examples to (generalized) 14-th Hilbert’s Problem in the case of char.k > 0. (Various extensions of classical results in (geometric) invariant theory to the case of characteristic p > 0 were discussed at length in [MFK, Appendices].) It will be more interesting to have examples with G, H connected groups. A relation with the subalgebra of invariants of a Grosshans subgroup of a reductive group acting rationally upon a finitely generated commutative algebra is given in the following 2.2.16. Theorem. [Gr2, Theorem 9.3].) Let k be an algebraically closed field. For any closed subgroup H of a reductive group G, all defined over k, 17 the following are equivalent. a) k[G]H is a finitely generated k-algebra. b) For any finitely generated, commutative k-algebra A on which G acts rationally, the algebra of invariants AH is a finitely generated k-algebra. We consider the following relative version of this theorem. 2.2.17. Theorem. ([TB]) Let k be an infinite perfect field, H a closed k-subgroup of a connected reductive k-group G. Consider the following conditions. a ) k[G]H is a finitely generated k-algebra. b ) For any finitely generated, commutative k-algebra Ak on which G acts k-rationally, the algebra of invariants AH k is a finitely generated k-algebra. c ) For any finitely generated, commutative k-algebra Ak on which G acts H(k) k-rationally, the algebra of invariants Ak is a finitely generated k-algebra. Then with notations as in Theorem 2.2.16 we have a) ⇔ a ) ⇔ b) ⇔ b ) ⇒ c ). If, moreover, H(k) is Zariski dense in H, then all conditions above are equivalent. Some rationality properties of subparabolic subgroups. We consider the following relative notions of quasi-parabolic and subparabolic subgroups. a) For a k-group G, a subgroup Q of G0 is said to be k-quasiparabolic in G if Q = G0v for a highest weight vector v ∈ V (k) of some absolutely irreducible k − G0 -module V . Here V (k) denotes the set of k-points of V with respect to a fixed k-structure of V ([Bo2, Section 11.1]). b) For a k-group G, a subgroup H of G is called k-subparabolic if it is defined over k and there is a k-quasiparabolic subgroup Q of G0 such that H 0 ⊆ Q and Ru (H) ⊆ Ru (Q). Note that in the literature, a closed subgroup Q of G0 is called quasiparabolic ¯ if it is k-quasiparabolic and a closed subgroup H of G is called subparabolic ¯ if it is k-subparabolic and then we are back to the usual notions introduced above. a’) For a k-group G, a subgroup Q of G0 is said to be quasiparabolic over k (or quasiparabolic k-subgroup) if it is defined over k and quasiparabolic. b’) For a k-group G, a subgroup H of G is called subparabolic over k (or 18 subparabolic k-subgroup) if it is defined over k and subparabolic. H is called strongly subparabolic over k if there is a quasiparabolic k-subgroup Q of G0 such that H 0 ⊆ Q and Ru (H) ⊆ Ru (Q). (Thus, being strongly subparabolic over k is a priori stronger than just being subparabolic over k.) One of important theorems in geometric invariant theory is due to Bogomolov which relates the stabilizer subgroup of an unstable vector to some quasiparabolic subgroup. Its relative version below provides the abundance of k-quasiparabolic subgroups. It is also one of main results of this paper. ¯ 2.2.18. Theorem. (Cf. [Bog1, Theorem 1], [Gr2, Theorem 7.6] when k = k, [BaT3] when k is perfect) Let k be a perfect field, G a connected reductive k-group and let V be a finite dimensional k-G-module. Let v ∈ V (k) \ {0}. If v is unstable for the action of G on V (i.e., 0 ∈ G.v), then Gv is contained in a proper k-quasiparabolic subgroup Q of G. Remark. We note that the original proof in [Bog1] (cf. also [Bog2], [Ro2]) is given for algebraically closed fields and does not seem to extend to arbitrary perfect fields. Since we make an essential use of Kempf - Rousseau results (see below), which does not seem to be extended to the case of non-perfect fields as noted in [Ro1] (cf. also [He]), our approach does not cover this case. As an application of Theorem 2.2.18 and also of other results, we establish the following second main result of this section about rationality properties of quasiparabolic, subparabolic and observable subgroups of a linear algebraic group G defined over a perfect field k. 2.2.19. Theorem. ([BaT3]) Let k be a perfect field, G a linear algebraic k-group, H a closed k-subgroup of G. We consider the following statements. 1) H is k-quasiparabolic; 2) H is quasiparabolic over k; 3) H is observable over k; 4) H is k-subparabolic; 5) H is strongly subparabolic over k. 6) H is subparabolic over k. Then we have 1) ⇒ 2) ⇒ 3) ⇔ 4) ⇔ 5) ⇔ 6). If, moreover, G is semisimple, then 1) ⇔ 2). 19 Remarks. 1) In general, there are examples show that in Theorem 2.2.19, 3) ⇒ 2) ⇒ 1). ¯ 3) ⇔ 4) above is Sukhanov’s Theorem (cf. [Suk], 2) In the case k = k, [Gr2]). The proof of Sukhanov’s Theorem in the absolute case (see [Suk], or [Gr2, Theorem 7.3], with some refinements) makes an essential use of the important theorem due to Bogomolov mentioned above. The same happens while we prove the relative version: we make an essential use of Theorem 2.2.17 and other related results. 3 3.1 Stability in geometric invariant theory over non-algebraically closed fields Stability To establish the relative version of Bogomolov’s theorem and also other versions of Sukhanov’s theorem, we need to use the Kempf’s-Rousseau’s Instability theory. This appears to be very important in the arithmetic invariant theory. In fact, from the very beginning of the modern geometric invariant theory, to study the closedness of G-orbits, one encounters immediately the notion of stability, which plays a crucial role. Let an affine algebraic group G act on a vector space V via a linear representation ρ : G → GL(V ), all are defined over a field k. We say (after Kempf [Ke]) that a non-zero vector v ∈ V is stable (resp. unstable, semistable) if G.v is closed (resp. G.v is not closed, Cl(G.v) does not contain 0). A stable vector is called properly stable if moreover, the stabilizer Gv of v is finite. The following important theorem has been proved by Kempf [Ke] and Rousseau [Ro1], [Ro2] (independently and differently). Let Y (G) := M or(Gm , G) the set of all one-parameter subgroups of G. Let f : Gm → V be a morphism of algebraic varieties. If f can be extended to a morhism f˜ : Ga → V , with f˜(0) = v, then we write f (t) → v while t → 0, or lim f (t) = v. t→0 3.1.1. Theorem (Cf. [Ke, Thm. 1.4]) Let G be a reductive group acting on an affine variety V, all defined over a perfect field k. Let v ∈ V (k) be a k-point, S a closed G-stable subvariety of V such that S ∩ Cl(G.v) = ∅. Then there exists a one-parameter subgroup λ ∈ Y (G) defined over k, such that the limit lim λ(t).v exists and belongs to S. t→0 20 Equivalently, it can be stated by making use of the numerical criterion for instability as follows. To each λ ∈ Y (G) there corresponds a representation ϕ : Gm → GL(V ), ϕ = λ ◦ ρ, which is diagonalizable. We write V = ⊕Vn , where Vn := {x ∈ V | ϕ(t).x = tn .x, ∀ t ∈ Gm } and set v = n vn , vn ∈ Vn and define µ(v, λ) := M ax{−n | vn = 0}. 3.1.1.(bis) Theorem. (Cf. [Ro2]) With above notation, v is an unstable vector for G if and only if there exists λ ∈ Y (G) defined over k, such that µ(v, λ) < 0. These theorems vastly generalizes a classical result due to Hilbert - Mumford (cf. e.g. [Kr, Chap. III, Sec. 2.1]), which says that if k is algebraically closed, G a linearly reductive k-group acting on a k-vector space V via a linear k-representation, and v ∈ V is an unstable vector. Then there exists a one-parameter subgroup λ : Gm → G such that lim λ(t).v = 0. t→0 3.2 Some further extensions After the works by Kempf [Ke] and Rousseau [Ro1], [Ro2], on the one hand one should note the works by Hesselink [He], Raman - Ramanathan [RR] and Coia - Holla [CH], and [ADK] where some further more general results over fields, which is possibly imperfect. On the other hand, in other directions, we note the works by M. Bate, G. Martin and G. R¨ohrle ([BMRT1], [BMRT2]), where the investigation of stability in relation with the so-called Center conjecture due to Tits have been made. 3.3 Some other notions and questions related with stability In [Bo1], A. Borel raised several questions concerning the relation between the closedness of the orbits (geometric property) and the anisotropicity of the given group G (arithmetic property). We recall these questions below and indicate some developments on the way of finding answers to them. I) (Borel [Bo1, Sec.8.8]) Let k be a perfect field, G a connected reductive 21 k-group with X ∗ (G)(k) = {1}. a) Is there a faithful representation G → GL(V ), all defined over k, such that for all v ∈ V (k) \ {0}, cl(G.v) does not contain the 0 ? b) Is there a faithful representation G → GL(V ), all defined over k, such that for all v ∈ V (k) \ {0}, G.v is closed ? c) Is it true that for any representation G → GL(V ), all defined over k, and for all v ∈ V (k) \ {0}, cl(G.v) does not contain the 0 ? d) Is it true that for any representation G → GL(V ), all defined over k, and for all v ∈ V (k) \ {0}, G.v is closed ? II) Let k be an imperfect field, G an anisotropic connected reductive k-group. One considers the same questions as above, denoted by a’), b’), c’) and d’). As Borel noted, c’) and d’) may have the answer ”no” in the imperfect case. In connection with Borel’s questions, in one of the early papers on geometric invariant theory over non-algebraically closed fields, D. Birkes [Bi] introduced the following Properties A, B, C of representations of algebraic groups. Property A. (Cf. [Bi]) Let ρ : G → GL(V ) be any linear representation of an affine algebraic group G, all are defined over a field k. Let x ∈ V (k) be a point of instability for G, Y a non-empty G-invariant closed subset in Cl(G.x) \ G.x. Then there exists y ∈ Y , λ ∈ Y (G) defined over k, such that limt→0 λ(t).x = y. Thus, in the simplest case of action of algebraic groups on a vector space, it simply says that Theorem 3.1.1 holds. Therefore, by Kempf - Rousseau Theorem 3.1.1 (proved a bit later), any reductive group G has Property A over any perfect field. Property B. ([Bi]) Let ρ : G → GL(V ) be any linear representation of an affine algebraic group G, all are defined over a field k. Let x ∈ V (k), such that the stabilizer Gx contains a maximal k-split torus of G. Then the orbit G.x is closed. Property C. ([Bi]) Let ρ : G → GL(V ) be any linear representation of an affine algebraic group G, all are defined over a field k. Let x ∈ V (k). Then the orbit G.x is closed. 22 We want to sketch the proof of the fact that the Property B also holds for any reductive group over perfect fields, as a consequence of Property A (or Theorem 3.1.1). Thus, all the Borel’s questions I) have a positive answer. The necessary reduction has already been done by Birkes. Recall that a reductive k-group G is called anisotropic over k, if G has no non-trivial split k-subtori. 3.1.2. Lemma. ([Bi, Sec. 6]) 1) Let k be a field, G an affine algebraic k-group. If G◦ the connected component of the identity of G has Property B, then so does G. 2) If G is a connected k-group with unipotent radical Ru (G) defined over k. If G/Ru (G) has Property B, then so does G. 3) If every connected k-anisotropic reductive algebraic group defined over a field k has Property B, then so does every reductive group over k. 3.1.3. Lemma. ([Bi, Sec. 6,7]) Let k be a perfect field. If every connected reductive k-anisotropic group G over a perfect field k has Property B, then so does any affine algebraic group over k. Proof. ([Bi]) Let G be any k-group. By 3.1.2, 1), we may assume that G is connected. The unipotent radical Ru (G) of G is k-closed and since k is perfect, Ru (G) is also defined over k, so we are reduced to proving the same thing for G/Ru (G) by 3.1.2,2), and by 3) we are done. 3.1.4. Proposition. ([Bi], [Ke]) Over a perfect field k, any k-group G has Property B. Proof. By 3.1.3, we may assume that G is a connected, reductive and kanisotropic k-group. Let ρ : G → GL(V ) be any linear representation, all are defined over k. If there were an unstable vector x ∈ V (k), then Y := Cl(G.x) \ G.x = ∅ and by Theorem 3.1.1, there would exist a nontrivial one-parameter subgroup λ : Gm → G defined over k. The image of λ would then be a non-trivial k-split subtorus of G, which contradicts our assumption that G is anisotropic. 23 Regarding Property C, it was proved in [Bi, 10.1] that the converse statement for I), b) holds. More precisely we have 3.1.5. Lemma. (Cf. [Bi, Lem. 10.1]) Let G be an affine algebraic kgroup. If for some faithful representation G → GL(V ) over k, the orbit G.x is closed for all x ∈ V (k), then G is anisotropic over k. From above, assuming that k is perfect, one finds the following necessary and sufficient condition for G to have Property C. 3.1.6. Proposition. 1) (Cf. [Bi, Prop. 10.2]) Let G be an affine algebraic group defined over a field k. If G has Property B, then G has Property C if and only if G is anisotropic. 2) If k is perfect, then G has Property C if and only if G is k-anisotropic. Finally, we mention the following general result due to Birkes [Bi, Prop. 9.10], which says that the nilpotent groups has Property A over any field. 3.1.7. Proposition. ([Bi, Prop. 9.10.]) Let k be an arbitrary field, G a smooth nilpotent group acting linearly on a finitely dimensional vector space V via a representation ρ : G → GL(V ), all defined over k. If v ∈ V (k), Y is a non-empty G-stable closed subset of Cl(G.v) \ Gv, then there exist an element y ∈ Y ∩ V (k), a one-parameter subgroup λ : Gm → G defined over k, such that λ(t).v → y while t → 0. One should also note that for the Property A to hold, one cannot go beyond the class of nilpotent groups, or the class of reductive groups, as the example of [Bi, p. 474] shows. 24 4 Topology of relative orbits for actions of algebraic groups over completely valued fields Let G be a smooth affine (i.e. linear) algebraic group acting regularly on an affine variety X, all are defined over a field k. Due to the need of numbertheoretic applications, the local and global fields and relared rings k are in the center of research of arithmetic invariant theory. For example, let an algebraic k-group G act on a k-variety V , x ∈ V (k). We are interested in the set G(k).x, which is called relative orbit of x (to distinguish with geometric orbit G.x). One of the main steps in the proof of the analog of Margulis’ super-rigidity theorem in the global function field case (see [Ve], [Li], [Ma]) was to prove the (locally) closedness of some relative orbits G(k).x, x ∈ V (k), for some action of an almost simple simply connected group G on a k-variety V . Thus one may pose the General Problem: Study the orbits G.x, G(k).x, x ∈ V (k). Questions: a)(geometric case): When is G.x a Zariski closed subset of V ? b) (relative case): If k has a Hausdorff topology (e.g. k is a local field), x ∈ V (k), when is G(k).x is closed in Hausdorff topology in V (k) ? c) Relation between a) and b) ? Remarks. Question a) (geometric case) above is a subject of Geometric Invariant Theory, question b) (relative case) is a subject of geometric invariant theory over non-algebraically closed fields. Finally, the question c), if k is of arithmetic nature (say k is the ring of integers in a local or global field, or ad`ele ring of the later, it is a subject of Arithmetic Invariant Theory. 4.1 On Borel - Harish-Chandra and Bremigan’s results In this section we assume that k is a field, complete with respect to a nontrivial valuation v of real rank 1, (e.g. p-adic field or the field of real numbers R, i.e., a local field). Then for any affine k-variety V , we can endow V (k) with the (Hausdorff) v-adic topology induced from that of k. Let an affine algebraic group G act k-regularly on V , x ∈ V (k) be a k-point. We are interested in a connection between the Zariski-closedness of the orbit G.x of 25 x in V , and Hausdorff closedness of the relative orbit G(k).x of x in V (k). The first result of this type was obtained by Borel and Harish-Chandra ([BHC]) and then by Birkes ([Bi], see also Slodowy [Sl]) in the case the real field k = R and then by Bremigan (see [Bre, Sec. 5]) in the case of non-archimedean local fields of characteristic 0. In fact, it was shown the following. 4.1.1. Theorem. 1) (Cf. [BHC, Prop. 2.3]) With notation as above, let moreover k = R. If G(R).x is Hausdorff closed in V (R), then G.x is Zariski closed in V. 2) (Cf. [Bi, Corol. 5.3]) Conversely, if G is reductive over R and G.x is Zariski closed in V, then so is G(R).x in Hausdorff topology in V (R). 3) (Cf. [Bre, Prop. 5.3]) Let k be a local field of characteristic 0, G a reductive k-group. Then G.x is Zariski closed if and only if G(k).x is Hausdorff closed in V (k). Notice that some of the proofs previously obtained in [Bi], [BHC], [Bre] do not extend to the case of positive characteristic, thus it requires a special consideration in next section. 4.2 Topology of relative orbits over general fields The aim of this section is to see to what extent the above results still hold for more general class of algebraic groups and complete fields, especially in the case of positive characteristic. In the course of study, it turns out that this question has a close relation with the problem of equipping a topology on cohomology groups (or sets), which has important aspects, say in duality theory for Galois or flat cohomology of algebraic groups in general (see [Se], [Mi2], [Sh1,2]). We emphasize that, in the case char.k = p > 0, the stabilizer of a (closed) point needs not be a smooth subgroup, and the treatment of smoothness condition plays an important role here. The most satisfactory results are obtained for perfect fields, and also for a general class of groups over local fields. We have the following general results regarding some relations between the topology of relative orbits and that of geometric orbits. 4.2.1. Theorem. Let k be a field, complete with respect to a real valuation of rank 1, G a smooth affine k-group, acting k-regularly on an affine k-variety V, v ∈ V (k), and Gv the stabilizer of v in G. 26 1. ([BaT5]) If Gv is a k-group having a smooth k-subgroup of the same dimension, then the relative orbit G(k).v is Hausdorf closed in (G.v)(k). If moreover, G.v is Zariski closed in V, then G(k).v is Hausdorf closed in V (k). b) ([BaT2,BaT5, Bre]) If Gv is a smooth k-subgroup of G, then for any w ∈ (G.v)(k), the relative orbit G(k).w is open and closed in Hausdorff topology of (G.v)(k). 2. Conversely, let G(k).v be Hausdorff closed in V (k). If either a) G is nilpotent, or b) G is reductive and the action of G is strongly separable, then G.v is Zariski closed in V. Therefore, in these cases, G.v is Zariski closed in V if and only if G(k).v is Hausdorff closed in V (k). 3. Assume further that k is a perfect field, G = L × U , where L is a reductive and U is a unipotent subgroup of G, L is defined over k, V,v are as above. Then G(k).v is Hausdorff closed in V (k) if and only if G.v is Zariski closed in V. Here the action of G is said to be strongly separable (after [RR]) at v if for all x ∈ Cl(G.v), the stabilizer Gx is smooth, or equivalently, the induced morphism G → G/Gx is separable. One of the main tools to prove the theorem is the introduction of some specific topologies on the (Galois or flat) group cohomology and their relation with the problem of detecting the closedness of a given relative orbit. The main ingredient is the following theorem proved in [BaT4], where we refer to Section 1 for the notion of special and canonical topology on the cohomology set H1f lat (k, G) 4.2.2. Theorem. ([BaT1], [BaT6]) Let G be an affine group scheme of finite type defined over a field k, complete with respect to a valuation of real rank 1. Then 1) The special and canonical topologies on H1f lat (k, G) coincide. 2) Any connecting map appearing in the exact sequence of cohomology in degree ≤ 1 induced from a short exact sequence of affine group schemes of finite type involving G is continuous with respect to canonical (or special) topologies. As a consequence of the above results, we have the following 4.2.3. Theorem. Assume that k is a completely valued field, G acts on 27 a vector space V via a linear representation, all are defined over k. Then the set of all vectors v ∈ V (k), which have relative closed orbits, is Zariski dense in V. Proof. It is well-known that the set V (k) of k-rational points of V is Zariski dense. By a theorem of Rosenlicht, the set V s of all points v ∈ V such that v is stable (i.e. G.v is Zariski closed) is a Zariski open non-empty subset in V . Since it is non-empty, V s (k) = V s ∩ V (k) is Zariski dense in V s , thus also dense in V . Since for any v ∈ V s (k), G(k).v is Hausdorff closed by a result from [Bre] (see 4.1.1), in characteristic 0 and [BT] (see 4.2.1) in any characteristic, the theorem follows. We have the following relation between the ambient nilpotent groups and their maximal tori. 4.2.4. Proposition. ([BaT2]) Let k be a perfect field complete with respect to a non-trivial real valuation of rank 1. Assume that G is nilpotent, T is the unique maximal k-torus of G. Then the following statements are equivalent. a) G · v is closed in Zariski topology; b) T · v is closed in Zariski topology; c) G(k) · v is closed in Hausdorff topology; d) T (k) · v is closed in Hausdorff topology. In the case of solvable groups, we find some contrasts to the nilpotent case. 4.2.5. Proposition. ([BaT2]) Let k be either R or Qp , G a solvable linear algebraic group defined over k, T an arbitrary maximal k-torus of G, and let G act k-linearly on a finitely dimensional k-vector space V . Let v ∈ V (k) and consider the following statements. a) G.v is closed in Zariski topology; b) T.v is closed in Zariski topology; c) G(k).v is closed in Hausdorff topology; d) T (k).v is closed in Hausdorff topology. Then we have the following logical scheme : b) ⇔ d), a) ⇒ c), a) ⇒ b), 28 b) ⇒ a), c) ⇒ d), d) ⇒ c), c) ⇒ a). Here the non-implications means that there are examples of G, V, v, for which the implications do not hold. 4.3 Some examples The following examples show that in any characteristic, one should not expect for a close relationship between the two types of “closedness” for nonreductive groups. Example 1. Assume that k is a local field of characteristic 0, G = SL2 , B, the Borel subgroup of G, consisting of upper triangular matrices. Consider the standard representation of G with standard action of G on the space V2 of ¯ considhomogeneous polynomials in X, Y of degree 2 with coefficients in k, 2 ¯ ered as 3-dimensional k-vector space with the canonical basis {X , XY, Y 2 }. Then for v = (1, 0, 1) ∈ V2 , we have a) G.v is closed in the Zariski topology, the stabilizer Gv is finite and G(k).v is Hausdorff closed. b) B.v = {(x, y, z) | 4xz = y 2 + 4} \ {z = 0} is not Zariski closed. c) B(k).v = {(a2 + b2 , 2bd, d2 ) | ad = 1, a, b, c, d ∈ k} is closed in the Hausdorff topology in V (k), where k is either R or a p-adic field Qp with p = 2 or p ≡ 3 (mod. 4). We can show that the similar example works in any characteristic p > 0. The following example shows that one cannot hope for such a relationship even for semisimple groups. Example 2. Let p be a prime, k = Fq ((T )), q = pr , G = SL2 , B = Borel subgroup of G as in Example 1, ρ = the representation of G into 2dimensional k-vector space V given by ρ : G = SL2 → GL2 , g = a b c d → ap bp cp dp , v = (1, T ) ∈ V (k) = k × k. Then 1) G · v = V \ {(0, 0)} is open (and not closed) in the Zariski topology in V 29 and G(k) · v is closed in the Hausdorff topology in V (k). 2) B ·v = {(x, y) ∈ V | y = 0} is open (and not closed) in the Zariski topology in V and B(k) · v is closed in the Hausdorff topology in V (k). Acknowledgement. The present paper is based on the talk I gave in Kunming thanks to the invitation and support from Professor Shing Tung Yau, Professor LiZhen Ji and Professor Athanase Papadopoulos and the KUST, whom I would like to express my sincere thanks. The authors also thank Prof. J. Alper for interesting discussion related with topics of this paper. References [Al1] J. Alper, Good moduli spaces for Artin stacks, Annales de l’Institute Fourier (to appear) (see also arXiv:0804.2242v3 [mathAG] 19 October 2009). [Al2] J. Alper, Adequate moduli spaces and geometrically reductive group schemes, arXchiv:1005.2398v1 [math.AG], 13 May 2010. [AE] J. D. Alper and R. W. Easton, Recasting results in equivariant geometry: affine cosets, observable subgroups and existence of good quotients. Transform. Groups 17 (2012), no. 1, 1 20. [ADK] A. Asok, B. Doran and F. Kirwan, Yang - Mills theory and Tamagawa numbers: The fascination of unexpected links in Mathematics, Bull. London Math. Soc. v. 40 (2008), 533 - 567. [BaT1] D. P. Bac and N. Q. Thang, On the topology on group cohomology of algebraic groups over local fields, in: Proc. of the International Conf. on Research in Math. And Educations, Univ. Putra Malaysia. Malaysia, 2009. pp. 524 - 530. (ISBN: 978-967-344-092-4). [BaT2] D. P. Bac and N. Q. Thang, On the topology of relative orbits for actions of algebraic groups over complete fields, Proc. Jap. Acad. ser. A., v. 86 (2010), 133 - 138. [BaT3] D. P. Bac and N. Q. Thang, On a relative version of a theorem of Bogomolov and its applications, J. Algebra, v. 324 (2010), No. 5, 1259-1278. 30 [BaT4] D. P. Bac and N. Q. Thang, On the topology of relative orbits for actions of algebraic tori over local fields, J. Lie Theory, v. 22 (2012), 1025 - 1038. [BaT5] D. P. Bac and N. Q. Thang, On the topology of relative and geometric orbits for actions of algebraic groups over complete fields, J.Algebra, v.390 (2013), No. 1, 181 - 198; Erratum (2014, to appear). [BaT6] D. P. Bac and N. Q. Thang, On the topology on group cohomology of algebraic groups over complete fields, J.Algebra, v. 399 (2014), 561 - 580. [BMRT1] M. Bate, B. Martin, G. Roehrle, R. Tange, Closed orbits and uniform S -instability in geometric invariant theory. Trans. Amer. Math. Soc. 365 (2013), no. 7, 36433673. [BMRT2] M. Bate, B. Martin, G. Roehrle, The strong Centre Conjecture: an invariant theory approach. J. Algebra 372 (2012), 505530. [BB1] F. Bien et A. Borel, Sous-groupes ´epimorphiques de groupes lin´eaires alg´ebriques, I, C. R. Acad. Sci. Paris. Ser. I, 315 (1992), 649 - 653. [BB2] F. Bien et A. Borel, Sous-groupes ´epimorphiques de groupes lin´eaires alg´ebriques, II, C. R. Acad. Sci. Paris. Ser. I, 315 (1992), 1341 - 1346. [Bi] D. Birkes, Orbits of algebraic groups, Annals of Math. v. 93 (1971), 459 - 475. [Bog1] F. A. Bogomolov, Holomorphic tensors and vector bundles on projective varieties, Math. U. S. S. R. Izvestiya v. 13 (1979), 499-555. [Bog2] F. Bogomolov, Unstable vector bundles and curves on surfaces, in ”Proceedings of the International Congress of Mathematicians” (Helsinki, 1978), pp. 517–524, Acad. Sci. Fennica, Helsinki, 1980. [Bo1] A. Borel, Introduction aux groupes arithm´etiques, Hermann-Paris, 1968. 31 [Bo2] A. Borel, Linear algebraic groups (second enlarged version), GTM 126, Springer - Verlag, 1991. [BHC] A. Borel and Harish-Chandra, Arithmetic subgroups of algebraic groups, Annals of Math. v.75 (1962), 485 -535. [Bre] R. Bremigan, Quotients for algebraic group actions over nonalgebraically closed fields. J. reine und angew. Math. Bd. 453 (1994), 21- 47. [CH] F. Coiai and Y. Holla, Extension of structure group of principal bundle in positive characteristic, J. reine und angew. Math. Bd. 595 (2006), 1 - 24. [Do] I. Dolgachev, Lectures on invariant theory. London Mathematical Society Lecture Note Series, 296. Cambridge University Press, Cambridge, 2003. xvi+220 pp. [G-P] G. Greuel and G. Pfister, Geometric quotients of unipotent group actions, Proc. London. Math. Soc. v. 67 (1993), 75 -105. [Gr1] F. Grosshans, Observable groups and Hilbert’s fourteenth problem, Amer. J. Math. v. 95 (1973), 229 - 253. [Gr2] F. Grosshans, Algebraic Homogeneous Spaces and Invariant Theory. Lec. Notes in Math. 1673. Springer-Verlag, Berlin-HeidelbergNew York, 1997. [He] W. Hesselink, Uniform instability in reductive groups, J. reine. angew. Math. Bd. 303 (1978), 74 - . [Hum] J. Humphreys, Linear Algebraic Groups, Graduate Text in Math. v. 48, Second ed. Springer-Verlag, Berlin-Heidelberg-New York, 1984. [K] G. Kemper, A characterization of linearly reductive groups by their invariants. Transform. Groups v. 5 (2000), 8592. [Ke] G. Kempf, Instability in invariant theory, Annals of Math. v. 108 (1978), 299 - 316. [Kr] H. -P. Kraft, Geometrische Methoden in der Invariantentheorie, Vieweg und Sohn, 1985. 32 [KSS] H. Kraft, P. Slodowy and T. Springer (Eds.), Algebraische Transformationsgruppen und Invariantentheorie, DMV Sem., 13, Birkh¨auser, Basel, 1989. [Li] L. Lifschitz, Superrigidity theorems in positive characteristic, J. Algebra 229 (2000), 375 - 404. [Ma] G. Margulis, Discrete subgroups of semisimple Lie groups, Springer - Verlag, 1991. [Mi1] ´ J. Milne, Etale cohomology. Princeton Mathematical Series, 33. Princeton University Press, Princeton, N.J., 1980. [Mi2] J. Milne, Arithmetic duality theorems. Second edition. BookSurge, LLC, Charleston, SC, 2006. [Mu] D. Mumford, Geometric Invariant Theory, Ergeb. der Math. und ihrer Grenzgebiete Bd. 34, Springer - Verlag, 1965. [MFK] D. Mumford, J. Fogarty and F. Kirwan, Geometric Invariant Theory (expanded version) Ergeb. der Math. und ihrer Grenzgebiete Bd. 34. Springer-Verlag, Berlin-Heidelberg-New York, 1994. [Ra] M. Raghunathan, A note on orbits of reductive groups, Proc. Indian Math. Soc. v. 38 (1974), 65 - 70. [RR] S. Ramanan and A. Ramanathan, Some remarks on instability flag, Tohoku Math. J. v. (2) 36 (1984), 269-291. [Ro1] G. Rousseau, Instabilit´e dans les espaces vectoriels, in : Surfaces alg´ebriques (Orsay, 1976–78), pp. 263–276, Lecture Notes in Math., v. 868, Springer, Berlin-New York, 1981. [Ro2] G. Rousseau, Instabilit´e dans les fibr´es vectoriels (d’apr`es Bogomolov), in : Surfaces alg´ebriques (Orsay, 1976 -78), pp. 277 - 292, Lecture Notes in Math., v. 868, Springer, Berlin-New York, 1981. [Ro3] G. Rousseau, Immeubles sph´eriques et th´eorie des invariants. C. R. Acad. Sci. Paris Sr. A-B 286 (1978), no. 5, A247–A250. 33 [SaSe] Sastry, Pramathanath; Seshadri, C. S. Geometric reductivitya quotient space approach. J. Ramanujan Math. Soc. 26 (2011), no. 4, 415477. [Se] Seshadri, C., Geometric reductivity over arbitrary base, Advances in Math. 26 (1977), 225 - 274. [Sl] P. Slodowy, Zur Geometrie der Bahnen reeller reduktiver Gruppen. Algebraische Transformationsgruppen und Invariantentheorie, DMV Sem., 13, Birkh¨auser, Basel, 1989, pp. 133 - 143. [Sp] T. Springer, Aktionen reductiver Gruppen, Algebraische Transformationsgruppen und Invariantentheorie, DMV Sem., 13, Birkh¨auser, Basel, 1989, pp. 3 - 39. [Suk] A. A. Sukhanov, Description of the observable subgroups of linear algebraic groups, Math. U. S. S. R. Sbornik v. 65, n. 1, (1990), 97-108. [Su1] H. Sumihiro, Equivariant completion, I. J. Math. Kyoto Univ. v.14 (1974), 1 - 28. [Su1] H. Sumihiro, Equivariant completion, II. J. Math. Kyoto Univ. v.15 (1974), 573 - 605. [TB] Nguyen Quoc Thang and Dao Phuong Bac, Some rationality properties of observable group and related questions, Illinois J. Math. v. 49, n. 2, (2005), 431-444. [Ve] T. Venkataramana, On superrigidity and arithmeticity of lattices in semisimple groups over local fields of arbitrary characteristic, Invent. Math. Bd. 92 (1988), 255 - 306. [W] B. Weiss, Finite dimensional representations and subgroup actions on homogeneous spaces, Israel J. Math. 106 (1998), 189 - 207. 34 [...]... G(k).x is closed in Hausdorff topology in V (k) ? c) Relation between a) and b) ? Remarks Question a) (geometric case) above is a subject of Geometric Invariant Theory, question b) (relative case) is a subject of geometric invariant theory over non- algebraically closed fields Finally, the question c), if k is of arithmetic nature (say k is the ring of integers in a local or global field, or ad`ele ring of... the answer ”no” in the imperfect case In connection with Borel’s questions, in one of the early papers on geometric invariant theory over non- algebraically closed fields, D Birkes [Bi] introduced the following Properties A, B, C of representations of algebraic groups Property A (Cf [Bi]) Let ρ : G → GL(V ) be any linear representation of an affine algebraic group G, all are defined over a field k Let... theory over non- algebraically closed fields Stability To establish the relative version of Bogomolov’s theorem and also other versions of Sukhanov’s theorem, we need to use the Kempf’s-Rousseau’s Instability theory This appears to be very important in the arithmetic invariant theory In fact, from the very beginning of the modern geometric invariant theory, to study the closedness of G-orbits, one encounters... Then the following statements are equivalent a) G · v is closed in Zariski topology; b) T · v is closed in Zariski topology; c) G(k) · v is closed in Hausdorff topology; d) T (k) · v is closed in Hausdorff topology In the case of solvable groups, we find some contrasts to the nilpotent case 4.2.5 Proposition ([BaT2]) Let k be either R or Qp , G a solvable linear algebraic group defined over k, T an arbitrary... [Hum] J Humphreys, Linear Algebraic Groups, Graduate Text in Math v 48, Second ed Springer-Verlag, Berlin-Heidelberg-New York, 1984 [K] G Kemper, A characterization of linearly reductive groups by their invariants Transform Groups v 5 (2000), 8592 [Ke] G Kempf, Instability in invariant theory, Annals of Math v 108 (1978), 299 - 316 [Kr] H -P Kraft, Geometrische Methoden in der Invariantentheorie, Vieweg... are defined over a field k Due to the need of numbertheoretic applications, the local and global fields and relared rings k are in the center of research of arithmetic invariant theory For example, let an algebraic k-group G act on a k-variety V , x ∈ V (k) We are interested in the set G(k).x, which is called relative orbit of x (to distinguish with geometric orbit G.x) One of the main steps in the... classical results in (geometric) invariant theory to the case of characteristic p > 0 were discussed at length in [MFK, Appendices].) It will be more interesting to have examples with G, H connected groups A relation with the subalgebra of invariants of a Grosshans subgroup of a reductive group acting rationally upon a finitely generated commutative algebra is given in the following 2.2.16 Theorem... rank 1 Then 1) The special and canonical topologies on H1f lat (k, G) coincide 2) Any connecting map appearing in the exact sequence of cohomology in degree ≤ 1 induced from a short exact sequence of affine group schemes of finite type involving G is continuous with respect to canonical (or special) topologies As a consequence of the above results, we have the following 4.2.3 Theorem Assume that k is... of the ring of G◦ ∩ H-invariants in k[G◦ ] is equal to the field of G◦ ∩ H-invariants in k(G◦ ) If, moreover, H(k) is Zariski dense in H, then the above conditions are equivalent to the relative observability of H over k Some further extensions Recently, in a series of papers, J Alper et al (see e.g [AE]) have generalized some of the results considered above for observable subgroup schemes basing on... have those closed subgroups of G satisfying the condition (k[G] the following characterizations of epimorphic subgroups over an algebraically closed fields 2.2.10 Theorem ([BB1, Th´eor`eme 1], [Gr2, Lemma 23.7]) Let H be a closed subgroup of G, all defined over an algebraically closed field k Then the following are equivalent a) H is epimorphic, i e., (k[G]H ) = G b) k[G/H] = k c) k[G/H] is finite dimensional ... Kempf’s-Rousseau’s Instability theory This appears to be very important in the arithmetic invariant theory In fact, from the very beginning of the modern geometric invariant theory, to study the closedness... in the imperfect case In connection with Borel’s questions, in one of the early papers on geometric invariant theory over non- algebraically closed fields, D Birkes [Bi] introduced the following... an algebraically closed field However, since the very beginning of modern geometric invariant theory, as presented in [Mu], [MFK], there has been a need to consider the relative case of the theory