DSpace at VNU: PARALLEL ITERATIVE REGULARIZATION ALGORITHMS FOR LARGE OVERDETERMINED LINEAR SYSTEMS

January 6, S0219876210002313 2011 13:10 WSPC/0219-8762 196-IJCM International Journal of Computational Methods Vol 7, No (2010) 525–537 c World Scientific Publishing Company DOI: 10.1142/S0219876210002313 PARALLEL ITERATIVE REGULARIZATION ALGORITHMS FOR LARGE OVERDETERMINED LINEAR SYSTEMS Int J Comput Methods 2010.07:525-537 Downloaded from www.worldscientific.com by QUEENSLAND UNIVERSITY OF TECHNOLOGY on 10/20/14 For personal use only PHAM KY ANH∗ and VU TIEN DUNG† Department of Mathematics Vietnam National University, 334 Nguyen Trai Thanh Xuan, Hanoi, Vietnam ∗anhpk@vnu.edu.vn †tiendunga2@yahoo.com Received 22 March 2010 Accepted August 2010 In this paper, we study the performance of some parallel iterative regularization methods for solving large overdetermined systems of linear equations Keywords: Iterative regularization method; ill-posed problem; parallel computation Introduction Many scientific problems lead to the requirement of finding a vector x ∈ Rn , such that: Bx = g, (1) where B ∈ Rm×n and g ∈ Rm are given matrix and vector, respectively In the last years, many direct and iterative methods for solving linear systems (1) on vector and parallel computers have been studied intensively (see, e.g [Calvetti and Reichel (2002); Gallivan et al (1990); Saad and Vorst (2000)] and references therein) The aim of this paper is to implement some parallel iterative regularization methods proposed in Ref [Anh and Chung (2009)] for large overdetermined linear systems (1), when the number of equation is large compared to the number of unknowns, i.e., m n Large-scale linear discrete ill-posed problems arise in many practical problems, such as image restoration, computer tomography, and inverse problems in electromagnetics In what follows, we are interested in finding the minimal-norm solution of the consistent system (1) ∗ Corresponding author 525 January 6, S0219876210002313 526 2011 13:10 WSPC/0219-8762 196-IJCM P K Anh & V T Dung For a parallel computation purpose, we shall partition the given data B and g into N blocks     g1 B1  g2   B2      B =  ; g =  ; Bi ∈ Rmi ×n ; gi ∈ Rmi ,     Int J Comput Methods 2010.07:525-537 Downloaded from www.worldscientific.com by QUEENSLAND UNIVERSITY OF TECHNOLOGY on 10/20/14 For personal use only BN gN where N ≥ 2; ≤ mi ≤ m − 1, and N i=1 mi = m Clearly, x is a solution of Eq (1) if and only if it is a common solution of N subsystems Bi x = gi , i = 1, 2, , N (2) For solving a consistent system of ill-posed equations Ai (x) = 0, i = 1, 2, , N, (3) where Ai : H → H are given possibly nonlinear operators in a real Hilbert space H, two parallel methods, namely, the parallel implicit iterative regularization method (PIIRM) and the parallel explicit iterative regularization method (PEIRM), which are proposed in Ref [Anh and Chung (2009)] All the operators Ai (x) in Eq (3) are supposed to be inverse-strongly monotone (see [Liu and Nashed (1998)]), i.e., ∃ci , ∀x, y ∈ H Ai (x) − Ai (y), x − y ≥ Ai (x) − Ai (y) , ci i = 1, 2, , N (4) Clearly, each operator Ai (x) is Lipschitz continuous and monotone, but not necessarily strongly monotone, hence each Eq (3) may be ill-posed For the sake of simplicity, we may assume that all the constants ci in Eq (4) are the same, i.e., ci = c; i = 1, 2, , N Theorem 1.1 [Anh and Chung (2009), Theorem 2.1] Let αn and γn be two −αn | → sequences of positive numbers, such that αn → 0, γn → +∞, γn |αn+1 α2n ∞ αn as n → +∞ and = +∞ Suppose the nth-approximation x is found n n=1 γn (x0 is given) Then the following parallel regularization algorithm: αn Ai (xin ) + + γn xin = γn xn , i = 1, 2, , N, (5) N xn+1 = N N xin , n = 0, 1, 2, , (6) i=1 will converge to the minimal-norm solution x+ of the system (3) Since all the problems (5) are well posed and independent from each other, they can be solved stably by parallel processors Making a few iterates for approximating xin in Eq (5) and inserting approximate values of xin into Eq (6), we come to the PEIRM, whose convergence is guaranteed by the following theorem January 6, S0219876210002313 2011 13:10 WSPC/0219-8762 196-IJCM Parallel Iterative Regularization Algorithms 527 Theorem 1.2 [Anh and Chung (2009), Theorem 2.2] Suppose the sequences αn and γn satisfy all conditions in Theorem 1.1 Moreover, assume that ∀ n, N c + αn ≤q and γn → +∞ as n → +∞, there exists a number γ > 0, such that γn ≥ γ > for all n ≥ 0, hence BiT gi /γn ≤ ω12 /γ, i = 1, 2, , N ; n ≥ BiT gi 1δ ) ≤ xn − zn + ωγ2nδ for Thus, zni − xin ≤ xn − zn + 2ω γn (1 + xn + γn some positive constant ω2 From the last relation, we get xn+1 − zn+1 ≤ N N xin − zni ≤ xn − zn + i=1 ω2 δ , γn therefore, if z0 = x0 , then n−1 xn − zn ≤ ω2 δ k=0 ω2 nδ ≤ γk γ Choosing n = [δ −µ ], where µ ∈ (0, 1) is a fixed number, we have zn(δ) − x+ ≤ zn(δ) − xn(δ) + xn(δ) − x+ ≤ ωγ2 δ 1−µ + xn(δ) − x+ → as δ → Thus, we come to the following theorem Theorem 2.1 Suppose the given data are contaminated by errors, namely Bδ − B ≤ δ and gδ − g ≤ δ If the parameters αn and γn satisfy all the conditions in Theorem 1.1, then the iteration process (13), (14) with the termination index n(δ) = [δ −µ ], converges to the minimal-norm solution of (1)as the error level δ tends to zero Moreover, there holds the estimate zn(δ) − x+ ≤ ωγ2 δ 1−µ + xn(δ) − x+ , where ω2 and γ are some positive constants, n(δ) = [δ −µ ] and µ ∈ (0, 1) Now, we turn to the PEIRM (7), (8) For the sake of simplicity, we consider the method (9), where at each step only one inner iteration (7) is performed Letting January 6, S0219876210002313 530 2011 13:10 WSPC/0219-8762 196-IJCM P K Anh & V T Dung N N Int J Comput Methods 2010.07:525-537 Downloaded from www.worldscientific.com by QUEENSLAND UNIVERSITY OF TECHNOLOGY on 10/20/14 For personal use only T D = i=1 BiT Bi , Dδ = i=1 Bδi Bδi , Tn = (1 − αn βn )I − βn D and Tδn = (1 − N N T gδi , αn βn )I − βn Dδ , where βn = N γn and putting f = i=1 BiT gi , fδ = i=1 Bδi we can rewrite the iterative process (9) for solving Eq (2) in both noise-free and noisy data cases as xn+1 = Tn xn + βn f, (15) zn+1 = Tδn zn + βn fδ , (16) Moreover, if the sequences αn and γn satisfy all conditions in Theorem 1.2, then the sequence xn , defined by Eq (15) converges to the minimal-norm solution of Eq (1) For evaluating the discrepancy xn − zn , we need some more estimations T T Bδi ≤ (Bi − Bδi )T Bi + Bδi (Bi − Bδi ) ≤ First, observe that BiT Bi − Bδi 2ω1 δ, hence D − Dδ ≤ 2ω1 N δ, therefore Tδn − Tn ≤ 2ω1 N βn δ (17) Further, N { (Bδi − Bi )T gδi + BiT (gδi − gi ) } ≤ 2ω1 N βn δ βn (fδ − f ) ≤ βn (18) i=1 Finally, setting ξ = zn − xn , we have Tδn (zn − xn ) = (1 − αn βn )ξ − βn Dδ ξ = (1 − αn βn )2 ξ 2 − 2βn (1 − αn βn ) Dδ ξ, ξ + βn2 Dδ ξ Using the fact Dδ ξ, ξ ≥ DDδ ξδ and the condition find − αn βn = − Nαγnn > Thus, Tδn ξ ≤ (1 − αn βn )2 ξ − βn αn N γn 2(1 − αn βn ) − βn Dδ < N c+αn γn ≤ q < 1, we Dδ ξ (19) Since αn → and γn → +∞, without loss of generality, we can assume that γn > ω12 αn + , N n ≥ (20) N Using Eq (20) and the estimate Dδ ≤ i=1 Bδi ≤ N ω12 , one can show that 2(1 − αn βn ) − βn Dδ > 0, hence from Eq (19), it follows Tδ (zn − xn ) ≤ zn − xn Now, from Eqs (15) and (16), we have xn+1 −zn+1 ≤ Tδn −Tn xn + Tδn zn + βn fδ − f , hence using the estimates (17), (18), and (21), we find xn+1 − zn+1 ≤ 2ω1 N βn δ(1 + xn ) + xn − zn (21) xn − (22) Since the sequence xn converges to x+ , it is bounded and there exists a positive constant ω2 > 2ω1 (1 + xn ) The relation (22) implies xn+1 − zn+1 ≤ xn − ω2 zn + ωγ2nδ , hence starting with z0 = x0 , we have xn − zn ≤ ω2 δ n−1 k=0 γk ≤ γ nδ January 6, S0219876210002313 2011 13:10 WSPC/0219-8762 196-IJCM Parallel Iterative Regularization Algorithms 531 Thus, choosing n(δ) = [δ −µ ], 0