DSpace at VNU: Parallel regularized Newton method for nonlinear ill-posed equations

DSpace at VNU: Parallel regularized Newton method for nonlinear ill-posed equations tài liệu, giáo án, bài giảng , luận...

Numer Algor (2011) 58:379–398 DOI 10.1007/s11075-011-9460-y ORIGINAL PAPER Parallel regularized Newton method for nonlinear ill-posed equations Pham Ky Anh · Cao Van Chung Received: September 2010 / Accepted: 21 March 2011 / Published online: April 2011 © Springer Science+Business Media, LLC 2011 Abstract We introduce a regularized Newton method coupled with the parallel splitting-up technique for solving nonlinear ill-posed equations with smooth monotone operators We analyze the convergence of the proposed method and carry out numerical experiments for nonlinear integral equations Keywords Monotone operator · Regularized Newton method · Parallel splitting-up technique Mathematics Subject Classifications (2010) 47J06 · 47J25 · 65J15 · 65J20 · 65Y05 Introduction The Kaczmarz method for a system of linear algebraic equations was invented more than 70 years ago It has proved to be quite efficient in many applications ranging from computer tomography to digital signal processing Recently, there has been a great interest in the so-called Kaczmarz methods for solving ill-posed problems, namely, Landweber–Kaczmarz method, P K Anh (B) · C V Chung Department of Mathematics, Vietnam National University, 334 Nguyen Trai, Thanh Xuan, Hanoi, Vietnam e-mail: anhpk@vnu.edu.vn C V Chung e-mail: chungcv@vnu.edu.vn 380 Numer Algor (2011) 58:379–398 Newton–Kaczmarz method and the steepest-descent-Kaczmarz method The main idea of Kaczmarz methods is to split the initial ill-posed problem into a finite number of subproblems and to perform a cyclic iteration over the subproblems These methods not only reduce the overall computational effort but also impose less constraints on the nonlinearity of the operators Besides, experiments show that the Kaczmarz methods in some cases are performed better than standard iterative methods For a wide literature concerning Kaczmarz methods, please refer to [1–6] and references therein Clearly, Kaczmarz methods are inherently sequential algorithms When the number of equations N is large, the Kaczmarz like methods are costly on a single processor In this paper we introduce a parallel regularized Newton method, which may be regarded as a counterpart of the regularizing Newton– Kaczmarz method Our idea consists of splitting the given ill-posed problem into subproblems and performing synchronously one iteration of the regularized Newton method to each subproblem The next approximate solution is determined as a convex combination of the obtained iterations for subproblems The benifit of our approach is clear Based on parallel computation we can reduce the overall computational effort under widely used conditions on smooth monotone operators (cf [7–10]) Let us consider a nonlinear operator equation A(x) := F(x) − f = 0, (1.1) where F : H → H is a twice locally Frechet differentiable and monotone operator; f ∈ H is given and H is a real Hilbert space We assume that the set C ⊂ H of solutions of (1.1) is not empty, hence C is a convex and closed subset of H (see, e.g [7, 11]) Several problems arising in Quantum Mechanics, Wiener-type filtering theory or in physics, where dissipation of energy occurs, can be reduced to equations involving monotone operators (see [7–10]) If F is not strongly monotone or uniformly monotone, problem (1.1) in general is ill-posed In that case a process known as Lavrentiev regularization consisting of solving the singularly perturbed operator equation A(x) + αn x = 0, (αn > 0, n = 0, 1, 2, ) (1.2) is recommended (see [7, 11–13]) In [14] we proposed parallel iterative regularization methods for solving (1.1), combining the regularization and parallel splitting-up techniques In this paper we use the parallel splitting-up technique (see [15]) and the Newton method to solve (1.2) Throughout this paper we assume that for all x ∈ H, F(x) − f = N i=1 Fi (x) − N i=1 fi and put Ai (x) := Fi (x) − fi (i = 1, N) Further, we suppose that all Fi : H → H are twice locally Frechet differentiable and Numer Algor (2011) 58:379–398 381 monotone operators Then, knowing the n-th approximation zn we can determine the next approximation zn+1 as follows Ai (zn ) + αn αn + γn I (zin − zn ) = − Ai (zn ) + zn , i = 1, 2, , N, N N (1.3) zn+1 = N N zin , n = 0, 1, 2, , (1.4) i=1 where αn and γn are positive numbers Due to the nonnegativity of the derivative Ai (zn ) (see [7]), all the linear regularized equations (1.3) are wellposed Moreover, being independent from each other, they can be solved stably and synchronously by parallel processors The article is outlined as follows In Section we provide a convergence analysis of the proposed method in both exact data and noisy data cases In the final Section we verify all the assumptions required and perform numerical experiments for a model problem Convergence analysis We begin with some notions and auxiliary results Lemma 2.1 If αn is a converging to zero sequence of positive numbers, then for each n ∈ N, the regularized equation (1.2) has a unique solution x∗n and x∗n → x† := argmin x as n → +∞ Moreover, the following estimates hold x∈C i ii x∗n ≤ x† ; x∗n+1 − x∗n ≤ |αn+1 −αn | αn x† for all n The proof of this lemma can be found in [7, 11] We recall that an operator A : H → H is called c−1 —inverse-strongly monotone, if A(x) − A(y), x − y ≥ A(x) − A(y) , c ∀x, y ∈ H, where c is some positive constant (see, e.g [16]) Obviously, every inverse-strongly monotone operator is monotone and not necessarily strongly-monotone The following lemma shows that a system of inverse-strongly monotone operator equations may be reduced to a single equation involving the sum of these operators 382 Numer Algor (2011) 58:379–398 Lemma 2.2 Suppose Ai , i = 1, 2, , N are ci−1 -inverse-strongly monotone operators If the system of equations Ai (x) = 0, i = 1, 2, , N, (2.1) is consistent, then it is equivalent to the operator equation N Ai (x) = A(x) := (2.2) i=1 Proof Obviously, any common solution of (2.1) is a solution of (2.2) Conversely, let x∗ be a solution of (2.2) and y be a common solution of (2.1) Then, N N Ai (x∗ ), x∗ − y = 0= i [Ai (x∗ ) − Ai (y)], x∗ − y i N N ci−1 Ai (x∗ ) − Ai (y) Ai (x∗ ) − Ai (y), x∗ − y ≥ = i ≥ i Thus, Ai (x∗ ) = Ai (x∗ ) − Ai (y) = 0, hence x∗ is a solution of system (2.1) In what follows, we denote a closed ball centered at a ∈ H and with a radius r by B[a, r] The following result on the convergence rate of Lavrentiev regularization method is needed for our further study Lemma 2.3 Let x† = be a minimal-norm solution of (1.1) and assume Ai (x), i = 1, N, are twice continuously Frechet dif ferentiable operators in B[0, r] with a radius r > x† Moreover, let Ai (x) ≤ Li , i = 1, 2, , N for all x ∈ B[0, r] Then the following statements hold: (a) If Ai , i = 1, 2, , N, are monotone and there exists an element u ∈ H, such that N x† = A (x† )u = N Ai (x† )u Li u < and i=1 (2.3) i=1 Then the regularized solution x∗n of (1.2) converges to x† at the rate x∗n − x† = O(αn ) (2.4) (b) If Ai , i = 1, 2, , N, are ci−1 - inverse-strongly monotone and system (2.1) is consistent Moreover, assume that there exist wi , i = 1, 2, , N, such that N N Ai∗ (x† )wi x† = Li wi < and i=1 (2.5) i=1 Then the following convergence rate holds x∗n − x† = O(αn1/4 ) (2.6) Numer Algor (2011) 58:379–398 383 Proof The first estimate of Lemma 2.3 can be found in [11, 13] For the second estimate we can process as in [17] Indeed, Lemma 2.2 ensures that x† is a common solution of equations (2.1) Further, we have ∗ x − x+ n n = { x∗n 2 − x† − x† , xn − x† } N ≤ − x† , x∗n − x† = − Ai∗ (x† )wi , x∗n − x† i=1 N N Ai (x∗n ) − Ai (x† ) − Ai (x† )(x∗n − x† ), wi − = i=1 Ai (x∗n ), wi i=1 N = i=1 t Ai (x† + st(x∗n − x† ))(x∗n − x† )2 , wi dsdt N Ai (x∗n ), wi − i=1 Thus, we obtain the estimate ∗ x − x† n ≤ N N Li wi x∗n −x † Ai (x∗n ) wi + i=1 (2.7) i=1 Using the inverse-strong monotonicity of Ai (x) we get Ai (x∗n ) = Ai (x∗n ) − Ai (x† ) ≤ ci Ai (x∗n ) − Ai (x† ), x∗n − x† ⎧ ⎫ ⎨N ⎬ Aj(x∗n )− Aj(x† ), x∗n −x† − Aj(x∗n )− Aj(x† ), x∗n −x† = ci ⎩ ⎭ j=1 j=i ≤ − ci αn x∗n , x∗n − x† ≤ 2αn ci x† √ From the last inequality we find Ai (x∗n ) ≤ 2αn ci x† Taking into account N √ Li wi )−1 2αn x† the last estimate, from (2.7) we get x∗n − x† ≤ 2(1 − N √ ci wi , which implies (2.6) i=1 i=1 Before stating and proving convergence theorems we observe that Ai (x∗n ) + → Ai (x† ) as n → +∞, hence we can assume Ai (x∗n ) + αNn x∗n ≤ CA for all n = 0, 1, and i = 1, 2, , N Furthermore, we suppose that the operators Ai , i = 1, 2, , N, are twice continuously √ Frechet differentiable and Ai (x) ≤ ϕ for all x ∈ B[0, r], where r = M D∗ , and M > 1, D∗ ≥ max CA2 , x† are appropriately chosen constants Now we have the following convergence result αn ∗ x N n 384 Numer Algor (2011) 58:379–398 Theorem 2.1 Let {αn }, {γn } be two sequences of positive numbers, αn 0, γn +∞ as n → +∞, such that the following conditions hold for all n ∈ N and for some constants c1 ∈ (0, 1), c2 > γn αn4 ≥ γ0 α04 ; c1 γ03 γn3 (αn − αn+1 )2 ≤ αn5 α03 and γn (γn+1 − γn ) ≤ c2 γ02 (2.8) Moreover, the initial values α0 , γ0 and M satisfy the following relations lα0 D∗ ≤ √ < (M − 1)2 D∗ ; γ0 N ≤ 4γ0 α02 ; √ 2γ0 (1− c1 ) (2+c2 )(2Nγ0 +α0 ) where l := − √ 4Nc1 γ03 +(4c1 +2 c1 +c2 )α0 γ02 +2α0 2γ0 α02 l > 0, (2.9) ϕ2 Then starting from z0 = 0, the approximations zn of the parallel regularized Newton method (1.3) and (1.4) will converge to the minimal norm solution x† of (1.1) For the sake of clarity, we divide the proof of Theorem 2.1 into several steps Firstly, we establish a recurrence estimate for the distance between zn defined by (1.3), (1.4) and the regularized solution x∗n of (1.2) Lemma 2.4 The distances ||en || := ||zn − x∗n || satisfy the following inequality en+1 ≤ 1+ n N(1 + n ) √ γn + where n := N Ai (ξni ) en + N en i=1 N(1 + n )(αn+1 − αn )2 † x n αn 2 + NCA2 γn2 , (2.10) αn 2Nγn Proof Inserting x = zn ; h = −en := x∗n − zn ; y = ein := zin − x∗n in the Taylor formula A(x + h), y = A(x) + A (x)h + A (x + θ h)h2 ,y , 2! where x, y, h ∈ H and θ := θ(y) ∈ (0, 1), we get Ai (x∗n ) + αn ∗ i x , z − x∗n = Ai (zn ), zin − x∗n + Ai (zn )(x∗n − zn ), zin − x∗n N n n A (ξ i )(x∗ − zn )2 , zin − x∗n + i n n αn ∗ i (2.11) + x , z − x∗n N n n Numer Algor (2011) 58:379–398 385 Here ξni = θni (x∗n − zn ) + zn and θni ∈ (0; 1) depends on Ai , zin , x∗n On the other hand, from (1.2) it follows N Ai (x∗n ) + i=1 N αn ∗ i x , z − x∗n = N n n αn ∗ i x , z − zn N n n Ai (x∗n ) + i=1 N Ai (x∗n ) + + i=1 N αn ∗ i x , z − zn N n n Ai (x∗n ) + = αn ∗ x , zn − x∗n N n i=1 (2.12) Combining (2.11) with (2.12) we find N N Ai (zn ), zin − x∗n + i=1 + Ai (zn )(x∗n − zn ), zin − x∗n + i=1 N N Ai (ξni )(x∗n − zn )2 , zin − x∗n − i=1 Ai (x∗n ) + i=1 αn N N x∗n , zin − x∗n i=1 αn ∗ i x , z − zn = N n n (2.13) Multiplying both sides of (1.3) by zin − x∗n and summing up for i from to N, we have N N Ai (zn ), zin − x∗n + i=1 + Ai (zn )(zin − zn ), zin − x∗n i=1 N αn + γn N zin − zn , zin − x∗n + i=1 αn N N zn , zin − x∗n = (2.14) i=1 Subtracting (2.13) and (2.14), after a short computation we get γn N N Ai (zn )(zin − x∗n ), zin − x∗n + i=1 + i=1 αn Nγn = zin − zn , zin − x∗n 2γn − γn N zin − zn , zin − x∗n + zn − x∗n , zin − x∗n i=1 N Ai (ξni )(x∗n − zn )2 , zin − x∗n i=1 N Ai (x∗n ) + i=1 αn ∗ i x , z − zn N n n (2.15) 386 Numer Algor (2011) 58:379–398 Since Ai is monotone, Ai (x) is a linear positive operator for all x Therefore, the first term in the left-hand side of (2.15) is nonnegative Hence, from (2.15) we get N i=1 ≤ N αn Nγn zin − zn , zin − x∗n + zin − x∗n i=1 N 2γn N γn Ai (ξni )(x∗n − zn )2 , zin − x∗n − i=1 Ai (x∗n ) + i=1 αn ∗ i x , z − zn N n n Using the above notations we can rewrite the last inequality as follows N i=1 ≤ en , ein N i=1 ≤ ≤ γn = + ein γn − en + ein − en 1+ i=1 N i=1 N Ai (ξni ) + en i=1 N 2αn Nγn ein Ai (x∗n ) + αn ∗ i x , e − en N n n Ai (x∗n ) + αn ∗ x N n N i=1 N ein γn3 i=1 i=1 γn2 Ai (ξni )(−en )2 , ein + + i=1 γn N + ein − en i=1 NCA2 + γn2 N ein − en i=1 Ai (ξni )(−en )2 , ein ≤ √1 γn Ai (ξni ) γn √ 4αn γn − N √ 2Nγn γn N ein i=1 ≤ √ γn By the assumptions N ≤ 4α02 γ0 √ 4αn2 γn −N αn √ ≤ all n, therefore Nγ 2Nγn γn n 1+ ein − en , the last inequality is N γn Ai (ξni )(−en )2 , ein − Here we used the Young’s inequality en + √1 ein Thus, i=1 + αn ∗ i x , e − en N n n Ai (x∗n ) + i=1 N ≤ √ γn N N en i=1 γn i=1 Ai (ξni )(−en )2 , ein − N − 2 i=1 Since − equivalent to ein ein N 2γn ein N αn Nγn ein − en , ein + αn Nγn N ein i=1 and N Ai (ξni ) en + N en + i=1 αn4 γn NCA2 γn2 √ ≥ α04 γ0 , we have 2αn2 γn − N ≥ for Hence, from the last inequality, we find ≤ √ γn N Ai (ξni ) i=1 en + N en + NCA2 (2.16) γn2 Numer Algor (2011) 58:379–398 387 From (1.4) we get en+1 = zn+1 − x∗n+1 ≤ zn+1 − x∗n + x∗n+1 − x∗n Using the estimate x∗n+1 − x∗n ≤ |αn+1αn−αn | x† in Lemma 1.1, we have en+1 = zn+1 − x∗n+1 ≤ zn+1 − x∗n + ≤ N N ein + i=1 |αn+1 − αn | † x αn N ≤√ N 1/2 ein |αn+1 − αn | † x αn + i=1 |αn+1 − αn | † x αn Applying the inequality (a + b )2 ≤ (1 + n )(a2 + 1n b ) to the last relation with αn n := 2Nγn > and using (2.16), we come to the desired estimate (2.10) Now for completing the proof of Theorem 2.1, we use estimate (2.10) to show that en −→ 0, hence zn − x† = en + x∗n − x† −→ as n −→ ∞ Proof of Theorem 2.1 Setting vk := ek /λk and λk := (2.10) as vn+1 ≤ 1+ n N(1 + n ) λ2n vn2 √ 2λn+1 γn N Ai (ξni ) i=1 + √αk , γk we can rewrite Nλn λn+1 NCA2 N(1 + n )(αn+1 − αn )2 † (2.17) + x λn+1 γn2 n αn λn+1 √ By our assumptions, we have v0 ≤ √ l and ξ0i ≤ x† ≤ M D∗ Assume by i induction that ≤ l and √ ξn ≤ M D∗ for some n ≥ 0, we will show that i vn+1 ≤ l and ξn+1 < M D∗ From (2.17) and the estimates Ai (ξni ) ≤ ϕ, ≤ l, we get + vn+1 − l ≤ 1+ n (1 + n ) λ2n ϕ 2 √ l + 2λn+1 γn + λn 1+2 n − l λn+1 1+ n CA2 (1 + n )(αn+1 − αn )2 † + x 2 λn+1 γn n αn λn+1 Taking into account the relation max CA2 , x† inequality we find vn+1 − l ≤ ≤ D∗ ≤ lα0 √ , γ0 from the last 1+ n λ2n ϕ 2 lα0 n λn+1 − (λn − λn+1 ) l + √ √ l − (1 + n )λn+1 γn 1+ n γn γ0 + lα0 (1 + n )(αn+1 − αn )2 √ γ0 n αn2 388 Numer Algor (2011) 58:379–398 ≤ (1 + √ ϕ 2l αn+1 γn − √ · αn γn+1 2N + αn /γn √ √ γn (αn γn+1 − αn+1 γn ) α0 + +√ ·√ √ αn γn+1 γ0 γn αn2 n )αn l 3/2 (1 + n )λn+1 γn 3/2 + 2α0 (Nγn + αn )γn (αn − αn+1 )2 √ γ0 αn5 (2.18) Besides, a straightforward calculation yields √ √ γn (αn γn+1 − αn+1 γn ) √ αn2 γn+1 √ √ √ √ γn (αn − αn+1 )( γn+1 + γn ) + (αn+1 γn+1 − αn γn ) = √ αn2 γn+1 √ √ 2γn (αn − αn+1 ) γn (αn+1 − αn ) γn ( γn+1 − γn ) ≤ + + √ αn2 αn2 αn γn+1 ≤ γn (αn − αn+1 ) γn (γn+1 − γn ) γn (αn − αn+1 ) (γn+1 − γn ) + = + αn 2γn αn αn2 2αn From the assumptions αn+1 αn c1 αn3 γ03 α03 γn3 ≥1− and γn3 (αn −αn+1 )2 αn5 γn+1 γn ≤ ≤1+ c1 γ03 α03 c2 γ02 2γn2 and γn (γn+1 − γn ) ≤ c2 γ02 , it follows Hence, by (2.18) and the last relation, we get vn+1 − l ≤ (1 + n )αn l 3/2 (1 + n )λn+1 γn ϕ 2l γn (αn − αn+1 ) (γn+1 − γn ) + + αn2 2αn √ αn+1 γn α0 − √ · +√ ·√ αn γn+1 2N + αn /γn γ0 γn αn2 3/2 + ≤ (1 + (1 + 2 n )αn l 3/2 n )λn+1 γn 2α0 (Nγn + αn )γn (αn − αn+1 )2 √ γ0 αn5 ϕ 2l + c2 γ02 c1 γ03 αn α0 + +√ √ 2γn αn γ0 γn αn2 γn α0 2Nc1 γ05 2c1 αn γ05 αn + + − 2N + 2√ γn α0 γn α0 γn × 1+ c2 γ02 2γn2 −1 1− c1 αn3 γ03 α03 γn3 −1 Numer Algor (2011) 58:379–398 Since αn vn+1 − l ≤ 0, γn 389 +∞ as n → +∞ and γn αn4 ≥ γ0 α04 for all n, we have (1 + n )αn l 3/2 (1 + n )λn+1 γn √ ϕ 2l 4Nc1 γ03 + (4c1 + c1 + c2 )α0 γ02 + 2α0 + 2γ0 α02 √ 2γ0 (1 − c1 ) − (2 + c2 )(2Nγ0 + α0 ) = (2.19) On the other hand, we have ξni = θni (x∗n − zn ) + zn = x∗n + (1 − θni )(zn − x∗n ) From the assumption √αγnn l ≤ (M − 1)2 D∗ we find i ξn+1 = x∗n+1 + (1 − θni ) zn+1 − x∗n+1 ≤ x† + (1 − θni ) en+1 √ αn+1 √ ≤ x† + (1 − θni ) √ l ≤ D∗ + (M − 1) D∗ = M D∗ γ n+1 √ Thus, by induction, we have ≤ l and ξni ≤ M D∗ for all n Therefore, zn − x∗n ≤ l √αγnn → as n → ∞ Finally, Lemma 2.1 ensure that x∗n → x† , hence zn → x† , as n → ∞ Remark 2.1 The sequences αn := α0 (1 + n)− p , < p ≤ 18 and γn := γ0 × and c2 ≥ 12 (1 + n)1/2 satisfy all the conditions (2.8) for any constants c1 ≥ 64 and c2 = 12 ; γ0 := max 5; If we choose c1 = 64 M ≥ 3, then condition (2.9) holds 12ϕ D∗ ; α0 := 5Nγ0 and Remark 2.2 It can be shown that the above theorem remains valid if the boundedness of the second derivative F (x) is replaced with the Lipschitz continuity of F (x) in B[0, r] On the other hand, as observed in [5], even the Lipschitz continuity of F (x) does not imply the frequently used in literature “tangential cone condition”, so our hypothesis is not as very strong as it seems to be Now we assume that instead of the exact data Fi and fi we only have noisy ones, denoted by {Fn,i } and { fn,i } Suppose that the operators Fn,i : H → H possess the same properties as Fi , i.e., they are twice continuously Frechet differentiable monotone operators Moreover, let fn,i − fi ≤ δn (δn > 0, Fn,i (x) − Fi (x) ≤ hn g( x ) i = 1, 2, , N, (hn > 0, n ≥ 0), i = 1, 2, , N, n ≥ 0), (2.20) where g(t) is a continuous positive and nondecreasing function on (0, +∞) Let Fn (x) = N i=1 Fn,i (x) and fn = N i=1 † fn,i and denote An,i (x) := Fn,i (x) − fn,i , (i = 1, N) Since An,i (x∗n ) − Ai (x ) ≤ hn g( x† ) + δn + Ai (x∗n ) − Ai (x† ) , in the case hn , δn , αn → as n → ∞, we can assume that An,i (x∗n ) + αNn x∗n ≤ CA 390 Numer Algor (2011) 58:379–398 We √ suppose as before that there exists M > such that for all x ∈ B[0, M D∗ ], the second order derivatives An,i (x) ≤ ϕ, where the constant D∗ satisfies the inequality D∗ ≥ max CA2 , x† , (δ0 + h0 g( x† ))2 Now we consider the parallel regularized Newton method in the noisy data case An,i (zn ) + ( αn αn + γn )I (zin − zn ) = − An,i (zn ) + zn , i = 1, N, N N zn+1 = N (2.21) N zin , n = 0, 1, (2.22) i=1 Theorem 2.2 Let αn , γn be two sequences of positive numbers and αn 0, γn +∞ as n → +∞ Assume that (2.8) are satisf ied, and the following additional condition holds hn g( x† ) + δn ≤ † h0 g( x ) + δ0 γn (2.23) Besides, suppose all the conditions (2.9) are satisf ied with respect to √ √ 4Nc1 γ03 + (4c1 + c1 + c2 )α0 γ02 + 6α0 2γ0 (1 − c1 ) l := − (2 + c2 )(2Nγ0 + α0 ) ϕ2 2γ0 α02 Let z0 = 0, then the nth iteration zn of (2.21) and (2.22) will converge to the solution x† of (1.1) Proof From (1.2) and the properties of An,i , we have N An,i (x∗n ) + i=1 αn ∗ i x , z − x∗n = N n n N An,i (x∗n ) + i=1 αn ∗ i x , z − zn N n n N An,i (x∗n ) + + i=1 N An,i (x∗n ) + = i=1 αn ∗ x , zn − x∗n N n αn ∗ i x , z − zn N n n N An,i (x∗n ) − A(x∗n ), zn − x∗n + i=1 + A(x∗n ) + αn ∗ x , zn − x∗n N n N An,i (x∗n ) + = i=1 αn ∗ i x , z − zn + (Fn (x∗n ) N n n − F(x∗n )) − ( fn − f ), zn − x∗n Numer Algor (2011) 58:379–398 391 Similarly, as in the proof of Theorem 2.1, using the Taylor formula for An,i (zn ) at x∗n , and multiplying both sides of (2.21) by zin − x∗n , we find γn N An,i (zn )(zin − x∗n ), zin − x∗n i=1 N + zin zn , zin − − i=1 = zin − x∗n i=1 N 2γn − N αn + Nγn x∗n An,i (ξni )(x∗n − zn )2 , zin − x∗n − i=1 N γn An,i (x∗n ) + i=1 αn ∗ i x , z − zn N n n (Fn (x∗n ) − F(x∗n )) − ( fn − f ), zn − x∗n γn (2.24) Putting ein = zin − x∗n and en = zn − x∗n , and using the monotonicity of An,i , from (2.24) we get N ein − en , ein + i=1 αn Nγn N ein ≤ i=1 2γn − − N An,i (ξni )(−en )2 , ein i=1 N γn An,i (x∗n ) + i=1 αn ∗ i x , e − en N n n (Fn (x∗n ) − F(x∗n )) − ( fn − f ), en γn Taking into account (2.20) and the relation ein − en , ein = ein ein − en , we can rewrite the last inequality as N N ein − i=1 ≤ N en + i=1 γn ein − en + i=1 N An,i (ξni )(−en )2 , ein − i=1 γn 2αn Nγn − en ein − en 2 N ein i=1 N An,i (x∗n ) + i=1 αn ∗ i x , e − en N n n − (Fn (x∗n ) − F(x∗n )) − ( fn − f ), en γn ≤ √ γn N An,i (ξni ) en i=1 + N hn g( x† ) + δn + + γn3 en γn2 N ein i=1 + NCA2 + γn2 N i=1 + 392 Numer Algor (2011) 58:379–398 αn Nγn Using (2.8) and (2.9) we get find αn 1+ Nγn N i=1 4αn γn −N √ 2Nγn γn < ≤ √ γn ein 2√ Thus, from the last inequality, we N An,i (ξni ) en+1 ≤ (1 + ≤ n) N + n b ), where N ein + i=1 1+ n √ N(1 + n ) γn i=1 αn 2Nγn ein + N hn g( x† ) + δn en 1+ n λ2n vn2 ≤ √ N(1 + n ) 2λn+1 γn NCA2 λn+1 γn2 √αk , γk |αn+1 −αn | αn x† + N 1+ γn2 en NCA2 γn2 + N(1 + n )(αn+1 − αn )2 † x n αn (2.26) (2.26) becomes N An,i (ξni ) + i=1 + + > 0, from (2.25), we get i=1 + en (2.25) N + := n An,i (ξni ) + γn2 1/2 N √1 N (αn+1 − αn )2 † x n αn With the notations vk := ek /λk , λk := vn+1 + N 1+ i=1 Taking into account the inequalities en+1 ≤ n )(a en NCA2 + N hn g( x† ) + δn γn2 + and (a + b )2 ≤ (1 + Nλn 1 + λn+1 γn N hn g( x† ) + δn λn+1 N(1 + n )(αn+1 − αn )2 † x n αn λn+1 2 (2.27) √ By the assumptions (2.8), we have v0 ≤ l and ξ0i ≤ x† < M D∗ Let √ assume ≤ l and ξni ≤ M D∗ for some n ≥ 0, then from (2.27) and the estimates An,i (ξni ) ≤ ϕ, one gets vn+1 − l ≤ 1+ n λ2n ϕ 2 √ l + (1 + n ) 2λn+1 γn CA2 λn λn 1+2 n + − l + λn+1 λn+1 γn2 1+ n λn+1 γn2 hn g( x† ) + δn + λn+1 + (1 + n )(αn+1 − αn )2 † x n αn λn+1 Numer Algor (2011) 58:379–398 393 Taking into account relations hn g( x† ) + δn ≤ h0 g( xγn )+δ0 and D∗ ≤ √lαγ00 , and acting similarly as in Theorem 2.1 we come to √ (1 + n )αn2 l ϕ 2l αn+1 γn 1 vn+1 − l ≤ − · + √ 3/2 αn γn+1 2N + αn /γn γn αn (1 + n )λn+1 γn √ √ γn (αn γn+1 − αn+1 γn ) α0 + +√ ·√ √ αn2 γn+1 γ0 γn αn2 † 3/2 + 2α0 (Nγn + αn )γn (αn − αn+1 )2 √ γ0 αn5 (2.28) Performing similar computations as in the proof of Theorem 2.1, and using the relations αn+1 αn vn+1 − l ≤ c1 αn3 γ03 α03 γn3 ≥1− (1 + γn+1 γn and n )αn l 3/2 (1 + n )λn+1 γn ≤1+ c2 γ02 , 2γn2 we obtain ϕ 2l γn (αn − αn+1 ) (γn+1 − γn ) + + + αn2 2αn γn αn √ αn+1 γn α0 − √ · +√ ·√ αn γn+1 2N + αn /γn γ0 γn αn2 3/2 + ≤ (1 + (1 + 2 n )αn l 2α0 (Nγn + αn )γn (αn − αn+1 )2 √ γ0 αn5 ϕ 2l + 3/2 n )λn+1 γn c1 γ03 αn c2 γ02 2α0 + +√ √ + 2γn αn γ0 γn αn2 γn αn γn α0 2Nc1 γ05 2c1 αn γ05 αn + + − 2N + 2√ γn α0 γn α0 γn × 1+ c2 γ02 2γn2 −1 1− c1 αn3 γ03 α03 γn3 −1 Therefore, by our assumptions, we have vn+1 − l ≤ (1 + n )αn l 3/2 (1 + n )λn+1 γn = √ ϕ 2l 2γ0 (1 − c1 ) − (2 + c2 )(2Nγ0 + α0 ) √ 4Nc1 γ03 + (4c1 + c1 + c2 )α0 γ02 + 6α0 + 2γ0 α02 (2.29) √ √ α θi ) √4 γn+1 l n+1 i On the other hand, from (2.9) we get ξn+1 ≤ x† + (1 − < √ √ i M D∗ Hence, by induction, we have ≤ l and ξn < M D∗ for all n Therefore, zn − x∗n ≤ l √αγnn → as n → ∞, hence zn → x† as n → ∞ 394 Numer Algor (2011) 58:379–398 Remark 2.3 The sequences αn := α0 (1 + n)− p , < p ≤ 18 and γn := γ0 × , c2 = 12 , γ0 := max 5; (10ϕ D∗ )2 ; (1 + n)1/2 as well as the constants c1 = 64 α0 := 7.5Nγ0 and M ≥ 3, satisfy all the conditions of Theorem 2.2 Theorem 2.1 (Theorem 2.2, respectively) together with Lemma 2.3 lead to the following result Corollary 2.1 Let all the conditions of Theorem 2.1 (Theorem 2.2, respectively) be satisf ied i If condition (2.3) holds, then zn − x† = O(αn ) ii If Ai (x) (i = 1, N) are ci−1 -inverse strongly monotone, system (2.1) is consistent, and condition (2.5) holds, then zn − x† = O(αn1/4 ) Numerical experiments In this section we illustrate the proposed method by considering some nonlinear integral equations (e.g [7–10]) For the sake of simplicity, we restrict ourselves to the case, when the initial operator is decomposed into a sum of two smooth monotone operators acting on the real Hilbert space H = L2 [0, 1], i.e., A(x) = A1 (x) + A2 (x) = 0, (3.1) Starting from z0 = and knowing the n-th approximation, we compute the corrections h1n , h2n and the next approximation as An, j(zn ) + ( αn αn + γn )I hnj = − An, j(zn ) + zn , 2 zn+1 = zn + h1n + h2n , j = 1, n = 0, 1, 2, , 3.1 An experiment with twice differentiable monotone operators Let us consider (3.1) with A j := I − F j, and [F j(u)](t) := K j(t, s) f j(u(s))ds + g j(t), j = 1, 2, (3.2) x) where K1 (t, s) = ts3 , K2 (t, s) = 13 + ts + t+s , f1 (x) = 9e−x , f2 (x) = 6(sin√x+cos 97 and g j(t) are later chosen functions A direct computation shows that F j(u) Numer Algor (2011) 58:379–398 395 Table Experiment with exact data for 10,000 early iterations nmax RT OL (%) RAT (×10−4 ) 1,000 0.08205 3.89151 2,500 0.07239 3.85040 5,000 0.06639 3.85027 7,500 0.06317 3.85026 10,000 0.06088 3.85027 are nonexpansive, hence the operators Aj := I − Fj are monotone Further, Aj are continuously differentiable, and [A1 (u)h](t) = h(t) + tsu(s)exp(−u2 (s))h(s)ds; (3.3) [A2 (u)h](t) = h(t) − √ 97 1 t+s + ts + cos u(s) − sin u(s) h(s)ds (3.4) Moreover we have [Aj (u) − Aj (v)]h ≤ u − v h , ( j = 1, 2) According to Remark 2.2, instead of an upper bound ϕ of A j , one can set ϕ = All the integrals in (3.1), (3.3) and (3.4) are computed using the trapezoidal formula with the stepsize τ = 10−3 In the following experiments, let the exact solution x∗ (t) = χ, where χ > is a later chosen constant, and g j(t) := χ − Kj(t, s) f j(χ)ds, j = 1, All computations were carried out on an IBM cluster 1350 with eight computing nodes with total 51.2 GFlops Each node contains two Intel Xeon dual core 3.2 GHz, GBRam We use the following notations: nmax T OL RT OL = T OL/ x∗ RAT = T OL/αn – – – Total number of iterations Tolerance zn − x∗ Relative tolerance (in percent) Ratio of tolerance to αn In the first experiment, we use χ = and choose αn , γn as in Remark 2.1 Table shows the results for 10,000 early steps, when αn are still large From Table 2, we see that when n is sufficiently large (nmax = O(108 )), i.e., αn is sufficiently small, the tolerance xn − x∗ is as small as O(αn ) In the next experiment, we consider the noisy case with χ = 0.5 and αn , γn (n > 0) are chosen as in Remark 2.3 Let Fn,i (x) = Fi (x) + χρ2γn n(t) x and fn,i = fi + χρn (t) , where ρn (t) 2γn := nt and n ∈ [0, 1] are normally distributed Table Experiment with exact data for small α αn RT OL (%) RAT (×10−4 ) 0.06517 0.0025092 3.85027 0.05750 0.0022142 3.85027 0.05000 0.0019251 3.85025 0.04955 0.0019078 3.85027 0.04500 0.0017326 3.85026 396 Numer Algor (2011) 58:379–398 Table Experiment with noisy data for first 10,000 iterations nmax T OL (×10−3 ) RT OL (%) RAT (×10−4 ) 500 1,000 2,500 5,000 10,000 1.31759 1.05234 0.92303 0.84637 0.77619 0.26352 0.21047 0.18461 0.16927 0.15524 5.73019 4.99178 4.90947 4.90949 4.90947 Table Experiment with inexact data for small α αn T OL (×10−5 ) RT OL (%) RAT (×10−4 ) 0.0750 0.0675 0.0625 0.0575 0.0550 3.682103 3.313886 3.068419 2.822951 2.700209 0.0073642 0.0066278 0.0061368 0.0056459 0.0054004 4.90947 4.90946 4.90947 4.90948 4.90947 Table Experiment with exact data for 1,000 early iterations nmax RT OL (%) RAT (×10−2 ) 100 0.5713 1.9108 250 0.4920 1.9097 500 0.4381 1.9102 750 0.4142 1.9008 1,000 0.3871 1.8990 0.04981 0.09468 1.90083 0.03749 0.07129 1.90051 0.03512 0.06679 1.90076 Table Results with exact data for small α αn RT OL (%) RAT (×10−2 ) 0.08891 0.16801 1.90011 Table Experiment with inexact data for first 10,000 steps 0.06682 0.12542 1.98996 nmax T OL (×10−3 ) RT OL (%) RAT (×10−2 ) 500 1,000 2,500 5,000 10,000 7.759 4.431 3.253 2.966 2.734 1.5518 0.8862 0.6506 0.5532 0.5468 3.6801 2.1305 1.7310 1.7309 1.7310 Numer Algor (2011) 58:379–398 • 397 random numbers with zero mean Table shows the results for first 10,000 iterations Table shows us the behaviour of tolerances with respect to sufficiently small αn (nmax = O(108 )) 3.2 An experiment with differentiable monotone operators In the second example, we consider (3.1) with [A j(x)](t) = [F j(x)](t) − f j(t), j = 1, and e− j|t−s| x(s)ds + [arctan( jx(t))]3 , [F j(x)](t) := j = 1, (3.5) Originated from the Wiener-type filtering theory, nonlinear equations involving operators F j(x) have been intensively studied by Hoang and Ramm (see [8–10]) For the monotonicity, the smoothness of A j and the ill-posedness of (3.1) please refer to [8–10] Furthermore, we have e− j|t−s| h(s)ds + [F j(x)h](t) := j[arctan( jx(t))]2 h(t), + ( jx(t))2 j = 1, (3.6) Let the exact solution x∗ (t) = χ as in the previous subsection, hence, x∗ = χ and e− j|t−s| ds + [arctan( jχ)]3 , f j(t) = χ j = 1, All the integrals in (3.5) and (3.6) are computed using the trapezoidal formula with the stepsize τ = 10−3 Note that in this example, F j(x) is not Lipschitz continuous and we choose αn , γn , (n > 0) as in Remark 2.1, while γ0 , α0 are chose from experiments – – – In the first experiment, we set χ = and α0 := 0.5, γ0 := Table shows the results for 1,000 early steps, when αn are still large Table shows that when n is sufficiently large, for instance, nmax = (106 ), the tolerance xn − x∗ is as small as O(αn ) Finally, we consider the noisy case with χ = 0.5 and αn , γn (n > 0) are chosen as in Remark 2.3, but α0 = 0.5 and γ0 = 4, as in the above experiment The noise data are chosen as in the last experiment in Section 3.1 The results for first 10,000 iterations are shown in Table Conclusion Kaczmarz methods, being sequential algorithms for solving systems of linear and nonlinear equations, have many successful applications to real life 398 Numer Algor (2011) 58:379–398 problems In this paper we develop a quite different idea, which may be regarded as a counterpart of the regularizing Newton–Kaczmarz method Instead of performing a cyclic iteration over subproblems, we perform synchronously one iteration of the regularized Newton method to each subproblem and take their convex combination as a next approximate solution The convergence analysis in both free-noise and noisy data cases has been studied and some numerical experiments have been performed Acknowledgements The authors would like to express their special thanks to the referees, whose careful reading and many constructive comments led to a considerable improvement of the paper This work was supported by the Vietnam National Foundation for Science and Technology Development References Burger, M., Kaltenbacher, B.: Regularizing Newton–Kaczmarz methods for nonlinear illposed problems SIAM Numer Anal 44, 153–182 (2006) De Cezaro, A., Haltmeier, M., Leitão, A., Scherzer, O.: On steepest-descent-Kaczmarz method for regularizing systems of nonlinear ill-posed equations Appl Math Comput 202, 596–607 (2008) Haltmeier, M., Leitão, A., Scherzer, O.: Kaczmarz methods for regularizing nonlinear ill-posed equations, I Convergence analysis Inverse Probl Imaging 1(2), 289–298 (2007) Haltmeier, M., Kowar, R., Leitão, A., Scherzer, O.: Kaczmarz methods for regularizing nonlinear ill-posed equations, II Applications Inverse Probl Imaging 1(3), 507–523 (2007) Kaltenbacher, B., Neubauer, A., Scherzer, O.: Iterative Regularization Methods for Nonlinear Ill-Posed Problems Walter de Gruyter, Berlin (2008) Kowar, R., Scherzer, O.: Convergence analysis of a Landweber–Kaczmarz method for solving nonlinear ill-posed problems In: Ill-Posed and Inverse Problems (Book Series), vol 23, pp 69–90 (2002) Alber, Y., Ryazantseva, I.: Nonlinear Ill-Posed Problems of Monotone Type Springer, New York (2006) Hoang, N.S., Ramm, A.G.: An iterative scheme for solving nonlinear equations with monotone operators BIT Numer Math 48(4), 725–741 (2008) Hoang, N.S., Ramm, A.G.: Dynamical systems gradient methods for solving nonlinear equations with monotone operators Acta Appl Math 106(3) (2009) 473–499 10 Hoang, N.S., Ramm, A.G.: Dynamical systems method for solving nonlinear equations with monotone operators Math Comput 79(269) (2010) 239–258 11 Bakusinskii, A.B., Goncharskii, A.V.: Ill-Posed Problems Numerical Methods and Applications Moscow Univ Press In Russian (1989) 12 Janno, J.: Lavrent’ev regularization of ill-posed problems containing nonlinear near-tomonotone operators with application to autoconvolution equation Inverse Probl 16, 133–148 (2000) 13 Tautenhahn, U.: On the method of Lavrentiev regularization for nonlinear ill-posed problems Inverse Probl 18, 191–207 (2002) 14 Anh, P.K., Chung, C.V.: Parallel iterative regularization methods for solving systems of illposed equations Appl Math Comput 212, 542–550 (2009) 15 Lü, T., Neittaanmäki, P., Tai, X.-C.: A parallel splitting up method for partial differential equations and its application to Navier–Stokes equations RAIRO Math Model Numer Anal 26(6), 673–708 (1992) 16 Liu, F., Nashed, M.Z.: Regularization of nonlinear ill-posed variational inequalities and convergence rates Set-Valued Anal 6, 313–344 (1998) 17 Hein, T.: Convergence rates for multi-parameter regularization in Banach spaces Int J Pure Appl Math 43(4), 593–614 (2008) ... steepest-descent-Kaczmarz method for regularizing systems of nonlinear ill-posed equations Appl Math Comput 202, 596–607 (2008) Haltmeier, M., Leitão, A., Scherzer, O.: Kaczmarz methods for regularizing nonlinear ill-posed. .. iterative scheme for solving nonlinear equations with monotone operators BIT Numer Math 48(4), 725–741 (2008) Hoang, N.S., Ramm, A.G.: Dynamical systems gradient methods for solving nonlinear equations. .. nonlinear equations with monotone operators Acta Appl Math 106(3) (2009) 473–499 10 Hoang, N.S., Ramm, A.G.: Dynamical systems method for solving nonlinear equations with monotone operators Math Comput