CuuDuongThanCong.com Grundlehren der mathematischen Wissenschaften A Series of Comprehensive Studies in Mathematics Series editors M Berger P de la Harpe N.J Hitchin A Kupiainen G Lebeau F.-H Lin S Mori B.C Ngô M Ratner D Serre N.J.A Sloane A.M Vershik M Waldschmidt Editor-in-Chief A Chenciner J Coates CuuDuongThanCong.com S.R.S Varadhan 349 For further volumes: www.springer.com/series/138 CuuDuongThanCong.com Peter Bürgisser r Felipe Cucker Condition The Geometry of Numerical Algorithms CuuDuongThanCong.com Peter Bürgisser Institut für Mathematik Technische Universität Berlin Berlin, Germany Felipe Cucker Department of Mathematics City University of Hong Kong Hong Kong, Hong Kong SAR ISSN 0072-7830 Grundlehren der mathematischen Wissenschaften ISBN 978-3-642-38895-8 ISBN 978-3-642-38896-5 (eBook) DOI 10.1007/978-3-642-38896-5 Springer Heidelberg New York Dordrecht London Library of Congress Control Number: 2013946090 Mathematics Subject Classification (2010): 15A12, 52A22, 60D05, 65-02, 65F22, 65F35, 65G50, 65H04, 65H10, 65H20, 90-02, 90C05, 90C31, 90C51, 90C60, 68Q25, 68W40, 68Q87 © Springer-Verlag Berlin Heidelberg 2013 This work is subject to copyright All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer Permissions for use may be obtained through RightsLink at the Copyright Clearance Center Violations are liable to prosecution under the respective Copyright Law The use of general descriptive names, registered names, trademarks, service marks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made The publisher makes no warranty, express or implied, with respect to the material contained herein Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) CuuDuongThanCong.com Dedicated to the memory of Walter Bürgisser and Gritta Bürgisser-Glogau and of Federico Cucker and Rosemary Farkas in love and gratitude CuuDuongThanCong.com Preface Motivation A combined search at Mathscinet and Zentralblatt shows more than 800 articles with the expression “condition number” in their title It is reasonable to assume that the number of articles dealing with conditioning, in one way or another, is a substantial multiple of this quantity This is not surprising The occurrence of condition numbers in the accuracy analysis of numerical algorithms is pervasive, and its origins are tied to those of the digital computer Indeed, the expression “condition number” itself was first introduced in 1948, in a paper by Alan M Turing in which he studied the propagation of errors for linear equation solving with the then nascent computing machinery [221] The same subject occupied John von Neumann and Herman H Goldstine, who independently found results similar to those of Turing [226] Ever since then, condition numbers have played a leading role in the study of both accuracy and complexity of numerical algorithms To the best of our knowledge, and in stark contrast to this prominence, there is no book on the subject of conditioning Admittedly, most books on numerical analysis have a section or chapter devoted to it But their emphasis is on algorithms, and the links between these algorithms and the condition of their data are not pursued beyond some basic level (for instance, they contain almost no instances of probabilistic analysis of algorithms via such analysis for the relevant condition numbers) Our goal in writing this book has been to fill this gap We have attempted to provide a unified view of conditioning by making condition numbers the primary object of study and by emphasizing the many aspects of condition numbers in their relation to numerical algorithms Structure The book is divided into three parts, which approximately correspond to themes of conditioning in linear algebra, linear programming, and polynomial equation solving, respectively The increase in technical requirements for these subjects is reflected in the different paces for their expositions Part I proceeds leisurely and can be used for a semester course at the undergraduate level The tempo increases in Part II and reaches its peak in Part III with the exposition of the recent advances in and partial solutions to the 17th of the problems proposed by Steve Smale for the mathematicians of the 21st century, a set of results in which conditioning plays a paramount role [27, 28, 46] vii CuuDuongThanCong.com viii Preface As in a symphonic poem, these changes in cadence underlie a narration in which, as mentioned above, condition numbers are the main character We introduce them, along with the cast of secondary characters making up the dramatis personae of this narration, in the Overture preceding Part I We mentioned above that Part I can be used for a semester course at the undergraduate level Part II (with some minimal background from Part I) can be used as an undergraduate course as well (though a notch more advanced) Briefly stated, it is a “condition-based” exposition of linear programming that, unlike more elementary accounts based on the simplex algorithm, sets the grounds for similar expositions of convex programming Part III is also a course on its own, now on computation with polynomial systems, but it is rather at the graduate level Overlapping with the primary division of the book into its three parts there is another taxonomy Most of the results in this book deal with condition numbers of specific problems Yet there are also a few discussions and general results applying either to condition numbers in general or to large classes of them These discussions are in most of the Overture, the two Intermezzi between parts, Sects 6.1, 6.8, 9.5, and 14.3, and Chaps 20 and 21 Even though few, these pages draft a general theory of condition, and most of the remainder of the book can be seen as worked examples and applications of this theory The last structural attribute we want to mention derives from the technical characteristics of our subject, which prominently features probability estimates and, in Part III, demands some nonelementary geometry A possible course of action in our writing could have been to act like Plato and deny access to our edifice to all those not familiar with geometry (and, in our case, probabilistic analysis) We proceeded differently Most of the involved work in probability takes the form of estimates— of either distributions’ tails or expectations—for random variables in a very specific context We therefore included within the book a Crash Course on Probability providing a description of this context and the tools we use to compute these estimates It goes without saying that probability theory is vast, and alternative choices in its toolkit could have been used as well A penchant for brevity, however, prevented us to include these alternatives The course is supplied in installments, six in total, and contains the proofs of most of its results Geometry requirements are of a more heterogeneous nature, and consequently, we have dealt with them differently Some subjects, such as Euclidean and spherical convexity, and the basic properties of projective spaces, are described in detail within the text But we could not so with the basic notions of algebraic, differential, and integral geometry We therefore collected these notions in an appendix, providing only a few proofs Paderborn, Germany Hong Kong, Hong Kong SAR May 2013 CuuDuongThanCong.com Peter Bürgisser Felipe Cucker Acknowledgements A substantial part of the material in this book formed the core of several graduate courses taught by PB at the University of Paderborn Part of the material was also used in a graduate course at the Fields Institute held in the fall of 2009 We thank all the participants of these courses for valuable feedback In particular, Dennis Amelunxen, Christian Ikenmeyer, Stefan Mengel, Thomas Rothvoss, Peter Scheiblechner, Sebastian Schrage, and Martin Ziegler, who attended the courses in Paderborn, had no compassion in pointing to the lecturer the various forms of typos, redundancies, inaccuracies, and plain mathematical mistakes that kept popping up in the early drafts of this book used as the course’s main source We thank Dennis Amelunxen for producing a first LATEX version of the lectures in Paderborn, which formed the initial basis of the book In addition, Dennis was invaluable in producing the TikZ files for the figures occurring in the book Also, Diego Armentano, Dennis Cheung, Martin Lotz, and Javier Peña read various chapters and have been pivotal in shaping the current form of these chapters We have pointed out in the Notes the places where their input is most notable Finally, we want to emphasize that our viewpoint about conditioning and its central role in the foundations of numerical analysis evolved from hours of conversations and exchange of ideas with a large group of friends working in similar topics Among them it is impossible not to mention Carlos Beltrán, Lenore Blum, Irenée Briquel, Jean-Pierre Dedieu, Alan Edelman, Raphael Hauser, Gregorio Malajovich, Luis Miguel Pardo, Jim Renegar, Vera Roshchina, Michael Shub, Steve Smale, Henryk Wo´zniakowski, and Mario Wschebor We are greatly indebted to all of them The financial support of the German Research Foundation (individual grants BU 1371/2-1 and 1371/2-2) and the GRF (grant CityU 100810) is gratefully acknowledged We also thank the Fields Institute in Toronto for hospitality and financial support during the thematic program on the Foundations of Computational Mathematics in the fall of 2009, where a larger part of this monograph took definite form We thank the staff at Springer-Verlag in Basel and Heidelberg for their help and David Kramer for the outstanding editing work he did on our manuscript Finally, we are grateful to our families for their support, patience, and understanding of the commitment necessary to carry out such a project while working on different continents ix CuuDuongThanCong.com Contents Part I Condition in Linear Algebra (Adagio) Normwise Condition of Linear Equation Solving 1.1 Vector and Matrix Norms 1.2 Turing’s Condition Number 1.3 Condition and Distance to Ill-posedness 1.4 An Alternative Characterization of Condition 1.5 The Singular Value Decomposition 1.6 Least Squares and the Moore–Penrose Inverse 10 11 12 17 Probabilistic Analysis 2.1 A Crash Course on Integration 2.2 A Crash Course on Probability: I 2.2.1 Basic Facts 2.2.2 Gaussian Distributions 2.2.3 The χ Distribution 2.2.4 Uniform Distributions on Spheres 2.2.5 Expectations of Nonnegative Random Variables 2.2.6 Caps and Tubes in Spheres 2.2.7 Average and Smoothed Analyses 2.3 Probabilistic Analysis of Cwi (A, x) 2.4 Probabilistic Analysis of κrs (A) 2.4.1 Preconditioning 2.4.2 Average Analysis 2.4.3 Uniform Smoothed Analysis 2.5 Additional Considerations 2.5.1 Probabilistic Analysis for Other Norms 2.5.2 Probabilistic Analysis for Gaussian Distributions 21 22 27 28 33 35 38 39 41 46 48 50 51 53 55 56 56 57 Error Analysis of Triangular Linear Systems 3.1 Random Triangular Matrices Are Ill-conditioned 59 60 xi CuuDuongThanCong.com 538 Bibliography 162 J Renegar On the worst-case arithmetic complexity of approximating zeros of systems of polynomials SIAM Journal on Computing, 18:350–370, 1989 163 J Renegar On the computational complexity and geometry of the first-order theory of the reals I, II, III Journal of Symbolic Computation, 13(3):255–352, 1992 164 J Renegar Is it possible to know a problem instance is ill-posed? Journal of Complexity, 10:1–56, 1994 165 J Renegar Some perturbation theory for linear programming Mathematical Programming, 65:73–91, 1994 166 J Renegar Incorporating condition measures into the complexity theory of linear programming SIAM Journal on Optimization, 5:506–524, 1995 167 J Renegar Linear programming, complexity theory and elementary functional analysis Mathematical Programming, 70:279–351, 1995 168 J Renegar A Mathematical View of Interior-Point Methods in Convex Optimization SIAM, Philadelphia, 2000 169 W.C Rheinboldt Numerical Analysis of Parametrized Nonlinear Equations, volume of University of Arkansas Lecture Notes in the Mathematical Sciences Wiley, New York, 1986 170 J.R Rice A theory of condition SIAM Journal on Numerical Analysis, 3:217–232, 1966 171 R.T Rockafellar Convex Analysis, Princeton Landmarks in Mathematics Princeton University Press, Princeton, 1997 Reprint of the 1970 original, Princeton Paperbacks 172 J Rohn Systems of linear interval equations Linear Algebra and Its Applications, 126:39– 78, 1989 173 R Rosenblatt Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms Spartan Books, East Lansing, 1962 174 M Rudelson and R Vershynin Smallest singular value of a random rectangular matrix Communications on Pure and Applied Mathematics, 62(12):1707–1739, 2009 175 S.M Rump Structured perturbations part I: normwise distances SIAM Journal on Matrix Analysis and Applications, 25:1–30, 2003 176 S.M Rump Structured perturbations part II: componentwise distances SIAM Journal on Matrix Analysis and Applications, 25:31–56, 2003 177 J Sabia Algorithms and their complexities In Solving Polynomial Equations, volume 14 of Algorithms Comput Math., pages 241–268 Springer, Berlin, 2005 178 R Saigal A modern integrated analysis In Linear Programming, volume of International Series in Operations Research Management Science Kluwer Academic, Boston, 1995 179 A Sankar, D.A Spielman, and S.-H Teng Smoothed analysis of the condition numbers and growth factors of matrices SIAM Journal on Matrix Analysis and Applications, 28(2):446– 476, 2006 180 L.A Santaló Integral geometry in Hermitian spaces American Journal of Mathematics, 74:423–434, 1952 181 L.A Santaló Integral Geometry and Geometric Probability, volume of Encyclopedia of Mathematics and Its Applications Addison-Wesley, Reading, 1976 With a foreword by Mark Kac 182 E Schmidt Zur Theorie der linearen und nichtlinearen Integralgleichungen Mathematische Annalen, 63(4):433–476, 1907 183 R Schneider Convex Bodies: The Brunn-Minkowski Theory, volume 44 of Encyclopedia of Mathematics and Its Applications Cambridge University Press, Cambridge, 1993 184 R Schneider and W Weil Stochastic and Integral Geometry, Probability and Its Applications (New York) Springer, Berlin, 2008 185 A Schönhage The fundamental theorem of algebra in terms of computational complexity Technical Report, Institute of Mathematics, University of Tübingen, 1982 186 I.R Shafarevich Varieties in projective space In Basic Algebraic Geometry 1, 2nd edition Springer, Berlin, 1994 Translated from the 1988 Russian edition and with notes by Miles Reid 187 I.R Shafarevich Schemes and complex manifolds In Basic Algebraic Geometry 2, 2nd edition Springer, Berlin, 1994 Translated from the 1988 Russian edition by Miles Reid CuuDuongThanCong.com Bibliography 539 188 M Shub Some remarks on Bézout’s theorem and complexity theory In From Topology to Computation: Proceedings of the Smalefest, Berkeley, CA, 1990, pages 443–455 Springer, New York, 1993 189 M Shub Complexity of Bézout’s Theorem VI: geodesics in the condition (number) metric Foundations of Computational Mathematics, 9:171–178, 2009 190 M Shub and S Smale Computational complexity: on the geometry of polynomials and a theory of cost I Annales Scientifiques de L’Ecole Normale Supérieure, 18(1):107–142, 1985 191 M Shub and S Smale Computational complexity: on the geometry of polynomials and a theory of cost II SIAM Journal on Computing, 15(1):145–161, 1986 192 M Shub and S Smale Complexity of Bézout’s Theorem I: geometric aspects Journal of the American Mathematical Society, 6:459–501, 1993 193 M Shub and S Smale Complexity of Bézout’s Theorem II: volumes and probabilities In F Eyssette and A Galligo, editors, Computational Algebraic Geometry, volume 109 of Progress in Mathematics, pages 267–285 Birkhäuser, Basel, 1993 194 M Shub and S Smale Complexity of Bézout’s Theorem III: condition number and packing Journal of Complexity, 9:4–14, 1993 195 M Shub and S Smale Complexity of Bézout’s Theorem V: polynomial time Theoretical Computer Science, 133:141–164, 1994 196 M Shub and S Smale Complexity of Bézout’s Theorem IV: probability of success; extensions SIAM Journal on Numerical Analysis, 33:128–148, 1996 197 J.W Silverstein The smallest eigenvalue of a large-dimensional Wishart matrix Annals of Probability, 13(4):1364–1368, 1985 198 R.D Skeel Scaling for numerical stability in Gaussian elimination Journal of the ACM, 26:494–526, 1979 199 S Smale The fundamental theorem of algebra and complexity theory Bulletin of the American Mathematical Society, 4:1–36, 1981 200 S Smale Newton’s method estimates from data at one point In, R Ewing, K Gross, and C Martin, editors, The Merging of Disciplines: New Directions in Pure, Applied, and Computational Mathematics Springer, Berlin, 1986 201 S Smale Complexity theory and numerical analysis In A Iserles, editor, Acta Numerica, pages 523–551 Cambridge University Press, Cambridge, 1997 202 S Smale Mathematical problems for the next century The Mathematical Intelligencer, 20(2):7–15, 1998 203 R Solovay and V Strassen A fast Monte-Carlo test for primality SIAM Journal on Computing, 6:84–85, 1977 204 R Solovay and V Strassen Erratum on “A fast Monte-Carlo test for primality” SIAM Journal on Computing, 7:118, 1978 205 A.J Sommese and C.W Wampler II The Numerical Solution of Systems of Polynomials World Scientific, Hackensack, 2005 206 D.A Spielman and S.-H Teng Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time In Proceedings of the Thirty-Third Annual ACM Symposium on Theory of Computing, pages 296–305 ACM, New York, 2001 207 D.A Spielman and S.-H Teng Smoothed analysis of algorithms In Proceedings of the International Congress of Mathematicians, volume I, pages 597–606, 2002 208 D.A Spielman and S.-H Teng Smoothed analysis: why the simplex algorithm usually takes polynomial time Journal of the ACM, 51(3):385–463, 2004 209 M Spivak Calculus on Manifolds a Modern Approach to Classical Theorems of Advanced Calculus W A Benjamin, New York, 1965 210 J Steiner Über parallele Flächen Monatsber Preuss Akad Wiss., 114–118, 1840 211 G.W Stewart On the perturbation of pseudo-inverses, projections and linear least squares problems SIAM Review, 19(4):634–662, 1977 212 G.W Stewart Stochastic perturbation theory SIAM Review, 32(4):579–610, 1990 CuuDuongThanCong.com 540 Bibliography 213 G.W Stewart On the early history of the singular value decomposition SIAM Review, 35(4):551–566, 1993 214 G.W Stewart and J.-G Sun Matrix Perturbation Theory, Computer Science and Scientific Computing Academic Press, Boston, 1990 215 G Stolzenberg Volumes, Limits, and Extensions of Analytic Varieties, volume 19 of Lecture Notes in Mathematics Springer, Berlin, 1966 216 T Tao and V Vu Inverse Littlewood-Offord theorems and the condition number of random discrete matrices Annals of Mathematics Second Series, 169(2):595–632, 2009 217 T Tao and V Vu Smooth analysis of the condition number and the least singular value Mathematics of Computation, 79(272):2333–2352, 2010 218 J.A Thorpe Elementary topics in differential geometry In Undergraduate Texts in Mathematics Springer, New York, 1994 Corrected reprint of the 1979 original 219 L.N Trefethen and D Bau III Numerical Linear Algebra SIAM, Philadelphia, 1997 220 L.N Trefethen and R.S Schreiber Average-case stability of Gaussian elimination SIAM Journal on Matrix Analysis and Applications, 11:335–360, 1990 221 A.M Turing Rounding-off errors in matrix processes Quarterly Journal of Mechanics and Applied Mathematics, 1:287–308, 1948 222 B.L van der Waerden Modern Algebra Vol II Frederick Ungar, New York, 1950 Translated from the second revised German edition by Theodore J Benac 223 S.A Vavasis and Y Ye Condition numbers for polyhedra with real number data Operations Research Letters, 17:209–214, 1995 224 S.A Vavasis and Y Ye A primal-dual interior point method whose running time depends only on the constraint matrix Mathematical Programming, 74:79–120, 1996 225 D Viswanath and L.N Trefethen Condition numbers of random triangular matrices SIAM Journal on Matrix Analysis and Applications, 19:564–581, 1998 226 J von Neumann and H.H Goldstine Numerical inverting matrices of high order Bulletin of the American Mathematical Society, 53:1021–1099, 1947 227 P.-Å Wedin Perturbation theory for pseudo-inverses BIT, 13:217–232, 1973 228 N Weiss, G.W Wasilkowski, H Wo´zniakowski, and M Shub Average condition number for solving linear equations Linear Algebra and Its Applications, 83:79–102, 1986 229 J.G Wendel A problem in geometric probability Mathematica Scandinavica, 11:109–111, 1962 230 H Weyl Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung) Mathematische Annalen, 71(4):441–479, 1912 231 H Weyl The Theory of Groups and Quantum Mechanics Dover, New York, 1932 232 H Weyl On the volume of tubes American Journal of Mathematics, 61(2):461–472, 1939 233 E Wigner Random matrices in physics SIAM Review, 9:1–23, 1967 234 J.H Wilkinson Error analysis of direct methods of matrix inversion Journal of the Association for Computing Machinery, 8:281–330, 1961 235 J.H Wilkinson Rounding Errors in Algebraic Processes Prentice Hall, New York, 1963 236 J.H Wilkinson The Algebraic Eigenvalue Problem Clarendon Press, Oxford, 1965 237 J.H Wilkinson Modern error analysis SIAM Review, 13:548–568, 1971 238 J.H Wilkinson Note on matrices with a very ill-conditioned eigenproblem Numerische Mathematik, 19:176–178, 1972 239 J Wishart The generalized product moment distribution in samples from a normal multivariate population Biometrika, 20A(272):32–43, 1928 240 R Wongkew Volumes of tubular neighbourhoods of real algebraic varieties Pacific Journal of Mathematics, 159(1):177–184, 1993 241 H Wo´zniakowski Numerical stability for solving nonlinear equations Numerische Mathematik, 27(4):373–390, 1976/77 242 S Wright Primal-Dual Interior-Point Methods SIAM, Philadelphia, 1997 243 M.H Wright The interior-point revolution in optimization: history, recent developments, and CuuDuongThanCong.com Bibliography 541 lasting consequences Bulletin, New Series, of the American Mathematical Society, 42(1):39– 56, 2005 244 M Wschebor Smoothed analysis of κ(A) Journal of Complexity, 20(1):97–107, 2004 245 Y Ye Toward probabilistic analysis of interior-point algorithms for linear programming Mathematics of Operations Research, 19:38–52, 1994 246 T.J Ypma Historical development of the Newton-Raphson method SIAM Review, 37(4):531–551, 1995 CuuDuongThanCong.com Notation Symbols #(S), 345 #C (f ), 374 #R (f ), 393 #P, 516 #PR , 516 , v , 272 1A , 23 A acondϕ (x), 262 acondG (x), 266, 276 A† , 17 aff{a1 , , an }, 125 aff(M), 125 α(f, x), 397 αproj (f, x), 397 α† (f, x), 414 B Bn , 37 B(a, σ ), 42 Bsin (a, σ ), 439 BS (x, δ), 401 B∞ (x, η), 409 β(f, x), 397 βproj (f, x), 397 β† (f, x), 414 C C, 126 Cζ , 305 C (A), 134 C(d), 194 cap(p, α), 42 CG (W ), 520 Cn , 405 condϕ (a), xix ϕ cond[2] (a), 256 ϕ condst (f ), 264 ϕ condW (f ), 265 condG (x), 266 cone{a1 , , an }, 126 cone(M), 126 conv{a1 , , an }, 125 conv(M), 125 costA (a), 102 Cwϕ (a), xix, 65 D D , 310 deg Z, 488 det, 473 diag, 12 dims, xxiv disc(f ), 435 discd (f ), 436, 491 daff (x, y), 383 dH (K, K ), 458 dM (x, y), 478 dP (x, y), 273 dS (x, y), 135, 272 dsin (x, y), 42, 52, 274 ∂K, 127 E E, 29 en , 25 en , 175 Error, xxiii mach , xxii P Bürgisser, F Cucker, Condition, Grundlehren der mathematischen Wissenschaften 349, DOI 10.1007/978-3-642-38896-5, © Springer-Verlag Berlin Heidelberg 2013 CuuDuongThanCong.com 543 544 F F, xxii fl(q), xxiii FP(d), 171 ϕHR , 413 d ϕn , 33 a,σ ϕn , 33 G GLn (R), 26 Gn , 225 G(m, n), 487 Γ , 36 γk , xxii γn , 33 γna,σ , 33 γ (f, z), 288 γ (f, x), 397 γproj (f, z), 315 γ† (f, x), 414 H Hd , 297 Hd , 299 HdR , 372, 391 HdR [m], 414 Notation median, 37 M≤ε , 494 Mε , 494 Mϕ (a), xix MPf (x), 414 μ(f, ζ ), 302 μav (f ), 356 μmax (f ), 364 μnorm (f, ζ ), 307, 321 μ† (f, x), 414 μ˜ i (M), 500 μi (U ), 442 μ(M; ˜ X), 500 N N(a, σ In ), 33 NT (0, σ In ), 358 NT (a, σ In ), 367 Nf (z), 286, 313 NJψ(x), 344 ν(f ), 394 ν(f, x), 394 O OB(d), 169 o(h), 468 int(M), 130 O (h), 467 On , 36 O (n), 26, 446 Op,k (ε), 422 J Js (σ ), 461 Ω(h), 468 Ω n (M), 474 Ωn (V ), 473 K P K (d), 204 PCFP, 131, 184 Pk , 111 I KM,i (x), 441 KM,i (x, v), 500 κeigen (A), 433 κ(f ), 392 κF (A), 433 κfeas (f ), 416, 530 κrs (A), 7, 50 L Lζ , 305 Lk (V1 , , Vk ; W ), 287 LM (x), 441, 480 LoP, xxiii, xxiv M M , 347 MR , 372, 413 CuuDuongThanCong.com Pn , 269 Pn0 , 383 p(n, m), 235 p(n, m, α), 236 Prob, 29 P(V ), 269 ψδ (u), 316 ψ(u), 289 Q QD , 160 Qk , 111 QP , 160 Q (L), 460 QS , 460 QTriang (L), 459 Notation R Rζ , 304 r_costA (a), 356 Reg(Z), 484 RelError, xviii relint(K), 127 resd , 490 round, xxii ρ(A), 136 (d), 204 ρdeg (d), 205 D (d), 205 ρdeg P (d), 205 ρdeg ρHd , 368 ρsing (S), 204 ρst , 350 ρˆst , 351 ρVˆ , 369 S sconv(M), 239 SD , 160, 193 Sing(Z), 484 size(a), xxiv, 102 SLI, 144, 147 Sn , 21 S ◦ , 174 Sol(a), 143 SP , 160, 193 Stn,k , 280, 470 Σ , 265, 275, 302 ΣFP , 171 ΣI , 168 ΣOB , 169, 203 Σopt , 168, 203 Σζ , 308 T Ti (∂K, ε), 240 Ti⊥ (U, ε), 458 Tk , 113 CuuDuongThanCong.com 545 To (∂K, ε), 240 To⊥ (U, ε), 458 T (Sn−2 , ε), 44 T (U, ε), 240, 422, 440 T ⊥ (U, ε), 44, 441 Triang, 459 Θ(h), 468 θk , xxii U U , 309 U (n), 94, 297 U (Sn ), 38 V Vˆ , 346 Vζ , 308 Var, 29 Vε , 449 vol, 23 volRm , 23 volSn−1 , 23 VR , 372 VˆR , 413 W Wˆ , 347 W (m, n), 83 WR , 372 Wˆ R , 413 W , 202 WB , 203 X χn2 , 35 Z Z(f ), 481 ZP (f ), 310 ZS (f ), 393 Concepts Symbols δ-approximation, 383 ε-neighborhood, 44, 240, 422, 440 ε-tube, 44, 441 U (n)-equivariance, 301 A affine cone, 483 affine hull, 125 algebraic cone, 419 algebraic variety affine, 481 irreducible, 482 projective, 483 pure dimensional, 483 real, 485 algorithm Ren, 384 Adaptive_Homotopy, 285 ALH, 334 backward-stable, xx BP_Randomization_scheme, 354 Conj_Grad, 110 Ellip_Method, 151 FEAS_LP, 195 forward-stable, xxi FS, 64 Homotopy_Continuation, 284 ItRen, 385 Las Vegas, 342 LV, 342 MD, 382 Monte-Carlo, 342 nonuniform, 514 OB, 212 OB2, 216 Perceptron, 144 Primal–Dual IPM, 177 Primal-Dual_IPM_for_PCFP, 188 random_h, 355 Randomized_Primality_Testing, 341 Randomized_Primality_Testing_2, 341 random_system, 353 Underdetermined_Feasibility, 415 Zero_Counting, 408 almost all, 477 almost everywhere, 23 approximate zero, 287, 315 associated zero, 287, 315 atlas, 471 holomorphic, 472 average-case analysis, 21, 46 B backward-error analysis, xx balls in Euclidean space, 37 in spheres, 42 Banach fixed point theorem, 400 barrier function, 178 basin of quadratic attraction, 287 Bézout number, 310 Bézout series, 512 Bézout’s inequality, 489 Bézout’s theorem, 310 big oh, 467 big omega, 468 blocking set, 136 Borel-measurable set, 23 C Carathéodory’s theorem, 127 centering parameter, 174 central limit theorem, 81 P Bürgisser, F Cucker, Condition, Grundlehren der mathematischen Wissenschaften 349, DOI 10.1007/978-3-642-38896-5, © Springer-Verlag Berlin Heidelberg 2013 CuuDuongThanCong.com 547 548 central neighborhood, 182 central path, 174 chart, 471 Chebyshev polynomials, 113 chi-square distribution, 35 classical topology, 482 coarea formula, 344 complementary slackness condition, 158 theorem, 158 complete problem, 516 complexity, xxviii, 102 concentration inequalities, 80 condition geodesic, 528 condition length, 528 condition map, 266, 275 condition matrix, 266 condition metric, 528 condition number, xix la Renegar, 125, 256 absolute normwise, 262, 266, 276 componentwise, xix, 65 conic, 419, 439 Frobenius, 433 GCC, 134 Grassmann, 520 level-2, 256 maximum, 364 mean square, 356 mixed, xix normalized, 307 normwise, xix of a differentiable function, 262 RCC, 204, 524 relative normwise, 266 stochastic, 505 condition number theorem, 10 condition-based complexity analysis, xxviii conditional density, 29, 250, 346 conditional expectation, 32 conditional probability, 251 continuation methods, 283 contraction constant, 400 map, 400 convex body in Sp , 239, 455 smooth, in Sp , 456 convex cone, 126 pointed, 238 convex hull, 125 convex set, 125 correct digits, xxii cost, 102 CuuDuongThanCong.com Concepts algebraic, 103 average expected, 357 average randomized, 357 bit, 103 of conjugate gradient, 109, 111, 116 of Gaussian elimination, 102 randomized, 342, 356 covariance matrix, 34 covering processes, 236 critical value, 473 curvature, ith, 441, 480 curvature polynomials, 500 D data space, 22 degree of an algebraic variety cumulative, 489 of a pure dimensional, 489 of an irreducible, 488 degree pattern, 299 diffeomorphism, 24, 469 between manifolds, 472 dimension of a convex set, 125 of a manifold, 471 of an algebraic variety, 483 direct methods, 102 discriminant polynomial, 435, 436, 491 discriminant variety, 311 distance on projective space, 273, 478 on the sphere, 272, 478 distance to singularity, 204 distribution adversarial, 461 of a random variable, 31 double factorial, 451 dual cone, 126 dual set, 239 duality gap, 158 duality measure, 174 duality theorem of linear programming, 157 E ellipsoid, 147 ellipsoid method, 147, 151 error componentwise, xviii in a computation, xvii normwise, xviii relative, xviii Euler’s formula, 300 event, 28 expectation, 29 expected value, 29 Concepts F Farkas’s lemma, 126 floating-point numbers, xxii forms on a manifold, 474 forward-approximate solution, 192 Fubini–Study metric, 273, 478 Fubini’s theorem, 24 function concave, 41 integrable, 24 measurable, 23 G gamma function, 36 Gauss map, 448 Gaussian curvature, 442 Gaussian distribution center of, 34 centered, 34 isotropic multivariate, 33 standard, 33 truncated, 358 variance, 34 Gaussian elimination, 74, 102 general linear group, 26 general position of a hyperplane arrangement, 235 gradient method, 103 Grassmann manifold, 280, 487, 520 great circle segment, 238 group invariance, 225 growth factor, 521 H Haar measure (normalized), 426, 447, 479 Hadamard’s formula, 288 Hadamard’s inequality, 153 half-space closed, 126 open, 126 Hausdorff distance, 458 Helly’s theorem, 127 Hilbert’s basis theorem, 482 Hilbert’s Nullstellensatz, 481 homogeneous, 486 problem, 514 Hölder inequality, homotopy methods, 283 Householder matrix, 81 I ideal, 481 prime, 482 ill-posed solution pair, 265, 275 CuuDuongThanCong.com 549 ill-posedness, xxx, distance to, xxxi, 10, 16, 125, 204 implicit function theorem, 469 independence of data spaces, 28 of random variables, 32 indicator function, 23 inner ε-tube, 458 inner neighborhood, 240 inner volumes, 519 spherical, 519 integration in polar coordinates in Euclidean space, 25 on a sphere, 26 integration on manifolds, 474 interior-point method primal–dual, 173 iterative methods, 102 J Jacobian, 25 Jacobian matrix, 25 Jensen’s inequality, 41 K Kähler manifold, 492 Kantorovich’s inequality, 107 Karush–Kuhn–Tucker matrix, 176 Krylov spaces, 110 L Lagrange multipliers, 179 least squares, 18, 101 length of curves on a manifold, 478 level sets, 494 Lie group, 479 linear program basic optimal solution, 163 basis of, 163 bounded, 156 constraint, 155 degenerate solution, 166 dual, 156 dual basic solution, 163 dual degenerate, 166 basis, 205 dual heavy, 161 dual light, 161 extremal optimal solution, 162 feasible ill-posed, 202 feasible set, 156 feasible well-posed, 202 in standard form, 155, 159 nearly infeasible, 162 550 linear program (cont.) objective function, 156 optimal solution, 156 optimal value, 156, 201 optimizer, 156, 201 primal basic solution, 163 primal degenerate, 166 basis, 205 primal heavy, 161 primal light, 161 linear programming, 155 feasibility problem, 171 ill-posedness, 168 optimal basis problem, 169 optimal solution problem, 168 optimal value problem, 171 linearization map, 371 Lipschitz property of μnorm , 296 little oh, 468 locally closed set, 489 loss of precision, xxiii Löwner–John ellipsoid, 150 LU factorization, 74 M machine epsilon, xxii manifold, 471 complex, 472 oriented, 472, 479 submanifold, 469, 472 marginal density, 28, 346 Markov’s inequality, 30 measurable function, 23 set, 23 measure on a data space, 23 median, 37 Moore–Penrose inverse, 17 Moore–Penrose Newton’s iteration, 414, 513, 517 multinomial coefficients, 297 multiple zero, 302 multivariate discriminant, 311 N Newton’s method, 174, 286 cost, 313 on Riemannian manifolds, 513 on the sphere, 393 norm dual, Frobenius, of a matrix, CuuDuongThanCong.com Concepts of a vector, spectral, normal Jacobian, 344 normalized integrals of absolute curvature, 442 of curvature, 442 of curvature (modified), 500 O optimality conditions, 158 orthogonal group, 26, 446, 479 orthogonal invariance, 26, 34 outer ε-tube, 458 outer neighborhood, 240 overflow, xxi P partial pivoting, 522 partition of unity, 474 path-following method, 174 perturbation, xviii Poincaré’s formula, 496, 518 in complex projective space, 498 polyhedral cone feasibility problem, 131 polyhedral system, 193 polyhedron, 128 face of, 128 proper face of, 128 vertices of, 128 preconditioning, 51 primality testing, 340 principal curvatures, 441, 480 principal kinematic formula, 447, 501 probability density on a data space, 28 on a manifold, 344 probability distribution on a data space, 28 probability measure, 28 problem decision, 124 discrete-valued, 124 product measure, 23 projective γ -theorem, 317 projective Newton’s operator, 314 projective space, 269 tangent space, 270, 484 pseudorandom generators, 341 pushforward measure, 31, 345 Q QR factorization, 3, 74 loss of precision, 9, 22 Quermass integrals, 519 Concepts R Rand_Gaussian( ), 353 random data average case, 21 smoothed analysis, 21 random variable, 29 random vector, 34 Random_bit( ), 341 randomization, 341 realization set, 235 reduction, 516 regular point of a function, 473 of an algebraic variety, 484 regular value, 473 relative boundary, 127 relative interior, 127 Renegar’s trick, 506 representation basis, xxi exponent, xxi mantissa, xxi precision, xxi reproducing kernel Hilbert space, 298 reproducing property, 298 rescaled perceptron, 508 resultant, 490 Riemannian distance, 478 manifold, 478 metric, 478 round-off unit, xxii rounding map, xxii running time, 102 S Sard’s theorem, 475 scale invariance, 38 semialgebraic system, 417 separating hyperplane theorem, 125 separation of zeros, 293, 320 separation oracle, 151 set of ill-posed inputs, xxx, 120, 124, 256, 276, 419, 439 set of ill-posed solutions, 302 sign pattern, 235 simple zero, 302 sine distance, 42 on P(V ), 274 on product spaces, 52 singular point of an algebraic variety, 484 singular value decomposition, 12 singular values, 13 CuuDuongThanCong.com 551 singular vectors, 13 size, xxiv, 102 slack variables, 157 Smale’s 17th problem, 331, 526 7th problem, 514, 528 9th problem, 524 α-theorem, 398 γ -theorem, 289 smallest including cap, 136 smooth map, 469 on a manifold, 472 smoothed analysis, 21, 46 solution manifold, 265, 276, 300 solution map, 266, 275 space of inputs, 265 space of outputs, 265 sparse matrix, 65 spherical cap, 42 spherical convex hull, 239 spherically convex set, 238 proper, 238 stability backward, xx forward, xxi stabilizer, 476 standard chart, 25 standard distribution, 350 standard normal, 33 steepest descent, 103 Stiefel manifold, 280, 470 Stirling bounds, 36 strict complementarity theorem, 159 structured data, 119 ill-posedness, 119 perturbations, 119 submanifold of Rn , 469 of an abstract manifold, 472 system of linear inequalities, 144 T tangent space, 469 of a manifold, 472 tangent vector, 469 on a manifold, 472 theta, 468 Tonelli’s theorem, 24 topology classical, 482 Zariski, 482 transformation formula, 24 552 transversality, 475, 476 triangular systems, 59 backward error analysis, 64 componentwise condition, 65 U underdetermined linear systems, 18 underflow, xxi uniform distribution on a manifold, 343 on data spaces, 28 on spheres, 38 on spherical caps, 42 unitary group, 94, 297, 479 V vanishing ideal, 481 variance, 29 vectors A-orthogonal, 108 conjugated, 108 volume of a ball, 37 of a measurable set on a manifold, 479 of a sphere, 36 of a spherical cap, 42, 461 CuuDuongThanCong.com Concepts of a tube, 44, 422, 443 of an ellipsoid, 148 of complex projective space, 345 of irreducible varieties in projective space, 426 on algebraic varieties, 425 volume element, 479 volume form, 479 W Weingarten map, 441, 480 well-posed solution pair, 265 Weyl’s basis, 297 Weyl’s tube formula, 443, 500, 518 Weyl’s inner product on Hd , 299 on Hd , 297 Wirtinger’s inequality, 491 Wishart distribution, 83, 116 worst-case analysis, 47 Z Zariski almost all, 485 Zariski tangent space, 484 Zariski topology, 482 and the People Who Crafted Them A Abel, Niels Henrik, 261 Adleman, Leonard Max, 341 Amelunxen, Dennis, 511, 519 Armentano, Diego, 512, 515, 523 Azaïs, Jean-Marc, 517 B Barbier, Joseph-Émile, 518 Bau, David, 504, 523 Baur, Walter, 513 Belloni, Alexandre, 508 Beltrán, Carlos, 331, 512, 515, 517 Blaschke, Wilhelm Johann Eugen, 519 Blum, Lenore, 504, 517 Brothers, John, 518 Buffon, George-Louis Leclerc, Comte de, 518 C Chen, Zizhong, 507 Chern, Shiing-Shen, 519 Cheung, Dennis, 508, 510 Chistov, Alexander Leonidovich, 514 Courant, Richard, 503 D Dantzig, George Bernard, 509 Dedieu, Jean-Pierre, 513 Demmel, James Weldon, 503, 504, 511, 512, 517, 518, 523 Dongarra, Jack J., 507 Dunagan, John, 511 E Eckart, Carl Henry, 503 Edelman, Alan, 506, 518 Epelman, Marina A., 509 Euler, Leonhard, xvii F Federer, Herbert, 516, 518, 519 Fletcher, Roger, 505 Fourier, Jean-Baptiste Joseph, 509 Freund, Robert M., 508, 509 G Galois, Évariste, 261 Gastinel, Noël, 505 Gauss, Carl Friedrich, xvii, 513 Giusti, Marc, 514 Glasauer, Stefan, 519 Goffin, Jean-Louis, 508 Gohberg, Israel, 504 Goldstine, Herman Heine, vii, xxviii Golub, Gene Howard, 504 Grigoriev, Dimitri Yurevich, 514 Grötschel, Martin, 509 H Hauser, Raphael, 520 Heintz, Joos, 514 Hestenes, Magnus Rudolf, 508 Higham, Desmond J., 511 Higham, Nicholas John, 504 Hilbert, David, 503 Howard, Ralph, 518 Huang, Ming-Deh, 341 K Kahan, William Morton, 505 Kantorovich, Leonid Vitaliyevich, 283, 512 Karmarkar, Narendra K., 509 Khachiyan, Leonid Genrikhovich, 509 Klain, Daniel, 519 P Bürgisser, F Cucker, Condition, Grundlehren der mathematischen Wissenschaften 349, DOI 10.1007/978-3-642-38896-5, © Springer-Verlag Berlin Heidelberg 2013 CuuDuongThanCong.com 553 554 Koltrach, Israel, 504 Kostlan, Eric, 517 and the People Who Crafted Them M Malajovich, Gregorio, 513, 529 Minkowski, Hermann, 519 Muirhead, Robb J., 506 Müller, Tobias, 520 Mumford, David Bryant, 481, 512 Schrijver, Alexander, 509 Shafarevich, Igor Rostislavovich, 481 Shor, Naum Zuselevich, 509 Shub, Michael, 296, 512, 515, 517 Skeel, Robert, 504 Smale, Stephen, vii, xxix, 283, 296, 331, 504, 512, 514–517, 524, 526 Solovay, Robert Martin, 340 Sommese, Andrew J., 529 Spielman, Daniel Alan, 505, 511, 522 Steiner, Jakob, 518 Stewart, G.W (Pete), 504, 512 Stiefel, Eduard, 508 Stolzenberg, Gabriel, 491, 517 Strassen, Volker, 340, 513 Sun, Ji-Guang, 504, 512 N Nemirovsky, Arkady, 509, 510 Nesterov, Yury, 510 Newton, Sir Isaac, 512 Nijenhuis, Albert, 519 T Tao, Terence, 506 Teng, Shang-Hua, 505, 511, 522 Trefethen, Lloyd Nicholas, 504, 506, 523 Turing, Alan Mathison, vii, xxviii O Ocneanu, Adrian, 518 Oettli, Werner, 504 V van Loan, Charles Francis, 504 Vavasis, Stephen A., 509, 510 Vempala, Santosh S., 508 Vera, Jorge, 509 Vershynin, Roman, 507 Viswanath, Divakar, 506 von Neumann, John, vii, xxviii, xxix, 509 Vorobjov, Nikolai Nikolaevich, 514 Vu, Van, 506 L Lagrange, Joseph-Louis, xvii Lahaye, Edmond Léon, 512 Linnainmaa, Seppo, 513 Lotz, Martin, 506 Lovász, László, 509 P Pardo, Luis Miguel, 331, 512, 515, 517 Peña, Javier Francisco, 505, 508, 510 Plato, viii Poincaré, Jules Henri, 518 Prager, William, 504 Priouret, Pierre, 513 R Rabin, Michael Oser, 516 Renegar, James, xxix, 124, 383, 503, 508, 510, 511, 516, 517 Rice, John, 503, 512 Rohn, Jiˇrí, 504 Rojas, Joseph Maurice, 529 Rota, Gian-Carlo, 519 Rudelson, Mark, 507 Rump, Siegfried M., 508 S Sankar, Arvind, 507, 522 Santaló, Ls Antoni, 518, 519 Schmidt, Erhard, 503 Schneider, Rolf, 519 CuuDuongThanCong.com W Wedin, Per-Åke, 19, 504 Weil, Wolfgang, 519 Wendel, James G., 511 Weyl, Hermann Klaus Hugo, 503, 512, 518 Wigner, Eugene Paul, 506 Wilkinson, James Hardy, 59, 503, 512, 518, 521 Wishart, John, 506 Wongkew, Richard, 518 Wo´zniakowski, Henryk, 512 Wschebor, Mario, 58, 505, 517 Y Ye, Yinyu, 509, 510 Young, Gale, 503 Yudin, David, 509 ... advances was the invention of calculus And underlying the latter, the field of real numbers The dawn of the digital computer, in the decade of the 1940s, allowed the execution of these procedures... Overture: On the Condition of Numerical Problems Hence, | logβ mach | represents in both cases the precision of the data We therefore define the loss of precision in the computation of ϕ(a) to... exposition of the recent advances in and partial solutions to the 17th of the problems proposed by Steve Smale for the mathematicians of the 21st century, a set of results in which conditioning