numerical optimization - j. nocedal, s. wright

Numerical Optimization Jorge Nocedal Stephen J. Wright Springer Springer Series in Operations Research Editors: Peter Glynn Stephen M. Robinson Springer New York Berlin Heidelberg Barcelona Hong Kong London Milan Paris Singapore Tokyo [...]... Transportation Problem Continuous versus Discrete Optimization Constrained and Unconstrained Optimization Global and Local Optimization Stochastic and Deterministic Optimization Optimization Algorithms Convexity Notes and References vii xxi 2 4 4 6 6 7 7 8 9 Fundamentals of Unconstrained Optimization 2.1 What Is a Solution? ... 192 194 199 200 202 205 207 209 211 211 214 218 219 220 Large-Scale Quasi-Newton and Partially Separable Optimization 9.1 Limited-Memory BFGS Relationship with Conjugate Gradient Methods 9.2 General Limited-Memory Updating 222 224 227 229 Contents Compact Representation... xvii xviii Contents 16.3 Inequality-Constrained Problems Optimality Conditions for Inequality-Constrained Problems Degeneracy 16.4 Active-Set Methods for Convex QP Specification of the Active-Set Method for Convex QP An Example Further Remarks on the Active-Set Method Finite Termination of the Convex... Full Quasi-Newton Approximations Hessian of Augmented Lagrangian Reduced-Hessian Approximations 18.5 Merit Functions and Descent 18.6 A Line Search SQP Method 18.7 Reduced-Hessian SQP Methods Some Properties of Reduced-Hessian Methods Update Criteria for Reduced-Hessian Updating ... requirements, and between robustness and speed, and so on, are central issues in numerical optimization They receive careful consideration in this book The mathematical theory of optimization is used both to characterize optimal points and to provide the basis for most algorithms It is not possible to have a good understanding of numerical optimization without a firm grasp of the supporting theory Accordingly,... 418 420 422 424 426 429 431 432 436 436 16 Quadratic Programming An Example: Portfolio Optimization 16.1 Equality–Constrained Quadratic Programs Properties of Equality-Constrained QPs 16.2 Solving the KKT System Direct Solution of the KKT System Range-Space Method Null-Space Method A Method Based on Conjugacy 438... Modified Cholesky Factorization Gershgorin Modification Modified Symmetric Indefinite Factorization 6.4 Trust-Region Newton Methods Newton–Dogleg and Subspace-Minimization Methods Accurate Solution of the Trust-Region Problem Trust-Region Newton–CG Method ... Modeling, Regression, Statistics Linear Least-Squares Problems 10.2 Algorithms for Nonlinear Least-Squares Problems The Gauss–Newton Method The Levenberg–Marquardt Method Implementation of the Levenberg–Marquardt Method Large-Residual Problems Large-Scale Problems 10.3 Orthogonal Distance Regression ... Practical Reduced-Hessian Method 18.8 Trust-Region SQP Methods Approach I: Shifting the Constraints Approach II: Two Elliptical Constraints Approach III: S 1 QP (Sequential 1 Quadratic Programming) 18.9 A Practical Trust-Region SQP Algorithm 18.10 Rate of Convergence Convergence Rate of Reduced-Hessian Methods... solutions that are not global solutions In this book we treat global optimization only in passing, focusing instead on the computation and characterization of local solutions, issues that are central to the field of optimization We note, however, that many successful global optimization algorithms proceed by solving a sequence of local optimization problems, to which the algorithms described in this book . SQP Methods 546 Some Properties of Reduced-Hessian Methods 547 Update Criteria for Reduced-Hessian Updating 548 Changes of Bases 549 A Practical Reduced-Hessian Method 550 18.8 Trust-Region SQP. proficiency. EMPHASIS AND WRITING STYLE We have used a conversational style to motivate the ideas and present the numerical algorithms. Rather than being as concise as possible, our aim is to make the discussion. 250 10.1 Background 253 Modeling, Regression, Statistics 253 Linear Least-Squares Problems 256 10.2 Algorithms for Nonlinear Least-Squares Problems 259 The Gauss–Newton Method 259 The Levenberg–Marquardt