[...]... Conditions 11 .1 Constraints 11 .2 Tangent Plane 11 .3 First-Order Necessary Conditions (Equality Constraints) 11 .4 Examples 11 .5 Second-Order Conditions 11 .6 Eigenvalues in Tangent Subspace 11 .7 Sensitivity 11 .8 Inequality Constraints 11 .9 Zero-Order Conditions and Lagrange Multipliers 11 .10 Summary 11 .11 Exercises Chapter 12 Primal Methods 12 .1 Advantage of Primal Methods 12 .2 Feasible Direction Methods 12 .3... Chapter 10 Quasi-Newton Methods 10 .1 Modified Newton Method 10 .2 Construction of the Inverse 10 .3 Davidon–Fletcher–Powell Method 10 .4 The Broyden Family 10 .5 Convergence Properties 10 .6 Scaling 10 .7 Memoryless Quasi-Newton Methods ∗ 10 .8 Combination of Steepest Descent and Newton’s Method 10 .9 Summary 10 .10 Exercises PART III 285 285 288 290 293 296 299 304 306 312 313 Constrained Minimization Chapter 11 ... 14 .6 The Dual Viewpoint 14 .7 Cutting Plane Methods 14 .8 Kelley’s Convex Cutting Plane Algorithm 14 .9 Modifications 14 .10 Exercises Chapter 15 Primal-Dual Methods 15 .1 The Standard Problem 15 .2 Strategies 15 .3 A Simple Merit Function 15 .4 Basic Primal–Dual Methods 15 .5 Modified Newton Methods 15 .6 Descent Properties 15 .7 Rate of Convergence 15 .8 Interior Point Methods 15 .9 Semidefinite Programming 15 .10 ... Penalty and Barrier Functions 13 .4 Newton’s Method and Penalty Functions 13 .5 Conjugate Gradients and Penalty Methods 13 .6 Normalization of Penalty Functions 13 .7 Penalty Functions and Gradient Projection 13 .8 Exact Penalty Functions 13 .9 Summary 13 .10 Exercises Chapter 14 Dual and Cutting Plane Methods 14 .1 Global Duality 14 .2 Local Duality 14 .3 Dual Canonical Convergence Rate 14 .4 Separable Problems 14 .5... examples of areas in which linear programming problems arise naturally, we indicate how various other forms of linear programs can be converted to the standard form Example 1 (Slack variables) Consider the problem minimize c1 x1 + c2 x2 + · · · + cn xn subject to a 11 x1 + a12 x2 + · · · + a1n xn a 21 x1 + a22 x2 + · · · + a2n xn · · · am1 x1 + am2 x2 + · · · + amn xn x 1 0 x2 0 xn 0 and b1 b2 · · · bm In this... as shown below, any linear program can be transformed into the following standard form: minimize subject to and c1 x1 + c2 x2 + + cn xn a 11 x1 + a12 x2 + + a1n xn = b1 a 21 x1 + a22 x2 + + a2n xn = b2 · · · · · · am1 x1 + am2 x2 + · · · + amn xn = bm x1 0 x2 0 xn (1) 0 where the bi ’s, ci ’s and aij ’s are fixed real constants, and the xi ’s are real numbers to be determined We always assume that each... bm In this case the constraint set is determined entirely by linear inequalities The problem may be alternatively expressed as minimize c1 x1 + c2 x2 + · · · + cn xn subject to and a 11 x1 + a12 x2 + · · · + a1n xn + y1 = b1 a 21 x1 + a22 x2 + · · · + a2n xn + y2 = b2 · · · · · · am1 x1 + am2 x2 + · · · + amn xn + ym = b m x 1 0 x2 0 xn 0 and y1 0 y2 0 ym 0 The new positive variables yi introduced to convert... Set Methods 3 21 3 21 323 326 327 333 335 339 3 41 346 353 354 359 359 360 363 xii Contents 12 .4 The Gradient Projection Method 12 .5 Convergence Rate of the Gradient Projection Method 12 .6 The Reduced Gradient Method 12 .7 Convergence Rate of the Reduced Gradient Method 12 .8 Variations 12 .9 Summary 12 .10 Exercises Chapter 13 Penalty and Barrier Methods 13 .1 Penalty Methods 13 .2 Barrier Methods 13 .3 Properties... suppose in (1) , for example, that the restriction x1 0 is not present and hence x1 is free to take on either positive or negative values We then write x1 = u1 − 1 (3) where we require u1 0 and 1 0 If we substitute u1 − 1 for x1 everywhere in (1) , the linearity of the constraints is preserved and all variables are now required to be nonnegative The problem is then expressed in terms of the n + 1 variables... Semidefinite Programming 15 .10 Summary 15 .11 Exercises Appendix A Mathematical Review A .1 A.2 A.3 Sets Matrix Notation Spaces 367 374 382 387 394 396 396 4 01 402 405 407 416 418 420 4 21 425 429 430 435 435 4 41 446 447 4 51 456 460 463 465 466 469 469 4 71 472 474 479 4 81 485 487 4 91 498 499 507 507 508 509 Contents A.4 A.5 A.6 Appendix B Convex Sets B .1 B.2 B.3 B.4 Appendix C Eigenvalues and Quadratic Forms . Conditions 333 11 .6. Eigenvalues in Tangent Subspace 335 11 .7. Sensitivity 339 11 .8. Inequality Constraints 3 41 11. 9. Zero-Order Conditions and Lagrange Multipliers 346 11 .10 . Summary 353 11 .11 . Exercises. Minimization Chapter 11 . Constrained Minimization Conditions 3 21 11. 1. Constraints 3 21 11. 2. Tangent Plane 323 11 .3. First-Order Necessary Conditions (Equality Constraints) 326 11 .4. Examples 327 11 .5. Second-Order. of nonlinear programming. Linear Programming Linear programming is without doubt the most natural mechanism for formulating a vast array of problems with modest effort. A linear programming problem