Nonlinear programming

Nonlinear Programming SIAM's Classics in Applied Mathematics series consists of books that were previously allowed to go out of print These books are republished by SIAM as a professional service because they continue to be important resources for mathematical scientists Editor-in-Chief Robert E O'Malley, Jr., University of Washington Editorial Board Richard A Brualdi, University of Wisconsin-Madison Herbert B Keller, California Institute of Technology Andrzej Z Manitius, George Mason University Ingram Olkin, Stanford University Stanley Richardson, University of Edinburgh Ferdinand Verhulst, Mathematisch Instituut, University of Utrecht Classics in Applied Mathematics C C Lin and L A Segel, Mathematics Applied to Deterministic Problems in the Natural Sciences Johan G F Belinfante and Bernard Kolman, A Survey of Lie Groups and Lie Algebras with Applications and Computational Methods James M Ortega, Numerical Analysis: A Second Course Anthony V Fiacco and Garth P McCormick, Nonlinear Programming: Sequential Unconstrained Minimization Techniques F H Clarke, Optimization and Nonsmooth Analysis George F Carrier and Carl E Pearson, Ordinary Differential Equations Leo Breiman, Probability R Bellman and G M Wing, An Introduction to Invariant Imbedding Abraham Berman and Robert J Plemmons, Nonnegative Matrices in the Mathematical Sciences Olvi L Mangasarian, Nonlinear Programming *Carl Friedrich Gauss, Theory of the Combination of Observations Least Subject to Errors: Part One, Part Two, Supplement Translated by G W Stewart Richard Bellman, Introduction to Matrix Analysis U M Ascher, R M M Mattheij, and R D Russell, Numerical Solution of Boundary Value Problems for Ordinary Differential Equations K E Brenan, S L Campbell, and L R Petzold, Numerical Solution of InitialValue Problems in Differential-Algebraic Equations Charles L Lawson and Richard J Hanson, Solving Least Squares Problems J E Dennis, Jr and Robert B Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations Richard E Barlow and Frank Proschan, Mathematical Theory of Reliability *First time in print Classics in Applied Mathematics (continued) Cornelius Lanczos, Linear Differential Operators Richard Bellman, Introduction to Matrix Analysis, Second Edition Beresford N Parlett, The Symmetric Eigenvalue Problem Richard Haberman, Mathematical Models: Mechanical Vibrations, Population Dynamics, and Traffic Flow Peter W M John, Statistical Design and Analysis of Experiments Tamer Bajar and Geert Jan Olsder, Dynamic Noncooperative Game Theory, Second Edition Emanuel Parzen, Stochastic Processes Petar Kokotovic, Hassan K Khalil, and John O'Reilly, Singular Perturbation Methods in Control: Analysis and Design Jean Dickinson Gibbons, Ingram Olkin, and Milton Sobel, Selecting and Ordering Populations: A New Statistical Methodology James A Murdock, Perturbations: Theory and Methods Ivar Ekeland and Roger Témam, Convex Analysis and Variational Problems Ivar Stakgold, Boundary Value Problems of Mathematical Physics, Volumes I and II J M Ortega and W C Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables David Kinderlehrer and Guido Stampacchia, An Introduction to Variational Inequalities and Their Applications F Natterer, The Mathematics of Computerized Tomography Avinash C Kak and Malcolm Slaney, Principles of Computerized Tomographic Imaging R Wong, Asymptotic Approximations of Integrals O Axelsson and V A Barker, Finite Element Solution of Boundary Value Problems: Theory and Computation David R Brillinger, Time Series: Data Analysis and Theory Joel N Franklin, Methods of Mathematical Economics: Linear and Nonlinear Programming, Fixed-Point Theorems Philip Hartman, Ordinary Differential Equations, Second Edition Michael D Intriligator, Mathematical Optimization and Economic Theory Philippe G Ciarlet, The Finite Element Method for Elliptic Problems This page intentionally left blank Nonlinear Programming Otvi L Mangasarian University of Wisconsin Madison, Wisconsin siajTL Society for Industrial and Applied Mathematics Philadelphia Library of Congress Cataloging-in-Publication Data Mangasarian, Olvi L., 1934Nonlinear programming / Olvi L Mangasarian p cm (Classics in applied mathematics ; 10) Originally published: New York : McGraw-Hill, 1969, in series: McGraw-Hill series in systems science Includes bibliographical references and indexes ISBN 0-89871-341-2 Nonlinear programming I Title II Series T57.8.M34 1994 519.7'6-dc20 94-36844 109876543 All rights reserved Printed in the United States of America No part of this book may be reproduced, stored, or transmitted in any manner without the written permission of the Publisher For information, write the Society for Industrial and Applied Mathematics, 3600 University City Science Center, Philadelphia, PA 19104-2688 Copyright © 1994 by the Society for Industrial and Applied Mathematics This SIAM edition is a corrected republication of the work first published in 1969 by the McGraw-Hill Book Company, New York, New York Siam is a registered trademark To Josephine Mangasarian, my mother, and to Claire This page intentionally left blank Preface to the Classics Edition Twenty-five years have passed since the original edition of this book appeared; however, the topics covered are still timely and currently taught at the University of Wisconsin as well as many other major institutions At Wisconsin these topics are taught in a course jointly listed by the Computer Sciences, Industrial Engineering, and Statistics departments Students from these and other disciplines regularly take this course Each year I get a number of requests from the United States and abroad for copies of the book and for permission to reproduce reserve copies for libraries I was therefore pleased when SIAM approached me with a proposal to reprint the book in its Classics series I believe that this book is an appropriate choice for this series inasmuch as it is a concise, 'igorous, yet accessible account o ' the fundamentals of constrained optimization theory that is useful to both the beginning student as well as the active researcher I am appreciative that SIAM has chosen to publish the book and to make the corrections that I supplied I am especially grateful to Vickie Kearn and Ed Block for their friendly and professional handling of the publication process My hope is that the mathematical programming community will benefit from this endeavor Olvi L Mangasarian ix Nonlinear Programming Charnes, A., and W W Cooper: "Management Models and Industrial Applications of Linear Programming," vols I, II, John Wiley & Sons, Inc., New York, 1961 Cottle, R W.: Symmetric Dual Quadratic Programs, Quarterly of Applied Mathematics 21: 237-243 (1963) Courant, R.: "Differential and Integral Calculus," vol II, 2d ed., rev., Interscience Publishers, New York, 1947 Courant, R., and D Hilbert: "Methods of Mathematical Physics," pp 231-242, Interscience Publishers, New York, 1953 Dantzig, G B.: "Linear Programming and Extensions," Princeton University Press, Princeton, N.J., 1963 Dantzig, G B., E Eisenberg, and R W Cottle: Symmetric Dual Nonlinear Programs, Pacific Journal of Mathematics, 15: 809-812 (1965) Dennis, J B.: "Mathematical Programming and Electrical Networks," John Wiley & Sons, Inc., New York, 1959 Dieter, U.: Dualitat bei konvexen Optimierungs—(Programmierungs—)Aufgaben, Unternehmensforschung 9: 91-111 (1965a) Dieter, U.: Dual External Problems in Locally Convex Linear Spaces, Proceedings of the Colloquium on Convexity, Copenhagen, 52-57 (1965b) Dieter, U.: Optimierungsaufgaben in topologischen Vektorraumen I: Dualitatstheorie, Zeitschrift fur \Vahrscheinlichkeitstheorie und Verwandte Gebiete, 5: 89-117 (1966) Dorn, W S.: Duality in Quadratic Programming, Quarterly of Applied Mathematics, 18: 155-162 (1960) Dorn, W S.: Self-dual Quadratic Programs, Society for Industrial and Applied Mathematics Journal on Applied Mathematics, 9: 51-54 (1961) Duffin, R J.: Infinite Programs, in [Kuhn-Tucker 56], pp 157-170 Fan, K., I Glicksburg, and A J Hoffman: Systems of Inequalities Involving Convex Functions, American Mathematical Society Proceedings, 8: 617-622 (1957) Farkas, J.: Uber die Theorie der einfachen Ungleichungen, Journal fur die Heine und Angewandte Mathematik, 124: 1-24 (1902) Fenchel, W.: "Convex Cones, Sets and Functions," Lecture notes, Princeton University, 1953, Armed Services Technical Information Agency, AD Number 22695 Fiacco, A V.: Second Order Sufficient Conditions for Weak and Strict Constrained Minima, Society for Industrial and Applied Mathematics Journal on Applied Mathematics, 16: 105-108 (1968) 106 References Fiacco, A V., and G P McCormick: "Nonlinear Programming: Sequential Unconstrained Minimization Techniques," John Wiley &Sons, Inc., New York, 1968 Fleming, W H.: "Functions of Several Variables," McGraw-Hill Book Company, New York, 1965 Friedrichs, K 0.: Ein Verfahren der Variationsrechnung des Minimum eines Integral als das Maximum eines Anderen Ausdruckes Darzustellen, Nachrichten von der Gesellschajl der Wissenschaften zu Gottingen Mathematische—Physikalische Klasse, 1320(1929) Gale, D.: "The Theory of Linear Economic Models," McGraw-Hill Book Company, New York, 1960 Gale, D., H W Kuhn, and A W Tucker: Linear Programming and the Theory of Games, in [Koopmans 51], pp 317-329 Gass, S.: "Linear Programming," 2d ed., McGraw-Hill Book Company, New York, 1964 Gol'stein, E G.: Dual Problems of Convex and Fractionally-convex Programming in Functional Spaces, Soviet Mathematics-Doklady (English translation), 8: 212-216 (1967) Gordan, P.: Uber die Auflosungen linearer Gleichungen mit reelen Coefficienten, Mathematische Annalen, 6: 23-28 (1873) Graves, R L., and P Wolfe (ed.): "Recent Advances in Mathematical Programming," McGraw-Hill Book Company, New York, 1963 Hadley, G F.: "Linear Programming," Addison-Wesley Publishing Company, Inc., Mass., 1962 Hadley, G F.: "Nonlinear and Dynamic Programming," Addison-Wesley Publishing Company, Inc., Reading, Mass., 1964 Halkin, H.: A Maximum Principle of the Pontryagin Type for Systems Described by Nonlinear Difference Equations, Society for Industrial and Applied Mathematics Journal on Control, 4: 90-111 (1966) Halkin, H., and L W Neustadt: General Necessary Conditions for Optimization Problems, Proc Nat Acad Sci U.S., 56: 1066-1071 (1966) Halmos, P R.: "Finite-dimensional Vector Spaces," D Van Nostrand Company, Inc., Princeton, N.J., 1958 Hamilton, N T., and J Landin: "Set Theory: The Structure of Arithmetic," Allyn and Bacon, Inc., Boston, 1961 Hanson, M A.: A Duality Theorem in Nonlinear Programming with Nonlinear Constraints, Australian Journal of Statistics 3: 64-71 (1961) 207 Nonlinear Programming: Hanson, M A.: Bounds for Functionally Convex Optimal Control Problems, Journal of Mathematical Analysis and Applications, 8: 84-89 (1964) Hanson, M A.: Duality for a Class of Infinite Programming Problems, Society for Industrial and Applied Mathematics Journal on Applied Mathematics, 16: 318-323 (1968) Hanson, M A., and B Mond: Quadratic Programming in Complex Space, Journal of Mathematical Analysis and Applications, 23: 507-514 (1967) Hestenes, M R.: "Calculus of Variations and Optimal Control Theory," John Wiley & Sons, Inc., New York, 1966 Hu, T C.: "Integer Programming and Network Flows," Addison-Wesley Publishing Company, Inc., Reading, Mass., 1969 Huard, P.: Dual Programs, 75.17 J Res Develop., 6: 137-139 (1962) Hurwicz, L.: Programming in Linear Spaces, in [Arrow et al 58], pp 38-102 Jacob, J.-P., and P Rossi: "General Duality in Mathematical Programming," IBM Research Report, 1969 Jensen, J L W V.: Sur les fonctions convexes et les ine'galite's entre les valeurs moyennes, Acta Mathematica, 30: 175-193 (1906) John, F.: Extremum Problems with Inequalities as Subsidiary Conditions, in K Friedrichs, O E Neugebauer, and J J Stoker, (eds.), "Studies and Essays: Courant Anniversary Volume," pp 187-204, Interscience Publishers, New York, 1948 Karamardian, S.: "Duality in Mathematical Programming," Operations Research Center, University of California, Berkeley, Report Number 66-2, 1966; Journal of Mathematical Analysis and Applications, 20: 344-358 (1967) Karlin, S.: "Mathematical Methods and Theory in Games, Programming, and Economics," vols I, II, Addison-Wesley Publishing Company, Inc., Reading, Mass., 1959 Koopman, T J (ed.): "Activity Analysis of Production and Allocation," John Wiley & Sons, Inc., New York, 1951 Kowalik, J., and M R Osborne: "Methods for Unconstrained Optimization Problems," American Elsevier Publishing Company, Inc., New York, 1968 Kretschmar, K S.: Programmes in Paired Spaces, Canadian Journal of Mathematics, 13: 221-238 (1961) Kuhn, H W., and A W Tucker: Nonlinear Programming, in J Neyman (ed.), "Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability," pp 481-492, University of California Press, Berkeley, Calif., 1951 908 Beferencea Kuhn, H W., and A W Tucker: "Linear Inequalities and Related Systems," Annals of Mathematics Studies Number 38, Princeton University Press, Princeton, N.J., 1956 Ktinzi, H P., W Krelle, and W Oettli: "Nonlinear Programming," Blaisdell Publishing Company, Waltham, Mass., 1966 Larsen, A., and E Polak: Some Sufficient Conditions for Continuous-linearprogramming Problems," International Journal of Engineering Science, 4: 583604 (1966) Lax, P D.: Reciprocal Extremal Problems in Function Theory, Communications on Pure and Applied Mathematics, 8: 437-453 (1955) Levinson, N.: Linear Programming in Complex Space, Journal of Mathematical Analysis and Applications, 14: 44-62 (1966) Levitin, E S., and B T Polyak: Constrained Minimization Methods, USSR Computational Mathematics and Mathematical Physics (English translation), 6(5):1-50 (1966) McCormick, G P.: Second Order Conditions for Constrained Minima, Society for Industrial and Applied Mathematics Journal on Applied Mathematics, 16: 641-652 (1967) Mangasarian, L.: Duality in Nonlinear Programming, Quarterly of Applied Mathematics, 20: 300-302 (1962) Mangasarian, L.: Pseudo-convex Functions, Society for Industrial and Applied Mathematics Journal on Control, 3: 281-290 (1965) Mangasarian, O L., and S Fromovitz: A Maximum Principle in Mathematical Programming, in A V Balakrishnan and L W Neustadt (eds.), "Mathematical Theory of Control," pp 85-95, Academic Press Inc., New York, 1967 Mangasarian, L., and S Fromovitz: The Fritz John Necessary Optimality Conditions in the Presence of Equality and Inequality Constraints, J Math Analysis and Applications, 17:37-47 (1967a) Mangasarian, L., and J Ponstein: Minmax and Duality in Nonlinear Programming, Journal of Mathematical Analysis and Applications, 11: 504-518, (1965) Martos, B.: The Direct Power of Adjacent Vertex Programming Methods, Management Science, 12: 241-252 (1965) Minty, G J.: On the Monotonicity of the Gradient of a Convex Function, Pacific Journal of Mathematics, 14: 243-247 (1964) Mond, B.: A Symmetric Dual Theorem for Nonlinear Programs, Quarterly of Applied Mathematics, 23: 265-269 (1965) S09 Nonlinear Programming Mond, B., and R W Cottle: Self-duality in Mathematical Programming, Society for Industrial and Applied Mathematics Journal on Applied Mathematics, 14: 420-423 (1966) Mond, B., and M A Hanson: Duality for Variational Problems, Journal of Mathematical Analysis and Applications, 18: 355-364 (1967) Moreau, J.-J.: Proximite* et dualite* dans un espace Hilbertien, Bulletin de la Societe Mathematique de France, 93: 273-299 (1965) Motzkin, T S.: "Beitrage zur Theorie der Linearen Ungleichungen," Inaugural Dissertation, Basel, Jerusalem, 1936 Nikaido, H.: On von Neumann's Minimax Theorem, Pacific Journal of Mathematics, 4: 65-72 (1954) Opial, Z.: "Nonexpansive and Monotone Mappings in Banach Spaces," Lecture Notes 67-1, Division of Applied Mathematics, Brown University, Providence, Rhode Island, 1967 Ponstein, J.: Seven Types of Convexity, Society for Industrial and Applied Mathematics Review, 9: 115-119 (1967) Pontryagin, L S., V G Boltyanskii, R V Gamkrelidze, and E F Mishchenko: "The Mathematical Theory of Optimal Processes," John Wiley & Sons, Inc., New York, 1962 Rissanen, J.: On Duality Without Convexity, Journal of Mathematical Analysis and Applications, 18: 269-275 (1967) Ritter, K.: Duality for Nonlinear Programming in a Banach Space, Society for Industrial and Applied Mathematics Journal on Applied Mathematics, 15: 294-302 (1967) Rockafellar, R T.: "Convex Functions and Dual Extremum Problems," Doctoral dissertation, Harvard University, 1963 Rockafellar, R T.: Duality Theorems for Convex Functions, Bulletin of the American Mathematical Society, 70: 189-192 (1964) Rockafellar, R T.: Conjugates and Legendre Transforms of Convex Functions, Canadian Journal of Mathematics, 19: 200-205 (1967a) Rockafellar, R T.: Convex Programming and Systems of Elementary Monotonic Relations, Journal of Mathematical Analysis and Applications, 19: 543-564 (1967b) Rockafellar, R T.: Duality and Stability in Extremum Problems Involving Convex Functions, Pacific Journal of Mathematics, 21: 167-187 (1967c) Rockafellar, R T.: "Convex Analysis," Princeton University Press, Princeton, N.J., 1969 210 References Rosen, J B.: The Gradient Projection Method for Nonlinear Programming, Society for Industrial and Applied Mathematics Journal on Applied Mathematics, 8: 181-217 (1960) and 9: 514-553 (1961) Rubinstein, G S.: Dual Extremum Problems, Soviet Mathematics-Doklady (English translation), 4: 1309-1312 (1963) Rudin, W.: "Principles of Mathematical Analysis," 2d ed., McGraw-Hill Book Company, New York, 1964 Saaty, T L., and J Bram: "Nonlinear Mathematics," McGraw-Hill Book Company, New York, 1964 Simmonard, M.: "Linear Programming," Prentice-Hall, Inc., Englewood Cliffs, N.J., 1966 Simmons, G F.: "Introduction to Topology and Modern Analysis," McGrawHill Book Company, New York, 1963 Slater, M.: "Lagrange Multipliers Revisited: A Contribution to Nonlinear Programming," Cowles Commission Discussion Paper, Mathematics 403, November, 1950 Slater, M L.: A Note on Motzkin's Transposition Theorem, Econometrica, 19: 185-186 (1951) Stiemke, E.: t)ber positive Losungen homogener linearer Gleichungen, Mathematische Annalen, 76: 340-342 (1915) Stoer, J.: Duality in Nonlinear Programming and the Minimax Theorem, Numerische Mathematik, 5: 371-379 (1963) Stoer, J.: tlber einen Dualitatssatz der nichtlinearen Programmierung, Numerische Mathematik, 6: 55-58 (1964) Trefftz, E.: "Ein Gegenstiick zum Ritzschen Verfahren," Verhandlungen des Zweiten Internationalen Kongresses fur Technische Mechanik, p 131, Zurich, 1927 Trefftz, E.: Konvergenz und Fehlerschatzung beim Ritzschen Verfahren, Mathematische Annalen, 100: 503-521 (1928) Tucker, A W.: Dual Systems of Homogeneous Linear Relations, in [KuhnTucker 56], pp 3-18 Tuy, Hoang: Sur les ine'galite's line*aires, Colloquium Mathematicum, 13: 107-123 (1964) Tyndall, W F.: A Duality Theorem for a Class of Continuous Linear Programming Problems, Society for Industrial and Applied Mathematics Journal on Applied Mathematics, 13: 644-666 (1965) 111 Nonlinear Programming Tyndall, W F.: An Extended Duality Theorem for Continuous Linear Programming Problems, Notices of the American Mathematical Society, 14: 152-153 (1967) Uzawa, H.: The Kuhn-Tucker Theorem in Concave Programming, in [Arrow et al 58], pp 32-37 Vajda, S.: "Mathematical Programming," Addison-Wesley Publishing Company, Inc., Reading, Mass., 1961 Valentine, F A.: "Convex Sets," McGraw-Hill Book Company, New York, 1964 Van Slyke, R M., and R J B Wets: "A Duality Theory for Abstract Mathematical Programs with Applications to Optimal Control Theory," Mathematical Note Number 538, Mathematics Research Laboratory, Boeing Scientific Research Laboratories, October, 1967 Varaiya, P P.: Nonlinear Programming in Banach Space, Society for Industrial and Applied Mathematics Journal on Applied Mathematics, 16: 284-293 (1967) von Neumann, J.: Zur Theorie der Gesellschaftsspiele, Mathematische Annalen, 100: 295-320(1928) Whinston, A.: Conjugate Functions and Dual Programs, Naval Research Logistics Quarterly, 12: 315-322 (1965) Whinston, A.: Some Applications of the Conjugate Function Theory to Duality, in [Abadie 67], pp 75-96 Wolfe, P.: A Duality Theorem for Nonlinear Programming, Quarterly of Applied Mathematics, 19: 239-244 (1961) Zangwill, W I.: "Nonlinear Programming: A Unified Approach," Prentice-Hall, Inc., Englewood Cliffs, N.J., 1969 Zarantonello, E H.: "Solving Functional Equations by Contractive Averaging," Mathematics Research Center, University of Wisconsin, Technical Summary Report Number 160, 1960 Zoutendijk, G.: "Methods of Feasible Directions," Elsevier Publishing Company, Amsterdam, 1960 Zukhovitskiy, S I., and L I Avdeyeva: "Linear and Convex Programming," W B Saunders Company, Philadelphia, 1966 ata Indexes This page intentionally left blank Name Index Abadie, J., 97, 99, 100, 205 Almgren, F J., 62 Anderson, K W., 3, 205 Arrow, K J., 102, 103, 151, 205 Avdeyeva, L I., 212 Fiacco, A V., 206, 207 Fleming, W H., 2, 62, 71, 182, 188, 200, 207 Friedrichs, K 0., 113, 207 Fromovitz, S., 72, 163, 170, 172, 209 Bartle, R G., 200, 205 Berge, C., 3, 46, 55, 68, 122, 132, 182 189, 191, 205 Birkhoff, G., 187, 205 Bohnenblust, H F., 67, 205 Boltyanskii, V G., 72, 163, 210 Bracken, J., 205 Bram, J., 211 Brondsted, A., 122, 205 Browder, F E., 87, 205 Buck, R C., 182, 188, 205 Gale, D., 16, 33, 35, 113, 127, 177, 179, 207 Gamkrelidze, R V., 72, 163, 210 Gass, S., 15, 207 Ghouila Houri, A., 55, 68, 122, 132, 189, 191, 205 Glicksburg, I., 63, 65, 206 Gol'stein, S., 122, 207 Gordan, P., 31, 207 Graves, R L., 207 Canon, M., 72, 163, 170, 205 Carathe"odory, C., 43, 205 Charnes, A., 206 Cooper, W W., 206 Cottle, R W., 122, 206, 210 Courant, R., 2, 113, 206 Cullum, C., 72, 163, 170, 205 Daniel, J W., 130 Dantzig, G B., 15, 18, 113, 122, 206 Dennis, J B., 113, 206 Dieter, U., 122, 206 Dorn, W S., 117, 122, 124, 125, 206 Duffin R J 32, 206 Eisenberg, E., 122, 206 Enthoven, A C., 151, 205 Fan, K., 63, 65, 206 Farkas, J., 16, 31, 206 Fenchel, W., 55, 122, 206 Hadley, G F., 15, 207 Halkin, H., 72, 163, 170, 207 Hall, D W., 3, 205 Halmos, P R., 177, 207 Hamilton, N T., 3, 207 Hanson, M A., 113, 117, 118, 122, 137, 207, 208, 210 Hestenes, M R., 200, 208 Hilbert, D., 113, 206 Hoffman, A J., 63, 65, 206 Hu, T C., 208 Huard, P., 115, 117, 118, 122, 208 Hurwicz, L., 102, 103, 122, 205, 208 Jacob, J.-P., 122, 208 Jensen, J L W V., 61, 208 John, F., 77, 99, 208 Karamardian, S., 87, 117, 122, 137, 139, 208 Karlin, S., 67, 77-79, 205, 208 Koopmans, T J., 208 Kowalik, J., 208 Krelle, W., 209 Nonlinear Programming Kretschmar, K S., 122, 208 Kuhn, H W., 79, 94, 102, 105, 113, 127, 207-209 Kiinzi, H P., 209 Rosen, J B., 211 Rossi, P., 122, 208 Rubinstein, G S., 122, 211 Rudin, W., 182, 188, 200, 211 Landin, J., 3, 207 Larsen, A., 122, 209 Lax, P D., 113, 209 Levinson, N., 122, 209 Levitin, E S., 209 Saaty, T L., 211 Shapley, L S., 67, 205 Simmonard, M., 15, 113, 211 Simmons, G F., 182, 211 Slater, M L., 27, 78, 211 Stiemke, E., 32, 211 Stoer, J., 122, 211 McCormick, G P., 205, 207, 209 MacLane, S., 187, 205 Mangasarian, L., 72, 117, 118, 122, 131, 140, 151, 157, 163, 170, 172, 209 Martos, B., 132, 137, 151, 209 Minty, G J., 87, 209 Mishchenko, E F., 72, 163, 210 Mond, B., 113, 122, 208-210 Moreau, J.-J., 122, 210 Motzkin, T S., 28, 210 Neustadt, L W., 163, 207 Nikaid6, H., 131, 132, 210 Oettli, W., 209 Opial, Z., 87, 210 Osborne, M R., 208 Polak, E., 72, 163, 170, 205, 209 Polyak, B T., 209 Ponstein, J., 117, 118, 122, 209, 210 Pontryagin, L S., 72, 163, 210 Rissanen, J., 122, 210 Ritter, K., 122, 210 Rockafellar, R T., 87, 122, 210 216 Trefftz, E., 113, 211 Tucker, A W., 16, 20, 21, 24, 25, 28, 29, 79, 94, 102, 105, 113, 127, 207209, 211 Tuy, Hoang, 131, 140, 151, 211 Tyndall, W F., 122,211, 212 Uzawa, H., 77, 79, 80, 102, 103, 205, 212 Vajda, S., 212 Valentine, F A., 46, 55, 212 Van Slyke, R M., 122, 212 Varaiya, P P., 122, 212 von Neuman, J., 113, 212 Wets, R J B., 122, 212 Whinston, A., 122, 212 Wolfe, P., 114, 115, 121, 122, 207, 212 Zangwill, W L, 212 Zarantonello, E H., 87, 212 Zoutendijk, G., Ill, 212 Zukhovitskiy, S L, 212 Subject Index Alternative: table of theorems of, 34 theorems of, 27-37 Angle between two vectors, Axiom of real number system, 188 Ball: closed, 183 open, 182 Basis, 178 Bi-nonlinear function, 149 Bolzano-Weierstrass theorem, 189 Bounded function, 196 Bounded set, 188 Bounds: greatest lower and least upper, 187 lower and upper, 187 Caratheodory's theorem, 43 Cauchy convergence, 188 Cauchy-Schwarz inequality, Closed set, 183 Closure of a set, 183 Compact set, 189 Concave function, 56 differentiable, 83-91 minimum of, 73 strictly, 57 strictly concave: and differentiable, 87-88 and twice differentiable, 90-91 twice differentiable, 88-90 Concave functions, infimum of, 61 Constraint qualification: Arrow-Hurwicz-Uzawa, 102 modified, 172 weak, 154 with convexity, 78-81 with differentiability, 102-105, 154-156, 171-172 Karlin's, 78 Kuhn-Tucker, 102, 171 Constraint qualification: reverse convex, 103 weak, 155, 172 Slater's, 78 weak, 155 strict, 79 Constraint qualifications, relationships between, 79, 103, 155 Continuous function, 191-192 Convex: and concave functions, 54-68 generalized, 131-150 and pseudoconvex functions, relation between, 144, 146 Convex combinations 41 Convex function, 55 continuity of, 62 differentiable, 83-91 at a point, 83 on a set, 84 strictly, 56 strictly convex: and differentiable, Q7 fifi oi~ oo and twice differentiable, 90-91 twice differentiable, 88-90 Convex functions: fundamental theorems for, 63-68 supremum of, 61 Convex hull, 44 Convex set, 39 Convex sets, 38-53 intersection of, 41 Differentiable function: numerical, 200 vector, 201 Distance between two points, Dual linear problem (LDP), 127, 130 Dual (maximization) problem (DP), 114, 123 Dual problem, unbounded, 119, 121, 126, 127 Dual variables, 71 Nonlinear Programming Duality, 113-130, 157-160, 174-176 in linear programming, 126-130 with nonlinear equality constraints, 174-176 in nonlinear programming, 114-123 in quadratic programming, 123-126 Duality theorem: Dorn's, 124 Dorn's strict converse, 125 Hanson-Huard strict converse, 118 linear programming, 127 strict converse, 117, 118, 124, 157, 160, 174, 176 weak, 114, 124 Wolfe's, 115 Equality constraints: linear, 80, 95 nonlinear, 75, 112, 161-176 Euclidean space Rn , Farkas' theorem, 16, 31, 37, 53 nonhomogeneous, 32 Feasible directions, method of, 111 Finite-intersection property, 189 Fractional function: linear, 149 nonlinear, 148 Fritz John saddlepoint necessary optimality theorem, 77 Fritz John saddlepoint problem (FJSP), 71 Fritz John stationary-point necessary optimality theorem, 99, 170 Fritz John stationary-point problem (FJP), 93 Function, 11 linear vector, 12 numerical, 12 vector, 12 Gale's theorems, 33 Gordan's theorem, 31 ais Gordan's theorem: generalized to convex functions, 65 Gradient of a function, 201 Halfspace, 40 Heine-Borel theorem, 189 Hessian matrix, 202 Implicit function theorem, 204 Inequalities, linear, 16-37 Infimum, 187, 195 of a numerical function, 196 Interior of a set, 183 Interior point, 182 Jacobian matrix, 202 Kuhn-Tucker saddlepoint necessary optimality theorem, 79 in the presence of equality constraints, 80 Kuhn-Tucker saddlepoint problem (KTSP), 71 Kuhn-Tucker stationary-point necessary optimality criteria, 105, 111, 112, 156, 173 Kuhn-Tucker stationary-point problem (KTP), 94 Kuhn-Tucker sufficient optimality theorem, 94, 153, 162 Lagrange multipliers, 71 Lagrangian function, 71 Limit, 186 Limit point, 186 Line, 38 segments, 39 Linear combination, nonnegative, Linear dependence, Subject Index Linear inequalities, 16-37 Linear minimization problem (LMP), 127, 130 Linear programming: duality, 126-130 optimality, 18-21 Linear systems, existence theorems, 21-26 Linearization lemma, 97, 153, 163 Local minimization problem (LMP), 70, 93 Mapping, 11 Matrices, 8-11 Matrix: nonvacuous, 11 submatrices of, 10 Maximum, 195 of a numerical function, 197 existence of, 198 Maximum principle (see Minimum principle) Mean-value theorem, 204 Metric space, Minimization problem (MP), 70, 93 local, 70, 93 solution set, 72 uniqueness, 73 Minimum, 195 of a numerical function, 197 existence of, 198 Minimum principle, 72, 162-170, 141-142 Motzkin's theorem, 28 Negative definite matrix, 91, 181 Negative semidefinite matrix, 89, 90, 181 Nonlinear programming problem 1-3, 14-15 Nonsingular matrix, 181 Norm of a vector, Notation, 13-15 Numerical function, 12 Open set, 183 Optimality criteria: with differentiability, 92-112, 151-176 necessary, 97-112, 153-157, 162-174 sufficient, 94-97, 151-153, 162 necessary saddlepoint, 76-82 saddlepoint, 69-82 sufficient saddlepoint, 74-76 Partial derivatives: of a numerical function, 200-201 of a vector function, 201-202 Plane, 40 Point of closure, 182 Polyhedron, 41 Poly tope, 41 Positive definite matrix, 91, 181 Positive semidefinite matrix, 89, 90, 181 Primal minimization problem (MP), 114, 122 Primal problem, no minimum, 121,126, 127 Product of a convex set with a real number, 45 Pseudoconcave function, 141 Pseudoconvex and strictly quasiconvex functions, relation between, 143, 146 Pseudoconvex function, 140-141 Quadratic dual problem (QDP), 124 Quadratic minimization problem (QMP), 123 Quadratic programming, 123-126 Quasiconcave function, 132 differentiable, 134 strictly, 137 Quasiconvex and strictly quasiconvex functions, relation between, 139, 146 119 Nonlinear Programming Quasiconvex function, 131 differentiable, 134 strictly, 137 Rank of a matrix: column, 179 row, 179 Relatively closed set, 183 Relatively open set, 183 Rn, Saddlepoint problem: Fritz John (FJSP), 71 Kuhn-Tucker (KTSP), 71 Scalar product, Semicontinuous function: lower, 192-193 upper, 193 Separating plane, 46 Separation theorem, 49 strict, 50 Separation theorems, 46-53 Sequence, 185 Set, complement of, element of, empty, product with a real number, 45 subset of, Set theoretic symbols, Sets: difference of, disjoint, 120 Sets: intersection of, product of, 4-5 sum of, 45 union of, Simplex, 42 Slater's theorem, 27 Stiemke's theorem, 32 Subspace, 40 Sum of two convex sets, 45 Supremum, 187, 195 of a numerical function, 196 Symbols, Taylor's theorem, 204 Triangle inequality, 8, 41 Tucker's theorem, 29 Twice differentiable numerical function, 202 Uniqueness of minimum solution, 73 Vector, addition, multiplication by a scalar, norm of, Vector function, 12 Vector space, fundamental theorem of, 177 Vectors: angle between two, scalar product of, Vertex, 41 [...]... 7 Optimality Criteria in Nonlinear Programming with Differentiability The minimization problems and the Fritz John and Kuhn-Tucker stationary-point problems 2 Sufficient optimality criteria 3 Necessary optimality criteria 90 92 1 93 96 97 Chapter 8 Duality in Nonlinear Programming 113 1 Duality in nonlinear programming 2 Duality in quadratic programming 3 Duality in linear programming 123 126 Chapter... Implicit function theorem 200 200 204 204 Bibliography 205 Name Index 215 217 Subject Index This page intentionally left blank Chapter One The Nonlinear Programming Problem, Preliminary Concepts, and Notation 1 The nonlinear programming problem f The nonlinear programming problem that will concern us has three fundamental ingredients: a finite number of real variables, a finite number of constraints... Classics Edition Preface To the Reader Chapter 1 The Nonlinear Programming Problem, Preliminary Concepts, and Notation 1 The nonlinear programming problem 2 Sets and symbols 3 Vectors 4 Matrices 5 Mappings and functions 6 Notation Chapter 2 Linear Inequalities and Theorems of the Alternative 1 2 3 4 Introduction The optimalily criteria of linear programming: An application of Farkas' theorem Existence... based on a course in nonlinear programming given in the Electrical Engineering and Computer Sciences Department and the Industrial Engineering and Operations Research Department of the University of California at Berkeley and in the Computer Sciences Department of the University of Wisconsin at Madison The intent of the book is to cover the fundamental theory underlying nonlinear programming for the... real values that the variables xi, • • , xn assume 1.1 Nonlinear Programming the variables The fundamental difference between this problem and that of the classical constrained minimization problem of the ordinary calculus [Courant 47, Fleming 65] f is the presence of the inequalities 1 As such, inequalities will play a crucial role in nonlinear programming and will be studied in some detail As an example... problem We shall then be concerned with obtaining necessary and/or sufficient conditions that a point (x\, ,xn) must satisfy in order for it to solve the nonlinear programming problem 1 to 3 These optimality conditions form the crux of nonlinear programming In dealing with problems of the above type we shall confine ourselves to minimization problems only Maximization problems can be easily converted... then x is said to be semipositive, and if x > 0 then x is said to be positive The relations =, ^, >, > defined above are called ordering relations (in Rn) The nonlinear programming problem By using the notation introduced above, the nonlinear programming problem 1.1.1 to 1.1.3 can be rewritten in a slightly more general form as follows Let X° C Rn, let g, h, and 6 be respectively an m-dimensional vector... obtain the nonlinear programming problem 1.1.1 to 1.1.3 If X° = Rn and 6, g, and h are all linear functions on Rn, then problem 9 becomes a linear programming problem: Find an x, if such exists, such that where 6, c, and d are given fixed vectors in Rn, Rm, and Rk respectively, and A and B are given fixed m X n and k X n matrices respectively There exists a vast literature on the subject of linear programming. .. linear programming problem [Simmonard 66, p 95] 16 Chapter Two Linear Inequalities and Theorems of the Alternative 1 Introduction It was mentioned in Chap 1 that the presence of inequality constraints in a minimization problem constitutes the distinguishing feature between the minimization problems of the classical calculus and those of nonlinear programming Although our main interest lies in nonlinear. .. firstyear graduate The only prerequisite would be a good course in advanced calculus or real analysis (Linear programming is not a prerequisite.) All the results needed in the book are given in the Appendixes I am indebted to J Ben Rosen who first introduced me to the fascinating subject of nonlinear programming, to Lotfi A Zadeh who originally suggested the writing of such a book, to Jean-Paul Jacob, Phillippe ... criteria 90 92 93 96 97 Chapter Duality in Nonlinear Programming 113 Duality in nonlinear programming Duality in quadratic programming Duality in linear programming 123 126 Chapter Generalizations... intentionally left blank Chapter One The Nonlinear Programming Problem, Preliminary Concepts, and Notation The nonlinear programming problem f The nonlinear programming problem that will concern... calculus and those of nonlinear programming Although our main interest lies in nonlinear problems, and hence in nonlinear inequalities, linearization (that is, approximating nonlinear constraints