Optimization for decision making linear and quadratic models

International Series in Operations Research & Management Science Volume 137 Series Editor Frederick S Hillier Stanford University, CA, USA Special Editorial Consultant Camille Price Stephen F Austin State University, TX, USA For further volumes: http://www.springer.com/series/6161 Katta G Murty Optimization for Decision Making Linear and Quadratic Models ABC Katta G Murty University of Michigan Dept Industrial and Operations Engineering 1205 Beal Avenue Ann Arbor MI 48109-2117 USA murty@umich.edu and Professor, Systems Engineering Department King Fahd University of Petroleum and Minerals Dhahran Saudi Arabia ISSN 0884-8289 ISBN 978-1-4419-1290-9 e-ISBN 978-1-4419-1291-6 DOI 10.1007/978-1-4419-1291-6 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: 2009932413 c Springer Science+Business Media, LLC 2010 All rights reserved This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) To the memory of my thesis advisor David Gale (my last teacher in the formal education system) who inspired me into doing research in Optimization, and to my mother Katta Adilakshmi (my first teacher in childhood at home) Preface I was fortunate to get my first exposure to linear programming in a course taught by the father of the subject, George Dantzig, at the University of California, Berkeley, in the fall of 1965 It was love at first sight! I fell in love with linear programming (LP), optimization models, and algorithms in general right then, and became inspired to work on them Another of my fortunes was to have as my thesis advisor David Gale, who along with Harold Kuhn and Albert Tucker contributed to the development of optimality conditions Eventually, I started teaching optimization in the IOE Department at the University of Michigan, Ann Arbor, and using it in applications myself, and I would now like to share this background with future generations of students Level of the Book and Background Needed This is a first-year graduate (Master’s) level textbook on optimization models, linear and quadratic, for decision making, how to formulate real-world problems using these models, use efficient algorithms (both old and new) for solving these models, and how to draw useful conclusions, and derive useful planning information, from the output of these algorithms It builds on the undergraduate (Junior) level book Optimization Models for Decision Making Volume on the same subject (Murty (2005) of Chap 1), which I posted at the public access website: http://ioe.engin.umich.edu/people/fac/books/murty/opti model/, from which you can download the whole book for a small contribution Readers who are new to the subject should read this Junior-level book to acquire the background for reading this graduate-level book vii viii Preface Why Another Book on Linear Programming When friends learned that I was working on this book, they asked me, “Why another book on linear programming (LP)?” There are two reasons: Almost all the best-known books on LP are mathematics books, with little discussion on how to formulate real-world problems as LPs and with very simple modeling examples Within a short time of beginning work on applications, I realized that modeling could actually be as complex as proving mathematical results and requires very special skills To get good results, it is important to model real-world problems intelligently To help the reader develop this skill, I discuss several illustrative examples from my experience, and include many exercises from a variety of application areas All the available books on LP discuss only the simplex method (developed based on the study of LP using the simplex, one of the solids in classical geometry) and perhaps existing interior point methods (developed based on the study of LP using the ellipsoid) All these methods are based on matrix inversion operations involving every constraint in the model in every step, and work well for LPs in which the coefficient matrix is very sparse We discuss also a new method being developed based on the study of LP using the sphere, which uses matrix inversion operations sparingly and seems well suited to solve large-scale LPs, and those that may not have the property of being very sparse Contents of the Book Chapter contains a brief account of the history of mathematical modeling, the Gasuss–Jordan elimination method for solving linear equations; the simplex method for solving LPs and systems of linear constraints including inequalities; and the importance of LP models in decision making Chapter discusses methods for formulating real-world problems, including those in which the objective function to be optimized is a piecewise linear convex function and multiobjective problems, as linear programs The chapter is illustrated with many examples and exercises from a variety of applications Chapter explains the need for intelligent modeling in order to get good results, illustrated with three case studies: one from a container terminal, the second at a bus-rental company, and the third at an international airport Chapter discusses the portion of the classical theory of polyhedral geometry that plays an important role in the study of linear programming and in developing algorithms for solving linear programs, illustrated with many numerical examples Chapter treats duality theory, optimality conditions for LP, and marginal analysis; and Chap discusses the variants of the revised simplex method Both chapters deal with traditional topics in linear programming In Chap we discuss also optimality conditions for continuous variable nonlinear programs and their relationship to optimality conditions for LP Preface ix Chapter discusses interior point methods (IPMs) for LP, including brief descriptions of the affine scaling method, which is the first IPM to be developed, and the primal-dual IPM, which is most commonly used in software implementations Chapter discusses the sphere methods, new IPMs that have the advantage of using matrix inversion operations sparingly, and thus are the next generation of methods for solving large-scale LPs Chapter discusses extensions of the sphere methods – to convex and nonconvex quadratic programs, and to 0–1 integer programs through quadratic formulations Additional Exercises Exercises offer students a great opportunity to gain a deeper understanding of the subject Modeling exercises open the student’s mind to a variety of applications of the theory developed in the book and to a variety of settings where such useful applications have been carried out This helps them to develop modeling skills that are essential for a successful career as a practitioner Mathematical exercises help train the student in skills that are essential for a career in research or a career as a higher-level practitioner who can tackle very challenging applied problems Because of the limitations on the length of the book, not all exercises could be included in it These additional exercises will be included in the website for the book at springer.com in the near future, and even more added over time Some of the formulation exercises at the website deal with medium-size applications; these problems can be used as computational project problems for groups of two or three students Formulating and actually solving such problems using an LP software package gives the student a taste of real-world decision making Citing References in the Text At the end of each chapter, we list only references that are cited in the text Thus the list of references is actually small; it does not provide extensive bibliographies of the subjects For readers who are interested, we refer them to other books available that have extensive bibliographies We use the following style for citing references: A citation such as “Wolfram (2002)” refers to the paper or book of Wolfram of year 2002 listed among references at the end of the current chapter where this citation appears Alternately, a reference such as “(Dikin (1967) of Chap 1) refers to the document of Dikin of year 1967 in the list of references at the end of Chap x Preface Solutions Manual Springer will host the solutions manual at springer.com, allowing token access to registered adopting faculty Acknowledgments I received comments, encouragement, and other help from many people in preparing this book, including Richard Chen, Jose Dula, Stein-Erik Fleten, Santosh Kabadi, Shantisri Katta, Justin Lin, Mohammad Oskoorouchi, A Ravi Ravindran, Romesh Saigal, Arvind Sharma, Eric Svaan, Volodymyr Babich I am grateful to all of them I am also grateful to my editors and supporters, Fred Hillier, Camille Price, and the Springer team for constant encouragement Finally I thank my wife Vijaya Katta for being my companion in all these years Conclusion Optimum decision making is all about improving lives As the Sanskrit proverb (jiivaa ssamastaa ssukhinoo bhavamtu) shown in Telugu script says: I hope readers will use these methods to improve the lives of all living beings! April 2009 Katta Gopalakrishna Murty Other Textbooks by Katta G Murty Linear and Combinatorial Programming, first published in 1976, available from R.E Krieger, Inc., P O Box 9542, Melbourne, FL 32901 Linear Programming, published in 1983, available from John Wiley & Sons, 111 River Street, Hoboken, NJ 07030-5774 Linear Complementarity, Linear and Nonlinear Programming, published in 1988 by Heldermann Verlag, Germany; now available as a download for a voluntary contribution at: http://ioe.engin.umich.edu/people/fac/books/murty/linear complementarity webbook/ Preface xi Operations Research: Deterministic Optimization Models, published in 1995, available from Prentice-Hall Inc Englewood Cliffs, NJ 07632 Network Programming, published in 1992, available from Prentice-Hall Inc Englewood Cliffs, NJ 07632; also as a download for a voluntary contribution at: http://ioe.engin.umich.edu/people/fac/ books/murty/ network programming/ Optimization Models for Decision Making: Volume 1, Junior Level, available as a download for a voluntary contribution at: http://ioe.engin.umich.edu/people/fac/ books/murty/opti model/ Computational and Algorithmic Linear Algebra and n-Dimensional Geometry, Sophomore level, available as a download for a voluntary contribution at: http://ioe engin.umich.edu/people/fac/books/murty/ algorithmic linear algebra/ Contents Linear Equations, Inequalities, Linear Programming: A Brief Historical Overview 1.1 Mathematical Modeling, Algebra, Systems of Linear Equations, and Linear Algebra 1.1.1 Elimination Method for Solving Linear Equations 1.2 Review of the GJ Method for Solving Linear Equations: Revised GJ Method 1.2.1 GJ Method Using the Memory Matrix to Generate the Basis Inverse 1.2.2 The Revised GJ Method with Explicit Basis Inverse 1.3 Lack of a Method to Solve Linear Inequalities Until Modern Times 1.3.1 The Importance of Linear Inequality Constraints and Their Relation to Linear Programs 1.4 Fourier Elimination Method for Linear Inequalities 1.5 History of the Simplex Method for LP 1.6 The Simplex Method for Solving LPs and Linear Inequalities Viewed as an Extension of the GJ Method 1.6.1 Generating the Phase I Problem if No Feasible Solution Available for the Original Problem 1.7 The Importance of LP 1.7.1 Marginal Values and Other Planning Tools that can be Derived from the LP Model 1.8 Dantzig’s Contributions to Linear Algebra, Convex Polyhedra, OR, Computer Science 1.8.1 Contributions to OR 1.8.2 Contributions to Linear Algebra and Computer Science 1.8.3 Contributions to the Mathematical Study of Convex Polyhedra 1.9 Interior Point Methods for LP 1.10 Newer Methods 1.11 Conclusions 1.12 How to Be a Successful Decision Maker? 1.13 Exercises References 1 11 14 15 17 18 19 19 21 22 27 27 27 28 29 30 30 30 31 37 xiii ... optimization models, linear and quadratic, for decision making, how to formulate real-world problems using these models, use efficient algorithms (both old and new) for solving these models, and how... Degenerate BFSs for Systems in Standard Form 4.6.2 Basic Vectors and Bases for a System in Standard Form 4.6.3 BFSs for Systems in Standard Form for Bounded... elimination method for solving linear equations; the simplex method for solving LPs and systems of linear constraints including inequalities; and the importance of LP models in decision making Chapter