Optimization in Practice with MATLAB® provides a unique approach tooptimization education. It is accessible to junior and senior undergraduate,and graduate students, as well as industry practitioners. It provides a stronglypractical perspective that allows the student to be ready to use optimization inthe workplace. It covers traditional materials, as well as important topicspreviously unavailable in optimization books (e.g., Numerical Essentials – forsuccessful optimization).Outstanding features include:• Provides practical applications of realworld problems using MATLAB.• Each chapter includes a suite of practical examples and exercises that helpstudents link the theoretical, the analytical and the computational. These includea robust set of realworld exercises.• Provides supporting MATLAB codes that offer the opportunity to applyoptimization at all levels, from students’ term projects to industry applications.• Offers instructors a comprehensive solution manual with solution codes alongwith lectures in PowerPoint with animations for each chapter. The MATLAB mfilesare available for download from the book’s website.• Instructors have the unique flexibility to structure one or twosemester coursesthat may range from gentle introductions to highly challenging, for undergraduateor graduate students.
OPTIMIZATION IN PRACTICE WITH MATLAB® FOR ENGINEERING STUDENTS AND PROFESSIONALS Optimization in Practice with MATLAB® provides a unique approach to optimization education It is accessible to junior and senior undergraduate, and graduate students, as well as industry practitioners It provides a strongly practical perspective that allows the student to be ready to use optimization in the workplace It covers traditional materials, as well as important topics previously unavailable in optimization books (e.g., Numerical Essentials – for successful optimization) Outstanding features include: • Provides practical applications of real-world problems using MATLAB • Each chapter includes a suite of practical examples and exercises that help students link the theoretical, the analytical and the computational These include a robust set of real-world exercises • Provides supporting MATLAB codes that offer the opportunity to apply optimization at all levels, from students’ term projects to industry applications • Offers instructors a comprehensive solution manual with solution codes along with lectures in PowerPoint with animations for each chapter The MATLAB mfiles are available for download from the book’s website • Instructors have the unique flexibility to structure one- or two-semester courses that may range from gentle introductions to highly challenging, for undergraduate or graduate students Dr Achille Messac received his BS, MS and PhD from MIT in Aerospace Engineering Dr Messac is a Fellow of the American Institute of Aeronautics and Astronautics (AIAA) and the American Society of Mechanical Engineers He has authored or co-authored more than 70 journal and 130 conference articles, chaired several international conferences, delivered several keynote addresses, and received the prestigious AIAA Multidisciplinary Design Optimization Award He has taught or advised undergraduate and graduate students in the areas of design and optimization for more than three decades at Rensselaer Polytechnic Institute, MIT, Syracuse University, Mississippi State and Northeastern University ® Optimization in Practice with MATLAB for Engineering Students and Professionals Achille Messac, PhD 32 Avenue of the Americas, New York, NY 10013-2473, USA Cambridge University Press is part of the University of Cambridge It furthers the University’s mission by disseminating knowledge in the pursuit of education, learning and research at the highest international levels of excellence www.cambridge.org Information on this title: www.cambridge.org/9781107109186 © Achille Messac 2015 This publication is in copyright Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press First published 2015 Printed in the United States of America A catalog record for this publication is available from the British Library ISBN 978-1-107-10918-6 Hardback Additional resources for this publication at www.cambridge.org/Messac Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party Internet websites referred to in this publication and does not guarantee that any content on such websites is, or will remain, accurate or appropriate MATLAB is a registered trademark of The MathWorks, Inc Contents List of Figures List of Tables Preface Contacting the Author Regarding this Book Book Website Book Organization A Message to Students A Message to Industry Practitioners A Message to Instructors Acknowledgements PART I HELPFUL PRELIMINARIES MATLAB® as a Computational Tool 1.1 Overview 1.2 MATLAB Preliminaries—Before Starting 1.2.1 What Is MATLAB? 1.2.2 Why MATLAB? 1.2.3 MATLAB Toolboxes 1.2.4 How to Use MATLAB in this Book 1.2.5 Acquiring MATLAB 1.2.6 MATLAB Documentation 1.2.7 Other Software for Optimization 1.3 Basics of MATLAB—Getting Started 1.3.1 Starting and Quitting MATLAB 1.3.2 MATLAB Desktop: Its Graphical User Interface 1.3.3 Matrices and Variables Operations 1.3.4 More MATLAB Expressions 1.4 Beyond the Basics of MATLAB 1.4.1 Input and Output, Directories and Files 1.4.2 Flow Control, Relational and Logical Operators 1.4.3 M-files 1.4.4 Global and Local Variables 1.4.5 MATLAB Help 1.5 Plotting Using MATLAB 1.5.1 Basic Plots 1.5.2 Special Plots: Contour, Scatter, fplot 1.5.3 3-D Mesh and Surface Plots 1.5.4 Using the Plot Editing Mode 1.6 Optimizing with MATLAB 1.7 Popular Functions and Commands, and More 1.8 Summary 1.9 Problems Bibliography of Chapter 1 Mathematical Preliminaries 2.1 Overview 2.2 Vectors and Geometry 2.2.1 Dot Product 2.2.2 Equation of a Line 2.2.3 Equation of a Plane 2.3 Basic Linear Algebra 2.3.1 Preliminary Definitions 2.3.2 Matrix Operations 2.3.3 Determinants 2.3.4 Inverse 2.3.5 Eigenvalues 2.3.6 Eigenvectors 2.3.7 Positive Definiteness 2.4 Basic Calculus: Types of Functions, Derivative, Integration and Taylor Series 2.4.1 Types of Functions 2.4.2 Limits of Functions 2.4.3 Derivative 2.4.4 Partial Derivative 2.4.5 Indefinite Integration 2.4.6 Definite Integration 2.4.7 Taylor Series 2.5 Optimization Basics: Single-Variable Optimality Conditions, Gradient, Hessian 2.5.1 Necessary Conditions for Local Optimum 2.5.2 Stationary Points and Inflection Points 2.5.3 Sufficient Conditions for Local Optima 2.5.4 Gradient and Hessian of a Function 2.6 Summary 2.7 Problems Bibliography of Chapter 2 PART II USING OPTIMIZATION—THE ROAD MAP Welcome to the Fascinating World of Optimization 3.1 Overview 3.2 What Is Optimization? What Is Its Relation to Analysis and Design? 3.3 Why Should Junior and Senior College Students Study Optimization? 3.4 Why Should Graduate Students Study Optimization? 3.5 Why Should Industry Practitioners Study Optimization? 3.6 Why Use this Book, and What Should I Expect from It? 3.7 How this Book Is Organized 3.8 How to Read and Use this Book 3.9 Summary 3.10 Problems Bibliography of Chapter 3 Analysis, Design, Optimization and Modeling 4.1 Overview 4.2 Analysis, Design and Optimization 4.2.1 What Is Analysis? 4.2.2 What Is Design? 4.2.3 What Is Optimization? 4.2.4 Interdependence of Analysis, Design and Optimization 4.3 Modeling System Behavior and Modeling the Optimization Problem 4.3.1 Modeling System Behavior 4.3.2 Modeling the Optimization Problem 4.3.3 Interdependence of System Behavior Modeling and Optimization Modeling 4.4 Summary 4.5 Problems Bibliography of Chapter 4 Introducing Linear and Nonlinear Programming 5.1 Overview 5.2 Problem Classes 5.3 Single Objective Optimization—An Inclusive Notion 5.4 Solution Approaches: Analytical, Numerical, Experimental and Graphical 5.4.1 Analytical Optimization 5.4.2 Numerical (or Algorithmic) Optimization 5.4.3 Experimental Optimization 5.4.4 Graphical Optimization 5.5 Software Options for Optimization 5.5.1 MATLAB Optimization Code—fmincon and linprog 5.5.2 Software for Optimization as Stand-Alone (SO-SA) 5.5.3 Software for Optimization Within Design Framework (SO-WDF) 5.5.4 Software for Optimization Within Analysis Package (SO-WAP) 5.6 Summary 5.7 Problems Bibliography of Chapter 5 PART III USING OPTIMIZATION—PRACTICAL ESSENTIALS Multiobjective Optimization 6.1 Overview 6.2 The Multiobjective Problem Definition 6.2.1 Example Problem 6.2.2 Multiobjective Optimization Problem Statement 6.3 Pareto Optimal Solution 6.3.1 Introducing the Pareto Solution 6.3.2 The Pareto Frontier 6.3.3 Obtaining Pareto Solutions 6.3.4 Aggregate Objective Function 6.4 The Weighted Sum Method 6.4.1 Two-Objective Case 6.4.2 Addressing More than Two Objectives 6.5 Compromise Programming 6.6 Generating the Pareto Frontier—with MATLAB 6.7 Reaching a Target—Goal Programming 6.8 Expressing a Preference—Physical Programming 6.9 Multiobjective Optimization Using MATLAB Optimization Toolbox 6.10 Summary 6.11 Problems Bibliography of Chapter 6 Numerical Essentials 7.1 Overview 7.2 Numerical Conditioning—Algorithms, Matrices and Optimization Problems 7.2.1 Reasons Why the Optimization Process Sometimes Fails 7.2.2 Exposing Numerical Conditioning Issues—Algorithms and Matrices 7.2.3 Exposing Numerical Conditioning Issues—Optimization Problems 7.3 Scaling and Tolerances 7.3.1 Understanding the Accuracy of the Reported Results 7.3.2 Design Variable Scaling—Order of Magnitude (DV-1) 7.3.3 Design Variable Scaling—Tolerance Definition (DV-2) 7.3.4 Design Variable Scaling—Optimization Code Decimal Accuracy Setting (DV-3) 7.3.5 Design Variable Scaling—Combining Order of Magnitude and Desired Tolerance (DV-4) 7.3.6 Design Variable Scaling—Setting Scaling Parameters (DV-5) 7.3.7 Objective Function Scaling 7.3.8 Behavioral Constraints Scaling 7.3.9 Setting MATLAB Optimization Options and Scaling Parameters: Syntax 7.3.10 Simple Scaling Examples 7.4 Finite Difference 7.4.1 Fundamentals of Finite Difference 7.4.2 Accuracy of Finite Difference Approximation 7.5 Automatic Differentiation 7.6 Other Important Numerical and Computational Issues 7.6.1 Sensitivity of Optimal Solutions in Nonlinear Programming 7.6.2 Optimization Termination Criteria and Optimization Termination Causes 7.6.3 Developing Confidence in Optimization Results 7.6.4 Problem Dimension and Computational Burden 7.6.5 Additional Numerical Pitfalls 7.7 Larger Scaling Example: Universal Motor Problem 7.7.1 Universal Motor Problem Definition 7.7.2 Design Variable Scaling 7.8 Summary 7.9 Problems Bibliography of Chapter 7 Global Optimization Basics 8.1 Overview 8.2 Practical Issues in Global Optimization 8.3 Exhaustive Search 8.4 Multiple Start 8.5 Role of Genetic Algorithms in Global Optimization 8.6 MATLAB Global Optimization Toolbox 8.7 Summary 8.8 Problems Bibliography of Chapter 8 Discrete Optimization Basics 9.1 Overview 9.2 Defining Discrete Optimization 9.3 Exhaustive Search 9.4 Relaxation Approach 9.5 Advanced Options: Genetic Algorithms, Simulated Annealing, and Branch and Bound 9.5.1 Genetic Algorithms 9.5.2 Simulated Annealing Queipo, N V 355, 363–365, 373 Ragsdell, K M 3, 42, 61, 69, 271, 278, 315, 331, 365, 374 Rais-Rohani, M 444 Rangaiah, G P 124, 157 Rangavajhala, S 151, 157, 236, 247, 393, 394, 398, 399, 405 Rao, R R 389, 404 Rao, S S 74, 81, 339, 347, 354, 357, 359, 374 Ratner, M 110, 120 Ravindran, A 3, 42, 61, 69, 271, 278, 315, 331, 365, 374 Reklaitis, G V 3, 42, 61, 69, 271, 278, 315, 331, 365, 374 Renaud, J E 380, 393, 402, 404 Rijckaert, M J 74, 81 Robert, C P 387, 404 Roh, L 185, 199 Roos, C 274, 278 Roozenburg, N 82, 92 Rosen, K H 339, 354 Ross, S M 226, 246, 382, 388, 403 Rozvany, G I 223, 246, 387, 403 Runger, G C 226, 227, 246 Ruszczynski, A 279, 308 Saccoman, J T 95, 119 Sakata, S 371, 374 Sallaberry, C J 384, 403 Santner, T J 360, 374 Sarker, R A 74, 81 Sauer, T 158, 198 Schaback, R 285, 309 Schaffer, J D 451, 459 Scheinberg, K 287, 309 Scherer, G 233, 246 Schmidt, L C 136, 157, 429, 444 Schoen, F 200, 212 Schraudolph, N N 447, 459 Schweiger, C A 243, 247 Seepersad, C 393, 405 Seiler, P 356, 373 Sentz, K 384, 403 Shan, S 363, 374, 392, 404 Sherali, H D 3, 42, 279, 308, 310, 331 Shetty, C M 3, 42, 279, 308, 310, 331 Shyy, W 355, 363–365, 373 Siarry, P 124, 157 Siddall, J N 74, 81, 376, 402 Sigmund, O 356, 373 Sim, M 399, 405 Simpson, T W 188, 199, 367, 374, 386, 393, 398, 403–405 Smith, R C 382, 403 Smith, S 110, 120, 352, 354 Snyman, J 46, 69 Sóbester, A 362, 374 Sobieski, I 83, 92 Sobieszczanski-Sobieski, J 83, 92, 237, 247, 453, 459 Sofer, A 111, 120, 310, 331 Song, J 392, 404 Sorensen, C 393, 405 Sorensen, J D 392, 404 Srinivas, N 451, 459 Srolovitz, D 89, 92 Stancu-Minasian, I M 399, 405 Stump, G 398, 405 Sturdza, P 185, 199 Sumathi, S 205, 212 Sun, W 237, 247 Sun, Y 356, 373 Sun, Z 384, 403 Surekha, P 205, 212 Swiler, L P 381, 402 Syslo, M M 339, 354 Taguchi, G 399, 405 Taguchi, S 399, 405 Terlaky, T 274, 278 Thapa, M N 251, 271, 276, 277 The MathWorks, Inc 4, 5, 7, 13, 23, 42, 210, 212, 371, 375 Thiele, L 451, 459 Tong, W 351, 354, 364, 374, 455, 459 Torczon, V 237, 247 Toropov, V V 187, 199 Tsui, K L 399, 405 Tsuzuki, M 218, 222, 454, 455, 459 Tu, J 392, 404 Tucker, P K 355, 363–365, 373 Urbina, A 381, 402 Utkin, L V 384, 403 Vaidyanathan, R 355, 363–365, 373 Van Impe, J 408, 428 Van Veldhuizen, D A 124, 157, 451, 459 Vanderbei, R J 251, 276, 278, 306, 309, 399, 405 Vanderplaats, G N 74, 81, 110, 120 Vanderplaats Research and Development, Inc 8, 43 Venter, G 453, 459 Vial, J.-Ph 274, 278 Vicente, L N 287, 309 Von Zuben, F J 454, 459 Wang, G 363, 374, 392, 404 Wang, P 384, 385, 403 Waren, A D 110, 113, 120 Watson, J 113, 120 Watson, L.T 371, 374 Wikle, C K 371, 375 Wilde, D J 74, 81 Williams, A C 243, 247 Williams, B J 360, 374 Wilson, B H 444 Winston, W L 114, 120 Wood, A S 187, 199 World, GAMS 352, 354 Wright, S J 8, 42, 109, 119, 237, 238, 247, 251, 274, 276, 277 Wu, W 56, 69 Xi., Z 385, 403 Xiu, D 389, 404 Xue, D 24, 43 Yang, R 392, 404 Ye, Y 95, 119, 251, 276, 277 Yip, S 89, 92 Youn, B D 384, 385, 392, 403, 404 Youngblood, R W 381, 402 Yuan, Y 237, 247 Yukish, M 398, 405 Zadeh, P M 187, 199 Zak, S H 56, 69 Zako, M 371, 374 Zaman, K 387, 404 Zeng, W 356, 373 Zenios, S 399, 405 Zhang, J 351, 354, 356, 364, 373, 374, 455, 459 Zhang, Junqiang 151, 157, 236, 247 Zhang, X 356, 373 Zhao, Y 356, 373 Zheng, Q P 97, 119 Zhou, J 384, 385, 403 Zitzler, E 451, 459 Zuo, M J 384, 403 Zwicknagl, B 285, 309 Subject Index Accuracy of results, 165 optimizaton code, 165 physics-based model, 165 Analysis, 83 engineering, 83 Analytical differentiation, 182 Ant colony optimization, 454 Automatic differentiation, 182, 183 ADIFOR, 185 Basis function nonparametric, 368 parametric, 368 Bi-objective NNC method, see Normalized normal constraint method anchor point, 412 normalized increments, 412 objective mapping, 412 utopia line points, 412 utopia line vector, 412 Bisection method, 281 Book, 3, 4, 7 helpful preliminaries, 1 how to read, 80 how to use Matlab, 7 Matlab related books, 7, 23 organization, 79 why use this book, 78 Branch and Bound, 218, 342 Computation time or expense, 91, 169, 203, 206, 210, 339, 355, 372, 387, 389, 393, 446 Computational complexity, 338 polynomial time, 339 Computational Fluid Dynamics, 355 Conjugate gradient, 295 Constrained nonlinear programming, 310 computational issues, 328 elimination method, 312 KKT conditions, see Karush-Kuhn Tucker conditions penalty method, 313 sequential linear programming, 323 sequential quadratic programming, 326 Constrained optimization, 62 Constraint, 90 behavior, 94 equality, 94 inequality, 94 side, 94 Convergence rate, 303 Covariance, 388 Cutting plane method Gomory constraint, 347 Gomory cut, 343, 346 Simplex tableau, 344, 348 Data fitting, 232 least squares, see Least squares method Decision variable, see Design variable Design, 84 conceptual, 85 detailed, 85 preliminary, 85 Design objective, 90 Design of Experiments, 179, 355, 357, 359, 364, 372 Design optimization software, 8 Design variable, 85, 90 Design variable linking, 357, 359 Design variable scaling, 190 decimal accuracy, 168 order of magnitude, 166 tolerance, 167 Desirability ranges, 430 acceptable range, 432 desirable range, 432 highly desirable range, 436 highly undesirable range, 432 ideal range, 432, 436 tolerable range, 432 unacceptable range, 432 undesirable range, 432 Differentiation analytical, 182 automatic, 182 Finite difference, 176 Discrete optimization, 96, 213, 336, 337 binary programming, 214 Branch and Bound, 218, 342 combinatorial, 214, 336 cutting plane, 343 discrete non-integer, 214 discrete non-integer programming, 336 exhaustive search, 214, 340 formulation, 213 genetic algorithms, see Genetic algorithms graphical method, 341 integer programming, 96, 214, 336 mixed-integer programming, 214, 336 relaxation approach, 215, 341 zero-one programming, 336 Duality, 272 dual, 272 primal-dual relationships, 273 Eigenvalue, 32, 54 Eigenvector, 32, 54 Elimination method, 312 Evolutionary algorithms, 205, 445 Ant colony optimization, 454 genetic algorithms, 446 Exhaustive search, 202, 214 Exhaustive search method, 340 Feasible solution, 94 Finite difference, 176, 182 backward, 177 central, 177 forward, 177 Finite Element Analysis, 355 First order methods, 293 conjugate gradient, 295 steepest descent, 293 Flow control, 20, 21, 38 fmincon.m, 101, 105, 114 fmincon.m scaling, 164 Function definite integration, 60 derivative, 59 gradient, 64 Hessian, 65 indefinite integration, 60 limit, 59 partial derivative, 60 Function type, 56 concave function, 58 continuous function, 56 convex function, 58 discrete function, 57 monotonically decreasing, 57 monotonically increasing, 57 multimodal function, 58, 201 unimodal function, 57, 201 Gauss Jordan Elimination, 259 Genetic algorithms, 446 crossover, 207, 448 elitist, 207, 448 mutation, 207, 448 reproduction, 207, 448 Global minimum, 62, 201 Global optimization, 200 evolutionary algorithms, see Evolutionary algorithms, 445 exhaustive search, 202 formulation, 200 genetic algorithms, see Genetic algorithms, see Genetic algorithms global minimum, 201 local minimum, 201 multiple start approach, 203 Global Optimization Toolbox, 5, 209, see MATLAB Toolboxes, 446, 449, 452 Goal programming, 439 Golden section search, 283 Inflection point, 63 Interior point method, 255, 274, 276, 313, 314 Interval halving method, 281 Karush-Kuhn Tucker conditions, 317 Lagrange multiplier, 317 Lagrangian function, 317 Least squares method, 366 Line search methods, see Steepest descent method, Conjugate gradient method, Newton method, Quasi-Newton methods Linear algebra, 46, see Matrix Linear physical programming, 430 goal programming comparison, 439 ideal range, 432 Linear programming, 251 duality, 272 graphical solution, 253 Simplex method, see Simplex method using Matlab, 255 Linear programming problem, 95 Linear programming solution types, 253 infinity, 255 no solution, 254 segment, 253 unique, 253 linprog.m, 103, 107, 108, 114 Local minimum, 62, 201 sufficient condition, 63 Local optimum necessary condition, 62 Mathematical preliminaries, 44 MATLAB, 3 MATLAB Optimization Toolbox, 137 MATLAB Toolboxes, 5, 210 Global Optimization Toolbox, 200 Optimization Toolbox, 137 Matrix characteristic equation, 54 cofactor, 51 eigenvalue, 54 eigenvector, 54 Inverse, 53 lower triangular, 48 multiplication, 49 non-singular, 52 positive definite, 55 principal diagonal, 48, 52 singular, 52 skew symmetric, 51 symmetric, 51 transpose, 50 upper triangular, 48 Mean, 380, 382, 388 Modeling, 88 mathematic, 88 optimization problem, 90 physics-based analytical, 89 process, 88 simulation-based, 89 surrogate, 89 system behavior, 88 Monte carlo method, 386 Sample size, 386 Multidisciplinary design, analysis and opt., 111, 112, 363, 393, 402 Multiobjective tradeoff, 96 Multiobjective genetic algorithms, 451 non-dominated sorting, 451 Multiobjective optimization, 123, 406, 429 aggregate objective function, 127, 128, 429 bi-objective, 125 compromise programming, 131 dominated solution/non-Pareto, 126, 418 genetic algorithms, 451 goal programming, 135, 439 multiojective genetic algorithms, 451 non-dominated solution, see Pareto optimal solution Pareto solution, 406, 429 physical programming, see Physical programming problem statement, 124 weighted sum method, 128 n-objective NNC method, see Normalized normal constraint method anchor point, 414 hyperplane points, 415 normalized increment, 415 objective mapping, 414 utopia plane vector, 415 Newton method, 298 Non-probabilistic methods, 384 Evidence theory, 384 Possibility theory, 384 Nonlinear physical programming, 436 Nonlinear programming, 279, 310 constrained, see Constrained nonlinear programming problem, 95 unconstrained, see Unconstrained nonlinear programming Normal boundary intersection method, 408, 422 Normal constraint method, see Normalized normal constraint method Normalized normal constraint method, 410, 413, 427 bi-objective, 410 n-objective, 413 Numerical conditioning, 158 ill-conditioned, 159 well-conditioned, 158 Numerical conditioning algorithm, see Numerical conditioning One vs Others (OVO) rule, 433 Optimization approaches algorithmic optimization, 99 analytical optimization, 98 experimental optimization, 101 graphical optimization, 101 numerical optimization, 99 Optimization classes, 94 constrained optimization, 95 continuous optimization, 96 deterministic optimization, 97 discrete optimization, see Discrete optimization global optimization, see Global optimization linear programming, see Linear programming local optimization, 97 multiobjective optimization, 96 nondeterministic optimization, 97 nonlinear programming, see Nonlinear programming single objective optimization, 96 unconstrained optimization, 95 Optimization formulation constraint function, see Constraint objective function, see Design objective Optimization software, 102 Optimization Toolbox, 5, 103 Parallel computing, 111 toolbox, see Matlab Toolboxes Pareto filter, 417 global Pareto, 417, 418 local Pareto, 417, 418 Pareto frontier, 126, 406, 428 anchor point, 407 utopia hyperplane, 407 utopia line, 407 utopia point, 407 Pareto optimal solution, 125, 126 Pareto optimality, 125, 417 Particle Swarm Optimization, 445, 453, 455 Pattern search method, 290 Penalty method, 313 Physical programming, 429 class function, 430, 433 convexity, 434 desirable range, 432 deviational variables, 435 hard classes, 430 highly undesirable range, 432 linear, 430 linear vs nonlinear, 436 nonlinear, 436 One vs others (OVO) rule, 432 one vs others rule, 433, 434 preference function, see Preference function PysPro, 111, 430 scaling, 433 soft classes, 430 splines, 437 tolerable range, 432 unacceptable range, 432 undesirable range, 432 weight algorithm, 435 PhysPro, 111, 430 Polynomial response surface, 365 Popular example problems, 336 capital budgeting, 337 Knapsack, 337 traveling salesman, 337 vehicle routing, 337 Preference function, 430 inter-criteria, 432 intra-criterion, 432 Probability density function, 227, 382 Definition, 382 Lognormal distribution, 382 Normal distribution, 382 Weibull distribution, 382 Quasi-Newton methods, 300 Radial basis function, 367 Random variable, 382 Reliability, 376, 390 Reliability based design optimization, 390, 391 Limit state, 392 Most probable point, 392 Failure probability, 391 Robust design optimization, 390, 393 Robustness, 376, 390 Row Echelon form, 261 Safety factors, 376 Scalar, 44 Scaling, 159, 165 behavioral constraint, 171 constraint, 172 design variable, see Design variable scaling fmincon.m, 164 objective function, 170 Second order methods, 298 Newton method, 298 quasi-Newton methods, 300 Sensitivity analysis, 360 Sequential linear programming, 323 Sequential quadratic programming, 326 Simplex method, 257 slack variable, 258 standard form, 257 surplus variable, 258 Simplex search method, 287 Simulated Annealing, 209, 210, 213, 217, 218, 445, 454 Simulated annealing, 454 Single variable optimization, 280 bisection method, 281 golden section search, 283 interval halving method, 281 polynomial approximation, 285 quadratic approximation, 285, 286, 305 Slack variable, 258 Software, see Software for/with optimization Software for/with optimization, 102 MATLAB optimization toolbox, see Optimization Toolbox ABAQUS, 113 Altair, 112 ANSYS, 113 Boss Quattro, 112, 352 BTN, 111 COMSOL, 113 CPLEX, 110, 351 DOT, 110 GAMS, 352 GENESIS, 110 GRG2, 110, 352 iSIGHT, 111 LINDO, 114 LSSOL, 110 Microsoft Excel, 113, 114 modeFRONTIER, 112 NASTRAN, 113 NEOS server, 109, 352 NPSOL, 110 PHX ModelCenter, 112 PhysPro, 111, 430 VisualDOC, 110 XPRESS, 351 Solution sensitivity, 191, 274 Standard deviation, 380, 382, 388 Stationary point, 63 Steepest descent, 293 Stochastic programming, 399 Surplus variable, 258 Surrogate models Artificial Neural Network, 355, 364, 371 Extended Radial Basis Functions, 370 Kriging, 355, 364, 371, 393 polynomial response surface, 365 radial basis function, 367 Tabu Search, 445, 453, 454 Taguchi methods, 399 Taylor series, 61, 388 Toolbox, see Matlab Toolboxes Tradeoff, 96, 397, 398 Training point, 367 Trust region, 240 Uncertainty propagation, 385 Polynomial Approximation, 387 Sampling methods, 385 Uncertainty quantification, 382 Uncertainty types, 380 Aleatory, 380 Epistemic, 380 Unconstrained nonlinear programming, 279 method comparison, 305 multivariable optimization, 287 necessary condition, 280 single variable optimization, 280 sufficient condition, 280 Unconstrained optimization, 62 Vector, 44 dot product, 44 inner product, 44 Weighted sum method, 429 Why MATLAB, 5 Why optimization for college students, 77 for graduate students, 77 for industry practitioners, 78 Zero order methods, 287 pattern search, 290 Simplex search, 287 [...]... Practicing Optimization – Larger Examples Part IV Going Deeper: Inside the Codes and Theoretical Aspects 11 Linear Programming 12 Nonlinear Programming with No Constraints 13 Nonlinear Programming with Constraints Part V More Advanced Topics in Optimization 14 Discrete Optimization 15 Modeling Complex Systems: Surrogate Modeling and Design Space Reduction 16 Design Optimization Under Uncertainty 17... Helpful Preliminaries 1 MATLAB as a Computation Tool 2 Mathematical Preliminaries Part II Using Optimization - The Road Map 3 Welcome to the Fascinating World of Optimization 4 Analysis, Design, Optimization, and Modeling 5 Introducing Linear and Nonlinear Programming Part III Using Optimization - Practical Essentials 6 Multiobjective Optimization 7 Numerical Essentials 8 Global Optimization Basics 9 Discrete Optimization Basics... beyond the basics Section 1.5 focuses on the MATLAB plotting capabilities In Sec 1.6, the MATLAB nonlinear and linear optimization capabilities are introduced For historical reasons, the terminology “linear programming” and “nonlinear programming” is often used synonymously with “linear optimization and “nonlinear optimization, ” respectively In Sec 1.7, a list of ... computing was a critical issue that hindered the broad application of optimization Fortunately, with the revolutionary decrease in the cost of computing in recent years, a desktop computer is often all that is needed to solve many practical optimization problems—making the application of optimization dramatically more practical The application of optimization in practical settings is increasingly becoming... succeed Interestingly, it would not be at all surprising that the expert may find it too difficult to obtain an adequate answer, when optimization can be successfully used to find one This is, in part, because optimization can intelligently examine thousands of design alternatives in less time than it takes the expert to examine a single design alternative Optimization can also perform an intelligent... human being may not be able to find through experience, intuition, or courageous trial-and-error Optimization can be defined as the art of making things better Interestingly, optimization very often does not simply allow us to do something better, but it may also make it possible to do something that we did not otherwise know how to do To take full advantage of the power of optimization in practice, there is no choice but to use a computer... computational and theoretical optimization process for linear programming and nonlinear programming with and without constraints This is equivalent to learning the basics of what is under the hood of a car Part V builds on the first parts of the book to provide a foundation for more advanced studies in optimization In Part V, we learn optimization at a deeper level, where advanced topics are introduced over six chapters... Final Simplex Tableau Before Adding Cutting Planes 14.5 Initial Simplex Tableau After Adding First Cutting Plane (Eq 14.55) 14.6 Final Simplex Tableau After Adding the First Cutting Plane (Eq 14.55) 14.7 Initial Simplex Tableau After Adding Second Cutting Plane (Eq 14.62) 14.8 Final Simplex Tableau After Adding Second Cutting Plane (Eq 14.62) 14.9 Broad Classification of Software for Optimization with Discrete Case 16.1 Sample Set of One Hundred Cross-Sectional Areas for a Batch-Produced Truss... In Part II, three chapters introduce the world of optimization in the form of a road map This part provides the basics of what should be known about optimization before attempting to use it In Part III, there are five chapters that teach the basic use of optimization In fact, learning the material up to Part III provides the practitioner with sufficient information for solving practical problems In doing so, student users will not be experts on how optimization works under the hood, so... The approaches do not fundamentally change with different optimization codes The changes that occur from problem to problem are readily handled with any optimization code, such as the size and other generic features of the problem MATLAB is an easy-to-use and very popular software that is useful in all areas of engineering It is also used in a growing number of non-engineering fields If you don’t yet know MATLAB, that is fine You will