Ying ping chen extending the scalability of link(bookfi)

Ying-ping Chen Extending the Scalability of Linkage Learning Genetic Algorithms Studies in Fuzziness and Soft Computing, Volume 190 Editor-in-chief Prof Janusz Kacprzyk Systems Research Institute Polish Academy of Sciences ul Newelska 01-447 Warsaw Poland E-mail: kacprzyk@ibspan.waw.pl Further volumes of this series can be found on our homepage: springeronline.com Vol 182 John N Mordeson, Kiran R Bhutani, Azriel Rosenfeld Fuzzy Group Theory, 2005 ISBN 3-540-25072-7 Vol 175 Anna Maria Gil-Lafuente Fuzzy Logic in Financial Analysis, 2005 ISBN 3-540-23213-3 Vol 183 Larry Bull, Tim Kovacs (Eds.) Foundations of Learning Classifier Systems, 2005 ISBN 3-540-25073-5 Vol 176 Udo Seiffert, Lakhmi C Jain, Patric Schweizer (Eds.) Bioinformatics Using Computational Intelligence Paradigms, 2005 ISBN 3-540-22901-9 Vol 177 Lipo Wang (Ed.) Support Vector Machines: Theory and Applications, 2005 ISBN 3-540-24388-7 Vol 178 Claude Ghaoui, Mitu Jain, Vivek Bannore, Lakhmi C Jain (Eds.) Knowledge-Based Virtual Education, 2005 ISBN 3-540-25045-X Vol 179 Mircea Negoita, Bernd Reusch (Eds.) Real World Applications of Computational Intelligence, 2005 ISBN 3-540-25006-9 Vol 180 Wesley Chu, Tsau Young Lin (Eds.) Foundations and Advances in Data Mining, 2005 ISBN 3-540-25057-3 Vol 181 Nadia Nedjah, Luiza de Macedo Mourelle Fuzzy Systems Engineering, 2005 ISBN 3-540-25322-X Vol 184 Barry G Silverman, Ashlesha Jain, Ajita Ichalkaranje, Lakhmi C Jain (Eds.) Intelligent Paradigms for Healthcare Enterprises, 2005 ISBN 3-540-22903-5 Vol 185 Dr Spiros Sirmakessis (Ed.) Knowledge Mining, 2005 ISBN 3-540-25070-0 Vol 186 Radim Bˇelohlávek, Vilém Vychodil Fuzzy Equational Logic, 2005 ISBN 3-540-26254-7 Vol 187 Zhong Li, Wolfgang A Halang, Guanrong Chen Integration of Fuzzy Logic and Chaos Theory, 2006 ISBN 3-540-26899-5 Vol 188 James J Buckley, Leonard J Jowers Simulating Continuous Fuzzy Systems, 2006 ISBN 3-540-28455-9 Vol 189 Hans-Walter Bandemer Handling Uncertainty by Mathematics, 2006 ISBN 3-540-28457-5 Vol 190 Ying-ping Chen Extending the Scalability of Linkage Learning Genetic Algorithms, 2006 ISBN 3-540-28459-1 Ying-ping Chen Extending the Scalability of Linkage Learning Genetic Algorithms Theory & Practice ABC Ying-ping Chen Natural Computing Laboratory Department of Computer Science and Information Engineering National Chiao Tung University No 1001, Dasyue Rd Hsinchu City 300 Taiwan E-mail: Library of Congress Control Number: 2005931997 ISSN print edition: 1434-9922 ISSN electronic edition: 1860-0808 ISBN-10 3-540-28459-1 Springer Berlin Heidelberg New York ISBN-13 978-3-540-28459-8 Springer Berlin Heidelberg New York This work is subject to copyright All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer Violations are liable for prosecution under the German Copyright Law Springer is a part of Springer Science+Business Media springeronline.com c Springer-Verlag Berlin Heidelberg 2006 Printed in The Netherlands The use of general descriptive names, registered names, trademarks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use Typesetting: by the authors and TechBooks using a Springer LATEX macro package Printed on acid-free paper SPIN: 11339380 89/TechBooks 543210 To my family Foreword It is a pleasure for me to write a foreword for Ying-ping Chen’s new book, Extending the Scalability of Linkage Learning Genetic Algorithms: Theory and Practice I first met Y.-p when he asked to an independent study project with me in the spring of 2000 at the University of Illinois He seemed very interested in genetic algorithms based on some previous studies back in Taiwan, and Georges Harik’s earlier work on the linkage learning genetic algorithm (LLGA) interested him most of all In designing the LLGA, Harik attempted to reconcile the differing time scales of allelic convergence and linkage convergence, by sustaining allelic diversity until linkage convergence occurred The mechanism for achieving this was elegant, and Harik also provided bounding analyses that helped us understand how the mechanism achieved its aims, but the work left us with as many questions as it answered Why did the LLGA work so well on badly scaled problems, and why did it seem to be so limited on uniformly scaled problems? This was the state of our knowledge when Y.-p tackled the problem Early attempts to improve upon the LLGA appeared to be dead ends, and both of us were growing frustrated, but then we decided to break it down into simpler elements, and Ying-ping made progress by performing an enormously clever series of analyses and experiments that showed the way to improved LLGA performance In the end, the work has left us with a better understanding of this particular mechanism and it has suggested that unimetric schemes – schemes that not use some auxiliary modeling metric – may be limited in the performance levels they can achieve Although the goals of this work were to improve an artificial adaptation procedure, we believe that it has important implications for the study of linkage adaptation in nature Thus, I recommend this book to readers in either natural or artificial systems, both for the important ideas that it clarifies and for its painstaking VIII Foreword method of experimentation and analysis Buy this book, read it, and assimilate its crucial insights on the difficulty and scalability of effective linkage learning Urbana, Illinois, USA July, 2005 David E Goldberg Preface There are two primary objectives of this monograph The first goal is to identify certain limits of genetic algorithms that use only fitness for learning genetic linkage Both an explanatory theory and experimental results to support the theory are provided The other goal is to propose a better design of the linkage learning genetic algorithm After understanding the cause of the observed performance barrier, the design of the linkage learning genetic algorithm is modified accordingly to improve its performance on the problems of uniformly scaled building blocks This book starts with presenting the background of the linkage learning genetic algorithm Then, it introduces the use of promoters on chromosomes to improve the performance of the linkage learning genetic algorithm on uniformly scaled problems The convergence time model is constructed by identifying the sequential behavior, developing the tightness time model, and establishing the connection in between The use of subchromosome representations is to avoid the limit implied by the convergence time model The experimental results suggest that the use of subchromosome representations may be a promising way to design a better linkage learning genetic algorithm The study depicted in this monograph finds that using promoters on the chromosome can improve nucleation potential and promote correct buildingblock formation It also observes that the linkage learning genetic algorithm has a consistent, sequential behavior instead of different behaviors on different problems as was previously believed Moreover, the competition among building blocks of equal salience is the main cause of the exponential growth of convergence time Finally, adopting subchromosome representations can reduce the competition among building blocks, and therefore, scalable genetic linkage learning for a unimetric approach is possible HsinChu City, Taiwan June, 2005 Ying-ping Chen 106 Conclusions amount of rearrangement disruption and provide better understanding of building genetic linkage models • Biological ramifications Since genetic and evolutionary algorithms were developed based on the paradigms and principles of evolution in nature, it is possible to find a way to channel the theoretical and computational results of this field into the related fields of biology For the linkage learning genetic algorithm, as a unimetric approach, the interpretation and translation of those theoretical models and practical frameworks in the context of biology might serve as highly simplified computational models and possibly provide insights from different points of view Therefore, interdisciplinary cooperation is in order for this research direction such that our understanding of genetic and evolutionary computation as well as biology and nature might be advanced 9.3 Main Conclusions This research project addresses the scalability issue of the linkage learning genetic algorithm from both theoretical and practical aspects By making use of simple, dimensional models, we gain better understanding of the behavior of the linkage learning genetic algorithm in theory By adopting coding mechanisms existing in genetics and biological systems, we improve the performance of the linkage learning genetic algorithm on uniformly scaled problems in practice According to the status and outcomes of this research project, the main conclusions that may be drawn from this study are listed as follows: • Promoters improve nucleation potential The observation of the genetic linkage learning process showed that the randomness of choosing cutting points, grafting points, and points of interpretation caused the instability of allele expression and led to misnucleation of building blocks By introducing promoters to the chromosome representation, such randomness was effectively reduced, and the modified linkage learning genetic algorithm was made able to solve more uniformly scaled building blocks than the original version could Therefore, promoters can improve nucleation potential and promote correct formation of building blocks • The linkage learning genetic algorithm has a consistent, sequential behavior It was previously believed that when solving a uniformly scaled problem, the linkage learning genetic algorithm worked on all building blocks simultaneously, while when solving an exponentially scaled problem, the linkage learning genetic algorithm worked on the most salient building block, the second most salient building block, and so on However, in this research project, we successfully identified a consistent, sequential behavior of the linkage learning genetic algorithm By identifying the sequential behavior, we gained better understanding about how the linkage learning genetic algorithm handled and processed the building blocks of different scalings 9.3 Main Conclusions 107 • Tightness time is derived based on linkage skew and linkage shift Harik and Goldberg [49] proposed two linkage learning mechanisms, linkage skew and linkage shift, to explain why and how the linkage learning genetic algorithm worked In this monograph, we extended and combined the two linkage learning mechanisms and derived the tightness time model for describing the time needed to tighten a single building block This time model was obtained from the micro view of the linkage learning process and played an essential role in constructing the convergence time model for the linkage learning genetic algorithm • Competition among building blocks of equal salience slows down the genetic linkage learning process According to Sect 7.7, one of the key components in the convergence time model that contributes to the time delay of the linkage learning genetic algorithm is the interaction or competition among multiple building blocks of equal salience The competition severely slows down the genetic linkage learning process because the probability of linkage learning events is significantly reduced • Convergence time grows exponentially with the number of building blocks of equal salience As shown by (7.30), given a fixed length of the building block and a desired genetic linkage, the convergence time for the linkage learning genetic algorithm to solve a uniformly scaled problem grows exponentially with the number of building blocks Such a high order of computational time poses a limit to competence of the linkage learning genetic algorithm because the time required to finish a computation renders the problem infeasible to be solved • Subchromosome representations reduce the competition among building blocks The subchromosome representation was proposed in this work to use in the linkage learning genetic algorithm for effectively lowering the number of building blocks at run time Such a representation reduced the competition among building blocks of equal salience by separating them on different subchromosomes An initial step to realize the subchromosome representation was taken in this study The preliminary experimental results indicated that the proposed coding scheme can improve the performance of the linkage learning genetic algorithm on uniformly scaled problems • Prior linkage information can be incorporated into the linkage learning genetic algorithm The initial step for implementing the subchromosome representation in this study also led to a possible way to make the linkage learning genetic algorithm capable of incorporating prior linkage information In the linkage learning genetic algorithm without subchromosomes, utilizing prior linkage information is extremely difficult if not impossible because of those necessary, significant modifications With the use of subchromosomes, the distribution of genes, non-coding segments, and building blocks can be appropriately arranged in order to utilize the available linkage information 108 Conclusions • Scalable genetic linkage learning for a unimetric approach is possible By using the subchromosome representation, the linkage learning genetic algorithm might be able to solve uniformly scaled problems of reasonable sizes Although we took only an initial step to implement the proposed representation, the experimental results obtained in this work indicated a promising direction which should lead to scalable genetic linkage learning for the linkage learning genetic algorithm as a unimetric approach More work along this line still needs to be done from both theoretical and practical aspects From the theoretical aspect, more accurate and sophisticated models of the linkage learning process are required for further understanding the nature of genetic linkage learning Advancing our knowledge on linkage learning or building-block identification in theory may shed light on better designs of genetic algorithms On the other hand, from the practical aspect, the results presented in this work reveal a promising path for achieving scalable genetic linkage learning New representations, linkage learning mechanisms, or building-block identification procedures should be investigated, constructed, and verified for improving the performance of the linkage learning genetic algorithm References M Abramowitz and I Stegun Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables Dover Publications, New York, NY, 1972 D H Ackley A connectionist machine for genetic hill climbing Kluwer Academic, Boston, 1987 T Bă ack Selective pressure in evolutionary algorithms: A characterization of selection mechanisms Proceedings of the Sixth International Conference on Genetic Algorithms (ICGA-95), pages 2–8, 1995 J D Bagley The Behavior of Adaptive Systems Which Employ Genetic and Correlation Algorithms PhD thesis, University of Michigan, Ann Arbor, MI, 1967 (University Microfilms No 68-7556) J E Baker Adaptive selection methods for genetic algorithms Proceedings of the International Conference on Genetic Algorithms and Their Applications, pages 101–111, 1985 T Blickle and L Thiele A mathematical analysis of tournament selection Proceedings of the Sixth International Conference on Genetic Algorithms (ICGA95), pages 9–16, 1995 T Blickle and L Thiele A comparison of selection schemes used in evolutionary algorithms Evolutionary Computation, 4(4):361–394, 1996 E Cant´ u-Paz Designing Efficient and Accurate Parallel Genetic Algorithms PhD thesis, University of Illinois at Urbana-Champaign, Urbana, IL, 1999 (Also IlliGAL Report No 99017) Y.-p Chen Unpublished Experimental Data, 2002 10 Y.-p Chen and D E Goldberg Introducing start expression genes to the linkage learning genetic algorithm Proceedings of the Seventh International Conference on Parallel Problem Solving from Nature (PPSN VII), pages 351–360, 2002 (Also IlliGAL Report No 2002007) 11 Y.-p Chen and D E Goldberg An analysis of a reordering operator with tournament selection on a GA-hard problem Proceedings of Genetic and Evolutionary Computation Conference 2003 (GECCO-2003), pages 825–836, 2003 (Also IlliGAL Report No 2003003) 12 Y.-p Chen and D E Goldberg Tightness time for the linkage learning genetic algorithm Proceedings of Genetic and Evolutionary Computation Conference 2003 (GECCO-2003), pages 837–849, 2003 (Also IlliGAL Report No 2003002) 110 References 13 Y.-p Chen and D E Goldberg Convergence time for the linkage learning genetic algorithm Proceedings of the 2004 Congress on Evolutionary Computation (CEC2004), pages 39–46, 2004 (Also IlliGAL Report No 2003025) 14 Y.-p Chen and D E Goldberg Introducing subchromosome representations to the linkage learning genetic algorithms Proceedings of Genetic and Evolutionary Computation Conference 2004 (GECCO-2004), pages 971–982, 2004 (Also IlliGAL Report No 2004001) 15 Y.-p Chen and D E Goldberg Convergence time for the linkage learning genetic algorithm Evolutionary Computation, 13(3):N/A, 2005 (To appear) 16 K A De Jong An Analysis of the Behavior of a Class of Genetic Adaptive Systems PhD thesis, University of Michigan, Ann Arbor, MI, 1975 (University Microfilms No 76-9381) 17 K Deb Binary and floating-point function optimization using messy genetic algorithms PhD thesis, University of Alabama, Tuscaloosa, AL, 1991 (Also IlliGAL Report No 91004) 18 K Deb and D E Goldberg mGA in C: A messy genetic algorithm in C IlliGAL Report No 91008, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL, 1991 19 K Deb and D E Goldberg Analyzing deception in trap functions Foundations of Genetic Algorithms 2, pages 93–108, 1993 20 K Deb and D E Goldberg Sufficient conditions for deceptive and easy binary functions Annals of Mathematics and Artificial Intelligence, 10:385–408, 1994 (Also IlliGAL Report No 92001) 21 K Deb, J Horn, and D E Goldberg Multimodal deceptive functions Complex Systems, 7(2):131–153, 1993 (Also IlliGAL Report No 92003) 22 W Feller An introduction to probability theory and its applications Wiley, New York, NY, 1970 23 S Forrest and M Mitchell Relative building-block fitness and the building-block hypothesis Foundations of Genetic Algorithms 2, pages 109–126, 1993 24 D R Frantz Nonlinearities in Genetic Adaptive Search PhD thesis, University of Michigan, Ann Arbor, MI, 1972 (University Microfilms No 73-11116) 25 D E Goldberg Simple genetic algorithms and the minimal, deceptive problem In L Davis, editor, Genetic Algorithms and Simulated Annealing, chapter 6, pages 74–88 Morgan Kaufmann Publishers, Los Altos, CA, 1987 26 D E Goldberg Genetic algorithms and Walsh functions: Part I, a gentle introduction Complex Systems, 3(2):129–152, 1989 (Also TCGA Report 88006) 27 D E Goldberg Genetic algorithms and Walsh functions: Part II, deception and its analysis Complex Systems, 3(2):153–171, 1989 (Also TCGA Report 89001) 28 D E Goldberg Genetic Algorithms in Search, Optimization, and Machine Learning Addison-Wesley Publishing Co., Reading, MA, January 1989 ISBN: 0-201-15767-5 29 D E Goldberg Six steps to GA happiness, 1991 Paper presented at Oregon Graduate Institute, Beaverton, OR 30 D E Goldberg The wright brothers, genetic algorithms, and the design of complex systems In J D Schaffer, editor, Proceedings of the Symposium on Neural-Networks: Alliances and Perspectives in Senri 1993, pages 1–7, Osaka, Japan, 1993 31 D E Goldberg The race, the hurdle, and the sweet spot: Lessons from genetic algorithms for the automation of design innovation and creativity In P Bentley, References 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 111 editor, Evolutionary Design by Computers, chapter 4, pages 105–118 Morgan Kaufmann, San Mateo, CA, 1999 D E Goldberg The Design of Innovation: Lessons from and for Competent Genetic Algorithms, volume of Genetic Algorithms and Evoluationary Computation Kluwer Academic Publishers, June 2002 ISBN: 1-4020-7098-5 D E Goldberg and C L Bridges An analysis of a reordering operator on a GAhard problem Biological Cybernetics, 62:397–405, 1990 (Also TCGA Report No 88005) D E Goldberg, K Deb, and J H Clark Genetic algorithms, noise, and the sizing of populations Complex Systems, 6(4):333–362, 1992 (Also IlliGAL Report No 91010) D E Goldberg, K Deb, and J Horn Massive multimodality, deception, and genetic algorithms Proceedings of the Second International Conference on Parallel Problem Solving from Nature (PPSN II), pages 37–46, 1992 (Also IlliGAL Report No 92007) D E Goldberg, K Deb, H Kargupta, and G Harik Rapid, accurate optimization of difficult problems using fast messy genetic algorithms Proceedings of the Fifth International Conference on Genetic Algorithms (ICGA-93), pages 56–64, 1993 (Also IlliGAL Report No 93004) D E Goldberg, K Deb, and B Korb Messy genetic algorithms revisited: Studies in mixed size and scale Complex Systems, 4(4):415–444, 1990 D E Goldberg, K Deb, and D Thierens Toward a better understanding of mixing in genetic algorithms Journal of the Society of Instrument and Control Engineers, 32(1):10–16, 1993 (Also IlliGAL Report No 92009) D E Goldberg, B Korb, and K Deb Messy genetic algorithms: Motivation, analysis, and first results Complex Systems, 3(5):493–530, 1989 (Also TCGA Report No 89003) D E Goldberg and G Liepens Theory tutorial, 1991 (Tutorial presented at the 1991 International Conference on Genetic Algorithms, La Jolla, CA) D E Goldberg and R Lingle, Jr Alleles, loci, and the traveling salesman problem Proceedings of the International Conference on Genetic Algorithms and Their Applications, pages 154–159, 1985 D E Goldberg and K Sastry A practical schema theorem for genetic algorithm design and tunning Proceedings of Genetic and Evolutionary Computation Conference 2001 (GECCO-2001), pages 328–335, 2001 (Also IlliGAL Report No 2001017) D E Goldberg, K Sastry, and T Latoza On the supply of building blocks Proceedings of Genetic and Evolutionary Computation Conference 2001 (GECCO2001), pages 336–342, 2001 (Also IlliGAL Report No 2001015) J J Grefenstette and J E Baker How genetic algorithms work: A critical look at implicit parallelism Proceedings of the Third International Conference on Genetic Algorithms (ICGA-89), pages 20–27, 1989 G Harik, E Cant´ u-Paz, D E Goldberg, and B L Miller The gambler’s ruin problem, genetic algorithms, and the sizing of populations Proceedings of the 1997 IEEE International Conference on Evolutionary Computation, pages 7–12, 1997 G Harik, E Cantu´ u-Paz, D E Goldberg, and B L Miller The gambler’s ruin problem, genetic algorithms, and the sizing of populations Evolutionary Computation, 7(3):231–253, 1999 112 References 47 G R Harik Learning gene linkage to efficiently solve problems of bounded difficulty using genetic algorithms PhD thesis, University of Michigan, Ann Arbor, MI, 1997 (Also IlliGAL Report No 97005) 48 G R Harik Linkage learning via probabilistic modeling in the ECGA IlliGAL Report No 99010, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL, 1999 49 G R Harik and D E Goldberg Learning linkage Foundations of Genetic Algorithms 4, pages 247–262, 1996 (Also IlliGAL Report No 96006) 50 G R Harik and D E Goldberg Learning linkage through probabilistic expression Computer Methods in Applied Mechanics and Engineering, 186(2–4):295– 310, June 2000 51 D L Hartl and E W Jones Genetics: principles and analysis Jones and Bartlett Publishers, Sudbury, MA, 4th edition, January 1998 ISBN: 0-76370489-X 52 J H Holland Genetic algorithms and the optimal allocation of trials SIAM Journal on Computing, 2(2), 1973 53 J H Holland Adaptation in natural and artificial systems University of Michigan Press, Ann Arbor, MI, 1975 ISBN: 0-262-58111-6 54 H Kargupta SEARCH, polynomial complexity, and the fast messy genetic algorithm PhD thesis, University of Illinois at Urbana-Champaign, Urbana, IL, 1995 (Also IlliGAL Report No 95008) 55 H Kargupta The gene expression messy genetic algorithm Proceedings of the 1996 IEEE International Conference on Evolutionary Computation, pages 814– 819, 1996 56 P Larra˜ naga and J A Lozano Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation, volume of Genetic algorithms and evolutionary computation Kluwer Academic Publishers, Boston, MA, October 2001 ISBN: 0-7923-7466-5 57 J R Levenick Inserting introns improves genetic algorithm success rate: Taking a cue from biology Proceedings of the Fourth International Conference on Genetic Algorithms (ICGA-91), pages 123–127, 1991 58 F G Lobo Personal Communications, 2001 59 F G Lobo, K Deb, D E Goldberg, G R Harik, and L Wang Compressed introns in a linkage learning genetic algorithm In Proceedings of the Third Annual Conference on Genetic Programming (GP 98), pages 551–558, University of Wisconsin, Madison, WI, August 1998 Morgan Kaufmann (Also IlliGAL Report No 97010) 60 F G Lobo and G R Harik Extended compact genetic algorithm in C++ IlliGAL Report No 99016, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL, 1999 61 L D Merkle Analysis of linkage-friendly genetic algorithms PhD thesis, Air Force Institute of Technology, Air University, Albuquerque, New Mexico, 1996 (Also Technical Report No AFIT/DS/ENG 96-11) 62 B L Miller Noise, Sampling, and Efficient Genetic Algorithms PhD thesis, University of Illinois at Urbana-Champaign, Urbana, IL, May 1997 (Also IlliGAL Report No 97001) 63 B L Miller and D E Goldberg Genetic algorithms, tournament selection, and the effects of noise Complex Systems, 9(3):193–212, 1995 (Also IlliGAL Report No 95006) References 113 64 B L Miller and D E Goldberg Genetic algorithms, selection schemes, and the varying effects of noise Evolutionary Computation, 4(2):113–131, 1996 (Also IlliGAL Report No 95009) 65 H Mă uhlenbein and T Mahnig FDA - a scalable evolutionary algorithm for the optimization for the optimization of additively decomposed functions Evolutionary Computation, 7(4):353–376, 1999 66 H Mă uhlenbein and G Paaò From recombination of genes to the estimation of distributions I Binary parameters Proceedings of the Fourth International Conference on Parallel Problem Solving from Nature (PPSN IV), pages 178187, 1996 67 H Mă uhlenbein and D Schlierkamp-Voosen Predicitive models for the breeder genetic algorithm: I continuous parameter optimization Evolutionary Computation, 1(1):25–49, 1993 68 M Munetomo Linkage identification based on epistasis measures to realize efficient genetic algorithms Proceedings of the 2002 Congress on Evolutionary Computation (CEC2002), pages 1332–1337, 2002 69 M Munetomo Linkage identification with epistasis measure considering monotonicity conditions Proceedings of the 4th Asia-Pacific Conference on Simulated Evolution and Learning (SEAL2002), pages 550–554, 2002 70 M Munetomo and D E Goldberg Identifying linkage by nonlinearity check IlliGAL Report No 98012, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL, 1998 71 M Munetomo and D E Goldberg Identifying linkage groups by nonlinearity/non-monotonicity detection Proceedings of Genetic and Evolutionary Computation Conference 1999 (GECCO-99), pages 433–440, 1999 72 M Pelikan and D E Goldberg Escaping hierarchical traps with competent genetic algorithms Proceedings of Genetic and Evolutionary Computation Conference 2001 (GECCO-2001), pages 511–518, 2001 (Also IlliGAL Report No 2001003) 73 M Pelikan, D E Goldberg, and E Cant´ u-Paz BOA: The bayesian optimization algorithm Proceedings of Genetic and Evolutionary Computation Conference 1999 (GECCO-99), pages 525–532, 1999 (Also IlliGAL Report No 99003) 74 M Pelikan, D E Goldberg, and E Cant´ u-Paz Linkage problem, distribution estimation, and bayesian networks Evolutionary Computation, 8(3):311–341, 2000 (Also IlliGAL Report No 98013) 75 M Pelikan, D E Goldberg, and F G Lobo A survey of optimization by building and using probabilistic models Computational Optimization and Applications, 21(1):5–20, 2002 (Also IlliGAL Report No 99018) 76 C Reeves Using genetic algorithms with small populations Proceedings of the Fifth International Conference on Genetic Algorithms (ICGA-93), pages 92–99, 1993 77 T P Riopka Intelligent Recombination Using Genotypic Learning in a Collective Learning Genetic Algorithm PhD thesis, The George Washington University, Washington, DC, 2002 78 T P Riopka and P Bock Intelligent recombination using individual learning in a Collective Learning Genetic Algorithm Proceedings of Genetic and Evolutionary Computation Conference 2000 (GECCO-2000), pages 104–111, 2000 79 R S Rosenberg Simulation of genetic populations with biochemical properties PhD thesis, University of Michigan, Ann Arbor, MI, 1967 (University Microfilms No 67-17836) 114 References 80 A A Salman, K Mehrotra, and C K Mohan Adaptive linkage crossover Proceedings of ACM Symposium on Applied Computing (SAC’98), pages 338– 342, 1998 81 A A Salman, K Mehrotra, and C K Mohan Linkage crossover operator Proceedings of Genetic and Evolutionary Computation Conference 1999 (GECCO99), pages 564–571, 1999 82 A A Salman, K Mehrotra, and C K Mohan Linkage crossover operator Evolutionary Computation, 8(3):341–370, 2000 83 K Sastry Evaluation-relaxation schemes for genetic and evolutionary algorithms Master’s thesis, University of Illinois at Urbana-Champaign, Urbana, IL, January 2002 (Also IlliGAL Report No 2002004) 84 J D Schaffer and A Morishima An adaptive crossover distribution mechanism for genetic algorithms Proceedings of the Second International Conference on Genetic Algorithms (ICGA-87), pages 36–40, 1987 85 A Sinha Designing efficient genetic and evolutionary algorithm hybrids Master’s thesis, University of Illinois, Urbana, IL, 2003 (Also IlliGAL Report No 2003020) 86 J Smith and T C Fogarty An adaptive poly-parental recombination strategy Proceedings of AISB-95 Workshop on Evolutionary computing, pages 48–61, 1995 87 J Smith and T C Fogarty Recombination strategy adaptation via evolution of gene linkage Proceedings of the 1996 IEEE International Conference on Evolutionary Computation, pages 826–831, 1996 88 D Thierens Analysis and Design of Genetic Algorithms PhD thesis, Katholieke Universiteit Leuven, Leuven, Belgium, 1995 89 D Thierens and D E Goldberg Mixing in genetic algorithms Proceedings of the Fifth International Conference on Genetic Algorithms (ICGA-93), pages 38–45, 1993 90 D Thierens and D E Goldberg Convergence models of genetic algorithm selection schemes Proceedings of the Third International Conference on Parallel Problem Solving from Nature (PPSN III), pages 116–121, 1994 91 D Thierens and D E Goldberg Elitist recombination: An integrated selection recombination ga Proceedings of the First IEEE International Conference on Evolutionary Computation, pages 508–512, 1994 92 D Thierens and D E Goldberg Domino convergence, drift, and the temporalsalience structure of problems Proceedings of the 1998 IEEE International Conference on Evolutionary Computation, pages 535–540, 1998 93 A S Wu and R K Lindsay Empirical studies of the genetic algorithm with noncoding segments Evolutionary Computation, 3(2):121–147, 1995 94 A S Wu and R K Lindsay A survey of intron research in genetics Proceedings of the Fourth International Conference on Parallel Problem Solving from Nature (PPSN IV), pages 101–110, 1996 95 A S Wu, R K Lindsay, and M D Smith Studies on the effect of non-coding segments on the genetic algorithm Proceedings of the Sixth IEEE Conference on Tools with Artificial Intelligence, 1994 96 T.-L Yu and D E Goldberg Toward an understanding of the quality and efficiency of model building for genetic algorithms Proceedings of Genetic and Evolutionary Computation Conference 2004 (GECCO-2004), pages 367–378, 2004 (Also IlliGAL Report No 2004004) References 115 97 T.-L Yu, D E Goldberg, A Yassine, and Y.-p Chen Genetic algorithm design inspired by organizational theory: Pilot study of a dependency structure matrix driven genetic algorithm Proceedings of Artificial Neural Networks in Engineering 2003 (ANNIE 2003), pages 327–332, 2003 (Also IlliGAL Report No 2003007) 98 D Zwillinger and S Kokoska CRC Standard Probability and Statistics Tables and Formulae Chapman & Hall/CRC, Boca Raton, Florida, 2000 ISBN: 158488-059-7 Index Adaptation in Natural and Artificial Systems 18 adaptive expression 104 adaptive linkage crossover 27, 29 algorithmic improvement 27 ALinX see adaptive linkage crossover allele allele configuration 42 allele convergence 1, 41, 42 Bayesian optimization algorithm 25, 27, 29 BB see building block biological plausibility 24, 26, 28 BOA see Bayesian optimization algorithm bounded difficulty 16, 47 building block badly scaled 42 bounded difficulty complex convergence time 10 deceptive decision making 10, 12 deep different scaling 43 disruption 97, 98 exchange 17 exponentially scaled 42, 43, 45 filtering 26, 31 formation 51, 53, 55–57, 63 growth 10 hard to separate identification 17, 20, 97 low-order misleading mixing 10 order of 30 processing 21 propagation 72 salience 12 schema partition 12 separation 55, 56 size 12 supply 10 takeover time 10 uniformly scaled 43, 45 central limit theorem 13 centralized-model approach 28 chromosome CLGA see collective learning genetic algorithm coding trap 20 collective learning genetic algorithm 29 combinatorial overload 91 communication cost 25 communication latency 25 competent genetic algorithm 9, 16 computational motivations 29 continual improvement convergence time 42 convergence time model 2, 3, 63, 74, 84, 87, 90–92, 98, 101, 103, 107 cross-fertilizing type of innovation crossover 118 Index probability 8, 11 cut and splice 30 deceptive function 21 decision-making model 11 dependency structure matrix driven genetic algorithm 29 design decomposition 16, 20, 101 differentiable signal 55 differential selection of linkage 2, 97 differential signal distributed-model approach 28, 32 donor 38 DSMGA see dependency structure matrix driven genetic algorithm ECGA see extended compact genetic algorithm EDA see estimation of distribution algorithm EPE see extended probabilistic expression epoch-wise iteration 30 estimation of distribution algorithm 25, 27, 29 evolutionary process 35, 42 exchange crossover 31, 35, 38, 42 exponentially scaled problem 42 extended compact genetic algorithm 25, 27, 29 extended probabilistic expression 37, 38 facetwise model 11 factorized distribution algorithm 27, 29 fast messy genetic algorithm 26, 27, 29–32 FBB model see first-building-block model FDA see factorized distribution algorithm first-building-block model 72, 74, 75, 90 fitness fmGA see fast messy genetic algorithm fully deceptive 21, 47 fundamental intuition GA see genetic algorithm gambler’s ruin model 11, 64, 95 Gaussian distribution 13 gemGA see gene expression genetic algorithm gene gene duplication 104 gene expression genetic algorithm 25 gene expression messy genetic algorithm 29 gene migration 104 general linkage crossover 27, 29 genetic algorithm 1, competent see competent genetic algorithm design decomposition 9, 17 facet of 23 facetwise model see facetwise model fundamental intuition see fundamental intuition innovation intuition see innovation intuition linkage learning see linkage learning genetic algorithm pseudo-code of selectomutative selectorecombinative 9, 11, 17 genetic algorithms selectorecombinative 45 genetic linkage see linkage, genetic genetic linkage models 105 genotype genotypic structure 26, 35, 36, 41, 51 GLinX see general linkage crossover grafting point 38 inconsistent behavior 43 individual initialization phase 30 innovation intuition introns 31 inversion 20, 25, 27, 30 juxtapositional phase 30 LEGO see linkage evolving genetic operator LIEM see linkage identification based on epistasis measures Index LIEM2 see linkage identification with epistasis measure considering monotonicity conditions LIMD see linkage identification by non-monotonicity detection LINC see linkage identification by nonlinearity check linkage convergence 1, 41 linkage evolution 35 linkage evolving genetic operator 25, 36 linkage group 36 linkage identification based on epistasis measures 29 linkage identification by nonmonotonicity detection 29, 36 linkage identification by nonlinearity check 29, 36 linkage identification with epistasis measure considering monotonicity conditions 29 linkage learning genetic algorithm 1, 17, 24, 25, 27, 29, 35 behavior of 43 lineage of 24 performance barrier of 43 position of 24 linkage problem 2, 17, 20, 23, 30, 41, 42 linkage shift 39, 40, 42 linkage skew 39, 40, 42 linkage, genetic 1, 17, 18 definition of 17–19 degree of 21 differential selection of 2, 97 evolution see linkage evolution group see linkage group importance of 17 information 21 learning 1, 17, 20, 23, 30, 35, 41 learning mechanism 35 learning process 1, 41, 42 learning techniques 23 loose see loose linkage mechanism 38, 40, 42 phenomenon 18, 19 physical see physical linkage problem see linkage problem, tight see tight linkage 119 virtual see virtual linkage LLGA see linkage learning genetic algorithm locus ordering of 20 loose linkage 2, 19, 41 meiosis 17 meiosis-crossover process 18 messy coding 30 messy genetic algorithm 26, 27, 29, 30, 36 messy operator 30 mGA see messy genetic algorithm misnucleation 53, 54, 101 moveable gene 36, 41, 93 multimetric approach 2, 32 mutation 7, probability niching technique 43 non-coding segment 30, 31, 36, 41, 93 compress 43 non-functional gene 36 nucleation 51, 53, 57, 63, 101, 106 operator, genetic children parents ordering problem 17, 20, 30, 35 over-specified 30 partially mapped crossover 30, 31 PE see probabilistic expression phenotype physical linkage 26, 32 PMBGA see probabilistic modelbuilding genetic algorithm PMX see partially mapped crossover POI see point of interpretation point of interpretation 37, 38 multiple 43 population population-sizing model 11 decision-making model see decisionmaking model gambler’s ruin model see gambler’s ruin model primordial phase 30 120 Index probabilistic expression 31, 35–38, 41, 42, 51, 93 probabilistic model-building genetic algorithm 27, 29 probabilistically complete initialization 31 promoter 51, 56, 63, 93, 101 punctuation mark 36 punctuation marks 25 race, the time-scale comparison 1, 41 recipient 38 redundant gene 104 reordering 30 reordering operator 20 representation 6, 17, 19 Royal Road function 31 scalability 29, 101 schema partition see building block, schema partition schema theorem 2, 11, 55, 92, 97 selection ordinal scheme pressure 11 proportionate scheme tournament sequential behavior 43, 63, 65, 101, 106 shadowed gene 37, 42 subchromosome 91, 92, 101 survival-of-the-fittest tag-based approach 30 technological motivations 25 tight linkage 1, 18, 20, 21, 35 tightness time 42 tightness time model 3, 63, 83–87, 90, 101, 103, 107 trap function 47, 52, 64, 95 traveling salesman problem 31 under-specified 30 uniformly scaled problem 43 unimetric approach 2, 24, 32 unitation 47 virtual linkage 26