Representations for genetic and evolutionary algorithms 2nd edition 2006 ISBN354025059X

Representations for Genetic and Evolutionary Algorithms Franz Rothlauf Representations for Genetic and Evolutionary Algorithms ABC Dr Franz Rothlauf Universität Mannheim 68131 Mannheim Germany E-mail: rothlauf@uni-mannheim.de Library of Congress Control Number: 2005936356 ISBN-10 3-540-25059-X Springer Berlin Heidelberg New York ISBN-13 978-3-540-25059-3 Springer Berlin Heidelberg New York This work is subject to copyright All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer Violations are liable for prosecution under the German Copyright Law Springer is a part of Springer Science+Business Media springer.com c Springer-Verlag Berlin Heidelberg 2006 Printed in The Netherlands The use of general descriptive names, registered names, trademarks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use AE Typesetting: by the author and TechBooks using a Springer LT X macro package Cover design: Erich Kirchner, Heidelberg Printed on acid-free paper SPIN: 11370550 89/TechBooks 543210 Fă ur meine Eltern Elisabeth und Alfons Rothlauf Preface Preface to the Second Edition I have been very pleased to see how well the first edition of this book has been accepted and used by its readers I have received fantastic feedback telling me that people use it as an inspiration for their own work, give it to colleagues or students, or use it for preparing lectures and classes about representations I want to thank you all for using the material presented in this book and for developing more efficient and powerful heuristic optimization methods You will find this second edition of the book completely revised and extended The goal of the revisions and extensions was to make it easier for the reader to understand the main points and to get a more thorough knowledge of the design of high-quality representations For example, I want to draw your attention to Chap where you find the core of the book I have extended and improved the sections about redundancy and locality of representations adding new material and experiments and trying to draw a more comprehensive picture In particular, the introduction of synonymity for redundant encodings in Sect 3.1 and the integration of locality and redundancy issues in Sect 3.3 are worth having a closer look at it These new concepts have been used throughout the work and have made it possible to better understand a variety of different representation issues The chapters about tree representations have been reorganized such that they explicitly distinguish between direct and indirect representations This distinction – including a new analysis of the edge-sets, which is a direct encoding for trees – emphasizes that the developed representation framework is not only helpful for analysis and design of representations, but also for operators The design of proper search operators is at the core of direct representations and the new sections demonstrate how to analyze the influence of such encodings on the performance of genetic and evolutionary algorithms (GEAs) Finally, the experiments presented in Chap have been completely revised considering new representations and giving a better understanding of the influence of tree representations on the performance of GEAs VIII Preface I would like to take this opportunity to thank everyone who took the time to share their thoughts on the text with me – all these comments were helpful in improving the book Special thanks to Kati for her support in preparing this work As with the first edition, my purpose will be fulfilled if you find this book helpful for building more efficient heuristic optimization methods, if you find it inspiring for your research, or if it is a help for you teaching students about the importance and influence of representations Mannheim August 2005 Franz Rothlauf Preface to the First Edition This book is about representations for genetic and evolutionary algorithms (GEAs) In writing it, I have tried to demonstrate the important role of representations for an efficient use of genetics and evolutionary optimization methods Although, experience often shows that the choice of a proper representation is crucial for GEA’s success, there are few theoretical models that describe how representations influence GEAs behavior This book aims to resolve this unsettled situation It presents theoretical models describing the effect of different types of representations and applies them to binary representations of integers and tree representations The book is designed for people who want to learn some theory about how representations influence GEA performance and for those who want to see how this theory can be applied to representations in the real world The book is based on my dissertation with the title “Towards a Theory of Representations for Genetic and Evolutionary Algorithms: Development of Basic Concepts and their Application to Binary and Tree Representations” To make the book easier to read for a larger audience some chapters are extended and many explanations are more detailed During the writing of the book many people from various backgrounds (economics, computer science and engineering) had a look at the work and pushed me to present it in a way that is accessible to a diverse audience Therefore, also people that are not familiar to GEAs should be able to get the basic ideas of the book To understand the theoretical models describing the influence of representations on GEA performance I expect college-level mathematics like elementary notions of counting, probability theory and algebra I tried to minimize the mathematical background required for understanding the core lessons of the book and to give detailed explanations on complex theoretical subjects Furthermore, I expect the reader to have no particular knowledge of genetics and define all genetic terminology and concepts in the text The influence of Preface IX integer and tree representations on GEA performance does not necessarily require a complete understanding of the elements of representation theory but is also accessible for people who not want to bother too much with theory The book is split up into two large parts The first presents theoretical models describing the effects of representations on GEA performance The second part uses the theory for the analysis and design of representations After the first two introductory chapters, theoretical models are presented on how redundant representations, exponentially scaled representations and the locality/distance distortion of a representation influence GEA performance In Chap the theory is used for formulating a time-quality framework Consequently, in Chap 5, the theoretical models are used for analyzing the performance differences between binary representations of integers Finally, the framework is used in Chap 6, Chap 7, and Chap for the analysis of existing tree representations as well as the design of new tree representations In the appendix common test instances for the optimal communication spanning tree problems are summarized Acknowledgments First of all, I would like to thank my parents for always providing me with a comfortable home environment I have learned to love the wonders of the world and what the important things in life are Furthermore, I would like to say many thanks to my two advisors, Dr Armin Heinzl and Dr David E Goldberg They did not only help me a lot with my work, but also had a large impact on my private life Dr Armin Heinzl helped me to manage my life in Bayreuth and always guided me in the right direction in my research He was a great teacher and I was able to learn many important things from him I am grateful to him for creating an environment that allowed me to write this book Dr David E Goldberg had a large influence on my research life He taught me many things which I needed in my research and I would never have been able to write this thesis without his help and guidance During my time here in Bayreuth, my colleagues in the department have always been a great help to overcome the troubles of daily university life I especially want to thank Michael Zapf, Lars Brehm, Jens Dibbern, Monika Fortmă uhler, Torsten O Paulussen, Jă urgen Gerstacker, Axel Pă urckhauer, Thomas Schoberth, Stefan Hocke, and Frederik Loos During my time here, Wolfgang Gă uttler and Tobias Grosche were not only work colleagues, but also good friends I want to thank them for the good time I had and the interesting discussions During the last three years during which I spent time at IlliGAL I met many people who have had a great impact on my life First of all, I would like to thank David E Goldberg and the Department of General Engineering for giving me the opportunity to stay there so often Then, I want to say thank you X Preface to the folks at IlliGAL I was able to work together with It was always a really great pleasure I especially want to thank Erick Cant´ u-Paz, Fernando Lobo, Dimitri Knjazew, Clarissa van Hoyweghen, Martin Butz, Martin Pelikan, and Kumara Sastry It was not only a pleasure working together with them but over time they have become really good friends My stays at IlliGAL would not have been possible without their help Finally, I want to thank the people who were involved in the writing of this book First of all I want to thank Kumara Sastry and Martin Pelikan again They helped me a lot and had a large impact on my work The discussions with Martin were great and Kumara often impressed me with his expansive knowledge about GEAs Then, I want to say thanks to Fernando Lobo and Torsten O Paulussen They gave me great feedback and helped me to clarify my thoughts Furthermore, Katrin Appel and Kati Sternberg were a great help in writing this dissertation Last but not least I want to thank Anna Wolf Anna is a great proofreader and I would not have been able to write a book in readable English without her help Finally, I want to say “thank you” to Kati Now I will hopefully have more time for you Bayreuth January 2002 Franz Rothlauf Foreword to the First Edition It is both personally and intellectually pleasing for me to write a foreword to this work In January 1999 I received a brief e-mail from a PhD student at the University of Bayreuth asking if he might visit the Illinois Genetic Algorithms Laboratory (IlliGAL) I did not know this student, Franz Rothlauf, but something about the tone of his note suggested a sharp, eager mind connected to a cheerful, positive nature I checked out Franz’s references, invited him to Illinois for a first visit, and my early feelings were soon proven correct Franz’s various visits to the lab brought both smiles to the faces of IlliGAL labbies and important progress to a critical area of genetic algorithm inquiry It was great fun to work with Franz and it was exciting to watch this work take shape In the remainder, I briefly highlight the contributions of this work to our state of knowledge In the field of genetic and evolutionary algorithms (GEAs), much theory and empirical study has been heaped upon operators and test problems, but problem representation has often been taken as a given In this book, Franz breaks with this tradition and seriously studies a number of critical elements of a theory of GEA representation and applies them to the careful empirical study of (a) a number of important idealized test functions and (b) problems of commercial import Not only is Franz creative in what he has chosen to study, he also has been innovative in how he performs his work In GEAs – as elsewhere – there appears sometimes to be a firewall separating theory and practice This is not new, and even Bacon commented on this phenomenon with his famous metaphor of the spiders (men of dogmas), the ants (men of practice), and the bees (transformers of theory to practice) In this work, Franz is one of Bacon’s bees, taking applicable theory of representation and carrying it to practice in a manner that (1) illuminates the theory and (2) answers the questions of importance to a practitioner This book is original in many respects, so it is difficult to single out any one of its many accomplishments I believe five items deserve particular comment: Decomposition of the representation problem Analysis of redundancy ... representations for genetic and evolutionary algorithms Although the importance of choosing proper representations for the performance of genetic and evolutionary algorithms is already recognized (Caruana and. . .Representations for Genetic and Evolutionary Algorithms Franz Rothlauf Representations for Genetic and Evolutionary Algorithms ABC Dr Franz Rothlauf Universität... representations for genetic and evolutionary algorithms The chapter provides the basis and definitions which are essential for understanding the content of this work Genetic and evolutionary algorithms