Genetic and Evolutionary Computation Rick Riolo W.P Worzel Mark Kotanchek Arthur Kordon Editors Genetic Programming Theory and Practice XIII Genetic and Evolutionary Computation Series Editors: David E Goldberg John R Koza More information about this series at http://www.springer.com/series/7373 Rick Riolo • W.P Worzel • Mark Kotanchek Arthur Kordon Editors Genetic Programming Theory and Practice XIII 123 Editors Rick Riolo Center for the Study of Complex Systems University of Michigan Ann Arbor, MI, USA W.P Worzel Evolution Enterprises Ann Arbor, MI, USA Arthur Kordon (Retired) Mark Kotanchek Evolved Analytics Midland, MI, USA ISSN 1932-0167 Genetic and Evolutionary Computation ISBN 978-3-319-34221-4 ISBN 978-3-319-34223-8 (eBook) DOI 10.1007/978-3-319-34223-8 Library of Congress Control Number: 2016947783 © Springer International Publishing Switzerland 2016 This work is subject to copyright All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed The use of general descriptive names, registered names, trademarks, service marks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made Printed on acid-free paper This Springer imprint is published by Springer Nature The registered company is Springer International Publishing AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Dedication This book is dedicated to John Henry Holland, whose work and kindness touched his students, his colleagues, his friends, and many others who knew him only from his writings PREQUEL Before the blank–full of fresh grain scent and flecked like oatmeal woven flat– canvas, before the blank canvas is stretched or strained tight as an egg, before then– sketch It doesn’t catch commencement: it won’t hook the scene like a rug, or strategize too far ahead It isn’t chess It doesn’t expect the homestretch or the check Each line braves rejection of the every, edits restless all into a space that’s still the space of least commitment, distilling latitudes in draft It would domesticate the feral dusk and stockpile dawn It would be commensurate, but settles for less, settles prairies in its channels Great plains roar and waterfall, yawn and frost v vi Dedication between the lines From hunger, from blank and black, it models erotic stopped tornadoes, the high relief of trees In advance or retreat, in terraced dynamics–its bets are hedged–with no deadbolt perspective Its point of view? One with the twister in vista glide, and the cricket in the ditch, with riverrain, and turbines’ trace Inside the flux of flesh and trunk and cloudy come, within the latent marrow of the egg, the amber traveling waves is where its vantage lies Entering the tornado’s core, entering the cricket waltzed by storm– to confiscate the shifting give and represent the without which —Alice Fulton Foreword In 2003, Carl Simon asked Rick Riolo and me to organize a workshop on genetic programming (GP) We decided to bring together people interested in the theory of GP with people whose main focus was applying GP to “real-world” problems and seeing what happens We also included daily keynote speakers who were in general not familiar with GP but who had challenging ideas in the areas of computer science, commercial applications, and biological sciences It was originally planned as a one-off workshop, but after the first workshop, there was a lot of enthusiasm to continue it, and so the Genetic Programming Theory and Practice (GPTP) workshop became an annual event at the University of Michigan in Ann Arbor This book is the 13th such book written by the attendees of GPTP Over the years, we have had an amazing series of participants working in a wide range of fields who have refined and expanded the understanding and application of GP It was entirely fitting then that the first keynote speaker at GPTP was John Holland For those who may not be familiar with John and his work, he is widely credited with being one of the originators of genetic algorithms and was a founder of the Santa Fe Institute, the Center for the Study of Complex Systems at the University of Michigan, and other key research centers focused on interdisciplinary studies He received what may have been the first PhD in computer science (from the U of M) in 1959, and his work in complexity theory was central to the development of complexity as an area of serious study John was a polymath who came of age in the heady times of computer science when everything is not only seemed possible but inevitable John never lost the enthusiasm of those days and passed it along to his students, shared it with his colleagues, and brought it to GPTP As the chain of GPTP workshops unrolled, John would stop in occasionally if there was a speaker he wanted to hear or a topic that intrigued him Though he never worked with GP himself, he had a knack for going to the heart of a problem and suggesting new ideas and questions that opened new vistas for exploration Perhaps more importantly, GPTP is infused with the spirit of the Center for the Study of Complex Systems (CSCS) and the BACH group in particular As such, it is multidisciplinary and mathematically inclined and looks to find grand patterns vii viii Foreword from simple principles This is really no surprise as many of the attendees at GPTP have been students or colleagues of John’s I believe that this world view is also the reason for the longevity of the workshop as its focus is not about this or that technique per se but is about the deeper workings of GP and how to manage it in the application to different problems At the memorial held for John at the University of Michigan in October of this year, Stephanie Forrest spoke about what it was like to be John’s graduate student, and she described his approach to advising as being the practice of “benign neglect.” As a student, she often found this difficult but said she had come to appreciate its virtues and had adopted it with her own students I believe that GPTP has benefited from the same quality of benign neglect as CSCS has given us time, space, and support to pursue a complex but fascinating subject for over a decade without bothering about how the workshop was structured, who we invited or how, or even if we published the results This freedom has become one of the hallmarks of GPTP, and every year, the participants comment on how much they enjoy the workshop as a result For more on John’s amazing career, the reader is encouraged to read the Santa Fe Institute’s obituary at http://www.santafe.edu/news/item/in-memoriamjohn-holland/ and, more importantly, to read his numerous, seminal papers and books as he was truly one of the leading founders of our discipline Ann Arbor, MI, USA November 2015 W.P Worzel Preface This book is about the Thirteenth Workshop on Genetic Programming Theory and Practice, a workshop held this year from May 14 to 16, 2015, at the University of Michigan under the auspices of the Center for the Study of Complex Systems The workshop is a forum for theorists and users of genetic programming to come together and share ideas, insights, and observations It is designed to be speculative in nature by encouraging participants to discuss ideas or results that are not necessarily ready for peer-reviewed publication To facilitate these goals, the time allotted for presentations is longer than is typical at most conferences, and there is also more time devoted for discussion For example, presenters usually have 40 to present their ideas and take questions, and then, before each break, there is open discussion on the ideas presented in a session Additionally, at the end of each day, there is a review of the entire day and the ideas and themes that have emerged during the sessions Looking back at the schedule, in a typical day, there was 240 of presentation and 55 of discussion or fully 19 % of the time spent in open discussion In addition to the regular sessions, each day starts with a keynote speaker who gets a full hour of presentation and 10 of Q&A By design, the keynotes are generally not about genetic programming but come from a related field or an application area that may be fertile ground for GP This year, the first keynote speaker was Dave Ackley from the University of New Mexico who delivered and addressed the topic titled “A Requiem for Determinism.” This provocative presentation argued that from the beginning of modern computing, people such as John von Neumann argued that hardware could not be relied on to work perfectly in all cases—just because of the nature of electronics in that they will fail some number of times These days, the growth of complexity of software has added to this problem Modern software depends on the user’s ability to reboot the system when things get out of sync or when hardware fail Ackley argues that the correct response (as foreseen by von Neumann) is to make systems that continue to function even when the system nominally fails Dave went on to suggest that given that GP takes its cues from nature, we should consider incorporating methods that survive “mistakes” in execution ix x Preface The second keynote speaker was Larry Burns, who had been an executive at General Motors and is now a consultant with Google on their autonomous vehicle project Larry’s talk was about the development of autonomous vehicles and the likely arc of adoption of autonomous vehicles, but he went on to discuss the fact that technology cannot be thought of in isolation and in particular that it exists in a cultural context and is co-dependent on the infrastructure As engineers, we tend to think only of the technology we are developing, but Larry made a strong case for thinking about work in a larger context The third keynote was Julian Togelius on “Games Playing Themselves: Challenges and Opportunities for AI Research in Digital Games.” Games have been at the center of AI development since the beginning of modern computers Turing mused on chess-playing computers Samuel’s checker playing system could be argued to be the beginning of neural nets, at least on an engineering level Deep Thought attracted worldwide attention when it beat Garry Kasparov, the then-reigning world chess champion Julian posed a number of interesting questions relating to AI, particularly about the human traits of curiosity and what it means to “like” something He turned the usual dynamic of interaction around by asking the questions whether games could be “curious” about people and later asked whether computers could “like” games or even “like” making good games It was an interesting reversal on the usual questions about AI work and was an interesting discussion in the context of GP While the keynotes at the workshop were provocative and interesting, the chapters in this book are the core of GPTP The first chapter by Kommenda et al is titled “Evolving Simple Symbolic Regression Models by Multi-objective Genetic Programming.” This interesting chapter revisits the question of evaluating the complexity of GP expressions as part of the fitness measure for evolution Most previous efforts focused either on the structural complexity of the expression or an expensive calculation of subtrees and their components This chapter proposes a lightweight semantic metric which lends itself to efficient multi-modal fitness calculations without using input data The second chapter, by Elyasaf et al., titled “Learning Heuristics for Mining RNA Sequence-Structure Motifs” explores the difficult problem correlating RNA sequences to biological functionality This is a critical problem to finding and understanding biological mechanisms derived from specific RNA sequences The authors use GP to create hyper-heuristics that find cliques within the graphs of RNA Though the chapter only describes the approach and does not show concrete results, it is a clever approach to a complex problem, and we look forward to seeing results in a future GPTP The next chapter, by de Melo and Banzhaf, “Kaizen Programming for Feature Construction for Classification” adopts the Japanese practice of Kaizen (roughly, continuous improvement) to GP in the domain of classification problems In this case, they use GP to generate new ideas in the Kaizen algorithm where in this case “ideas” mean classifier rules that are recursively improved, removed, or refined It is an interesting idea that takes advantage of GP’s ability to generate novel partial solutions and then refine them using the Kaizen approach ... Switzerland 2016 R Riolo et al (eds.), Genetic Programming Theory and Practice XIII, Genetic and Evolutionary Computation, DOI 10.1007/978-3-319-34223-8_2 21 22 A Elyasaf et al Keywords Genetic. .. Kotanchek M (eds) Genetic programming theory and practice XI Genetic and evolutionary computation Springer, New York Breiman L, Friedman J, Stone CJ, Olshen RA (1984) Classification and regression... Springer International Publishing Switzerland 2016 R Riolo et al (eds.), Genetic Programming Theory and Practice XIII, Genetic and Evolutionary Computation, DOI 10.1007/978-3-319-34223-8_1 M Kommenda