!"#$%&'()*+, )*%&./0,1",2"),+&3)%&45)*, This%book%is%copyrighted%by%Heaton%Research,%Inc.%%If%you % obtained%this% book%from%a%source%other%than%Heaton%Research%please%contact%us%at% support@heatonresearch.com.% % 6""7,8&)/*, Programming%Neural%Network%s%with%% Encog%2%in%Java% 9:6;, 1604390077%–%Rev%1%(March%2010)% <5)("%, Jeff%Heaton% =>6""7,?%&@*, $19.99%(USD)% % % % % Programming Neural Networks with Encog 2 in Java Programming Neural Networks with Encog 2 in Java By Jeff Heaton Heaton Research, Inc. St. Louis, MO USA vi Programming Neural Networks with Encog 2 in Java Publisher: Heaton Research, Inc Programming Neural Networks with Encog 2 in Java March, 2010 Author: Jeff Heaton Editor: WordsRU.com Cover Art: Carrie Spear ISBN’s for all Editions: 1-60439-007-7, Softcover 1-60439-011-5, Adobe PDF e-book Copyright © 2010 by Heaton Research Inc., 1734 Clarkson Rd. #107, Chesterfield, MO 63017-4976. World rights reserved. The author(s) created reusable code in this publication expressly for reuse by readers. Heaton Research, Inc. grants readers permission to reuse the code found in this publication or downloaded from our website so long as (author(s)) are attributed in any application containing the reusable code and the source code itself is never redistributed, posted online by electronic transmission, sold or commercially exploited as a stand-alone product. Aside from this specific exception concerning reusable code, no part of this publication may be stored in a retrieval system, transmitted, or reproduced in any way, including, but not limited to photo copy, photograph, magnetic, or other record, without prior agreement and written permission of the publisher. Heaton Research, Encog, the Encog Logo and the Heaton Research logo are all trademarks of Heaton Research, Inc., in the United States and/or other countries. TRADEMARKS: Heaton Research has attempted through out this book to distinguish proprietary trademarks from descriptive terms by following the capitalization style used by the manufacturer. The author and publisher have made their best efforts to prepare this book, so the content is based upon the final release of software whenever possible. Portions of the manuscript may be based upon pre-release versions suppled by software manufacturer(s). The author and the publisher make no representation or warranties of any kind with regard to the completeness or accuracy of the contents herein and accept no liability of any kind including but not limited to performance, vii merchantability, fitness for any particular purpose, or any losses or damages of any kind caused or alleged to be caused directly or indirectly from this book. Manufactured in the United States of America. SOFTWARE LICENSE AGREEMENT: TERMS AND CONDITIONS The media and/or any online materials accompanying this book that are available now or in the future contain programs and/or text files (the “Software”) to be used in connection with the book. Heaton Research, Inc. hereby grants to you a license to use and distribute software programs that make use of the compiled binary form of this book’s source code. You may not redistribute the source code contained in this book, without the written permission of Heaton Research, Inc. Your purchase, acceptance, or use of the Software will constitute your acceptance of such terms. The Software compilation is the property of Heaton Research, Inc. unless otherwise indicated and is protected by copyright to Heaton Research, Inc. or other copyright owner(s) as indicated in the media files (the “Owner(s)”). You are hereby granted a license to use and distribute the Software for your personal, noncommercial use only. You may not reproduce, sell, distribute, publish, circulate, or commercially exploit the Software, or any portion thereof, without the written consent of Heaton Research, Inc. and the specific copyright owner(s) of any component software included on this media. In the event that the Software or components include specific license requirements or end-user agreements, statements of condition, disclaimers, limitations or warranties (“End-User License”), those End-User Licenses supersede the terms and conditions herein as to that particular Software component. Your purchase, acceptance, or use of the Software will constitute your acceptance of such End-User Licenses. By purchase, use or acceptance of the Software you further agree to comply with all export laws and regulations of the United States as such laws and regulations may exist from time to time. viii Programming Neural Networks with Encog 2 in Java SOFTWARE SUPPORT Components of the supplemental Software and any offers associated with them may be supported by the specific Owner(s) of that material but they are not supported by Heaton Research, Inc Information regarding any available support may be obtained from the Owner(s) using the information provided in the appropriate README files or listed elsewhere on the media. Should the manufacturer(s) or other Owner(s) cease to offer support or decline to honor any offer, Heaton Research, Inc. bears no responsibility. This notice concerning support for the Software is provided for your information only. Heaton Research, Inc. is not the agent or principal of the Owner(s), and Heaton Research, Inc. is in no way responsible for providing any support for the Software, nor is it liable or responsible for any support provided, or not provided, by the Owner(s). WARRANTY Heaton Research, Inc. warrants the enclosed media to be free of physical defects for a period of ninety (90) days after purchase. The Software is not available from Heaton Research, Inc. in any other form or media than that enclosed herein or posted to www.heatonresearch.com. If you discover a defect in the media during this warranty period, you may obtain a replacement of identical format at no charge by sending the defective media, postage prepaid, with proof of purchase to: Heaton Research, Inc. Customer Support Department 1734 Clarkson Rd #107 Chesterfield, MO 63017-4976 Web: www.heatonresearch.com E-Mail: support@heatonresearch.com DISCLAIMER Heaton Research, Inc. makes no warranty or representation, either expressed or implied, with respect to the Software or its [...]... http://www.heatonresearch.com/forum Introduction xxv 26 Programming Neural Networks with Encog 2 in Java Chapter 1: Introduction to Encog 27 Chapter 1: Introduction to Encog The Encog Framework What is a Neural Network? Using a Neural Network Training a Neural Network Artificial neural networks are programming techniques that attempt to emulate the human brain's biological neural networks Artificial neural networks (ANNs)... 393 Counter-Propagation Neural Networks 399 Where to Go from Here 414 Appendix A: Installing and Using Encog 419 Installing Encog 419 xvi Programming Neural Networks with Encog 2 in Java Compiling the Encog Core 421 Compiling and Executing Encog Examples 422 Using Encog with the Eclipse IDE 424 Appendix B: Example Locations... xiv Programming Neural Networks with Encog 2 in Java Querying the Neural Network 1 12 Generating Code 114 Chapter 5: Propagation Training 119 Understanding Propagation Training 119 Propagation Training with Encog 122 Propagation and Multithreading 136 Chapter 6: Obtaining Data for Encog 147 Where to Get Data for Neural Networks ... 23 0 Examining the Lunar Lander Simulator 23 5 Training the Neural Pilot 24 7 Using the Training Set Score Class 25 1 Chapter 9: Unsupervised Training Methods 25 7 The Structure and Training of a SOM 25 8 Implementing the Colors SOM in Encog 26 5 Chapter 10: Using Temporal Data 27 7 xv How a Predictive Neural Network Works 27 7 Using... stated in appropriate files x Programming Neural Networks with Encog 2 in Java xi This book is dedicated to my wonderful wife, Tracy The first year of marriage has been great; I look forward to many more xii Programming Neural Networks with Encog 2 in Java xiii Table of Contents Table of Contents 13 Introduction 19 The History of Encog 19 Problem Solving with Neural. .. code in Java, Visual Basic or C# The Encog Workbench can be downloaded from the above URL 28 Programming Neural Networks with Encog 2 in Java What is a Neural Network? We will begin by examining what exactly a neural network is A simple feedforward neural network can be seen in Figure 1.1 This diagram was created with the Encog Workbench It is not just a diagram; this is an actual functioning neural. .. with Neural Networks 20 Structure of the Book 22 Chapter 1: Introduction to Encog 27 What is a Neural Network? 28 Using a Neural Network 32 Chapter 2: Building Encog Neural Networks 47 What are Layers and Synapses? 47 Understanding Encog Layers 48 Understanding Encog Synapses 54 Understanding Neural Logic... only way to train a neural network This chapter introduces simulated annealing and genetic algorithms as training techniques for Encog networks You are also shown how to create hybrid training algorithms Supervised training is not the only training option Chapter 9, “Unsupervised Training Methods” shows how to use unsupervised training with Encog Unsupervised training occurs when a neural network is... Basis Function Neural Network 450 Recurrent Self-Organizing Map 451 Self-Organizing Map 4 52 Glossary 455 Index 467 xvii xviii Programming Neural Networks with Encog 2 in Java Introduction xix Introduction Encog is an Artificial Intelligence (AI) Framework for Java and Net Though Encog supports several areas of AI outside of neural networks, the... what I would cover in an introduction book The goal of my “Introduction to Neural Networks with Java/ C#” is to teach someone how to implement basic neural networks of their own The goal of this book is to teach someone to use Encog to create more complex neural xx Programming Neural Networks with Encog 2 in Java network structures without the need to know how the underlying neural network code actually . USA vi Programming Neural Networks with Encog 2 in Java Publisher: Heaton Research, Inc Programming Neural Networks with Encog 2 in Java March, 20 10 Author: Jeff Heaton. 419 xvi Programming Neural Networks with Encog 2 in Java Compiling the Encog Core 421 Compiling and Executing Encog Examples 422 Using Encog with the Eclipse IDE 424 Appendix B: Example. with Encog 2 in Java Querying the Neural Network 1 12 Generating Code 114 Chapter 5: Propagation Training 119 Understanding Propagation Training 119 Propagation Training with Encog 122