Articles » General Programming » Algorithms & Recipes » Neural Networks An Introduction to Encog Neural Networks for Java By JeffHeaton, 17 Jan 2010 Download source code - 306 KB Introduction This article provides a basic introduction to neural networks and neural network programming using the Encog Artificial Intelligence Framework. Encog is an AI framework that is available for both Java and Microsoft .NET. In this article, I will show you how to create and train a very basic neural network with Encog. I will likely follow this up with more complex neural network examples in the future. However, for now, this article shows you how to get started with 4.67 (4 votes) Encog. Background Encog is an advanced Artificial Intelligence Framework. Using Encog, you can create advanced neural network applications. Though Encog supports other aspects of Artificial Intelligence programming, this article will focus on neural network programming. Neural Network programming is the primary focus of Encog, as of version 2.3. Encog is released under the Lesser GNU Public License (LGPL). Encog can be downloaded from the following URL: http://www.heatonresearch.com/encog/. This is a simple introductory example that shows how to create a neural network that recognizes the XOR operator. The XOR operator is essentially the "Hello World" of the neural network world. It is often used to demonstrate a new neural network. Before I show you how to create a neural network in Encog, it is important to understand how a neural network works. Nearly all neural networks contain layers. A layer is a group of neurons that behave similarly. There are many different layer types used by the different types of neural networks that are supported by Encog. However, there are two very important layers that nearly every neural network will have. These are the input and output layers. The input layer is how you feed data to the neural network. The output layer is how you get the response back from the neural network. The input and output from the neural network are both Java double values, ordinary floating-point numbers. Both the input and output layers will have some number of neurons. This determines how many floating-point numbers the layer will deal with. The input and output layers will typically have a different number of neurons. Deciding the structure of the input and output layer is how you define the "problem" that you are trying to solve to the neural network. Let's see how we could create a neural network to perform as an XOR operator. The XOR operator requires that the two inputs be different, for the output to be true. The XOR operator's truth table can be expressed as follows: 0 XOR 0 = 0 1 XOR 0 = 1 0 XOR 1 = 1 1 XOR 1 = 0 I would like to now create a neural network that can perform this operator. Such a network will have two input neurons and one output neuron. It will accept the two operands and return the result. Using the Code We will now take a look at how the code for this example is constructed. This example was created with Encog v2.3, which is the current version of Encog at the time of this writing. The Encog 2.3 JAR file is provided with the download for this example. However, you may wish to grab the latest Encog JAR from the URL provided earlier in this article. Neural networks must be trained before they are of any use. To train this neural network, me must provide training data. The training data is the truth table for the XOR operator. The XOR has the following inputs: public static double XOR_INPUT[][] = { { 0.0, 0.0 }, { 1.0, 0.0 }, { 0.0, 1.0 }, { 1.0, 1.0 } }; These are all of the possible inputs to the XOR operator. Likewise, the expected outputs are also stored as an array. public static double XOR_IDEAL[][] = { { 0.0 }, { 1.0 }, { 1.0 }, { 0.0 } }; These two arrays will be combined to create the training data for the XOR operator. The following line of code combines these two arrays to create training data: NeuralDataSet trainingSet = new BasicNeuralDataSet(XOR_INPUT, XOR_IDEAL); We must now create a neural network. The following lines of code do this: BasicNetwork network = new BasicNetwork(); network.addLayer(new BasicLayer(new ActivationSigmoid(), true,2)); network.addLayer(new BasicLayer(new ActivationSigmoid(), true,4)); network.addLayer(new BasicLayer(new ActivationSigmoid(), true,1)); network.getStructure().finalizeStructure(); network.reset(); You can see the three layers being created. The three addLayer methods above create the three layers. The first layer is the input layer, it has two neurons. The final layer created is the output layer, it has a single neuron. The layer created in the middle is called the hidden layer. The hidden layer helps the rest of the neural network learn. More hidden neurons must be added to allow the neural network to learn more complex patterns. Four hidden neurons is more than enough for the XOR operator. Picking the number of hidden neurons is usually a process of trial and error. The call to the reset method above randomizes the weights between all of the neurons in the network. These random weights will not allow the neural network to function as an XOR operator. However, the training process, that we will see shortly, will refine these weights and encourage the neural network to produce output similar to the XOR operator. The ActivationSigmoid class specifies the type of activation that is to be used. The sigmoid function is a good choice here because we use no negative values on the XOR operator. If the network needed to recognize negative, the hyperbolic tangent activation function would be more appropriate. Encog supports a number of different activation functions, all of which have their unique uses. A training object must be created to train the neural network. Encog supports a number of different training methods. Some are used for specific neural network types. Others perform better on different types of data. The best 'general purpose' trainer for this type of neural network is called resilient propagation (RPROP). The following code creates an RPROP trainer: final Train train = new ResilientPropagation(network, trainingSet); Once the trainer has been created, we should loop through a series of iterations. Each iteration will take the neural network closer to being able to function as an XOR operator. int epoch = 1; do { train.iteration(); System.out.println("Epoch #" + epoch + " Error:" + train.getError()); epoch++; } while(train.getError() > 0.01); At each iteration, or epoch, we check the error. We continue until the error is less than 1%. The error defines how far the output of the neural network is from the ideal output defined earlier. Once the neural network has been trained satisfactorily, we will make use of it. We will feed each of the inputs into the neural network and observe the output. The following code does this: System.out.println("Neural Network Results:"); for(NeuralDataPair pair: trainingSet ) { final NeuralData output = network.compute(pair.getInput()); System.out.println(pair.getInput().getData(0) + "," + pair.getInput().getData(1) + ", actual=" + output.getData(0) + ",ideal=" + pair.getIdeal().getData(0)); } We loop through each of the training data items and present the output from the neural network as well as the actual output. The output from the program is shown here: Epoch #1 Error:0.5494587158070631 Epoch #2 Error:0.5494587158070631 Epoch #3 Error:0.5142230676643447 Epoch #4 Error:0.5005001549387983 Epoch #5 Error:0.5004690225170658 Epoch #6 Error:0.5004317073796488 Epoch #40 Error:0.026928155758252966 Epoch #41 Error:0.014605778809936418 Epoch #42 Error:0.007180166468750018 Neural Network Results: 0.0,0.0, actual=0.003348914814615583,ideal=0.0 1.0,0.0, actual=0.995118794356202,ideal=1.0 0.0,1.0, actual=0.9999200785659956,ideal=1.0 1.0,1.0, actual=8.741041885168623E-4,ideal=0.0 As you can see, the neural network loops through a number of iterations before the error percent drops below 1%. After 42 iterations, the network is trained. When you run the application, it may take more or less, as the neural network starts with random weights. Once it is done, you see the actual output from the neural network. The output from the neural network does not exactly match the ideal output. This is expected. However, the value 0.99 is very close to 1.0. Points of Interest This article demonstrated a very simple neural network application. It is massive overkill to use a neural network to create an XOR operator. There are many more complex examples provided with Encog. I will likely write further articles here demonstrating some of the more advanced features of Encog. One of the interesting features of a neural network is that it is created completely differently from a traditional program. For a traditional program, you would think about how to implement the XOR operator and create all of the necessary programming logic to do it. For a neural network, you just provide input examples and the expected outputs. It is up to the neural network to learn how to provide these expected outputs. Usually, you really have no idea how it actually learns to provide its output. This is especially true with large neural networks that may have hundreds of neurons. License Permalink | Advertise | Privacy | Mobile Web04 | 2.6.121031.1 | Last Updated 17 Jan 2010 Article Copyright 2010 by JeffHeaton Everything else Copyright © CodeProject, 1999-2012 Terms of Use This article, along with any associated source code and files, is licensed under The GNU Lesser General Public License (LGPLv3) About the Author JeffHeaton Software Developer (Senior) United States Member Jeff Heaton is an author, consultant, artificial intelligence (AI) researcher and former college instructor. Heaton has penned more than a dozen books on topics including AI, virtual worlds, spiders and bots. Heaton leads the Encog project, an open source initiative to provide an advanced neural network and bot framework for Java and C#. A Sun Certified Java Programmer and a Senior Member of the IEEE, he holds a Masters Degree in Information Management from Washington University in St. Louis. Heaton lives in St. Louis, Missouri. Comments and Discussions 4 messages have been posted for this article Visit http://www.codeproject.com/Articles/52847/An- Introduction-to-Encog-Neural-Networks-for-Java to post and view comments on this article, or click here to get a print view with messages. . » Neural Networks An Introduction to Encog Neural Networks for Java By JeffHeaton, 17 Jan 2010 Download source code - 306 KB Introduction This article provides a basic introduction to neural networks. http://www .codeproject. com/Articles/52847 /An- Introduction- to- Encog- Neural- Networks- for- Java to post and view comments on this article, or click here to get a print view with messages. . worlds, spiders and bots. Heaton leads the Encog project, an open source initiative to provide an advanced neural network and bot framework for Java and C#. A Sun Certified Java Programmer and a Senior