Articles » General Programming » Algorithms & Recipes » Neural Networks Calculating the weights for an ADALINE using the Delta Rule By Ben Aldhouse, 25 Jun 2007 Download source code - 20.5 KB Download compiled application - 9.23 KB Introduction This is actually one of the simplest C++ programs I have ever completed. There really isn't much to it. It won't be of interest to either accomplished C++ programmers, or to experts in Neural Networks. I hope however, it may be of some interest to people who are roughly at the same stage as me in learning at Neural Networks, i.e., pretty much at the start. It is also a fairly easy Visual C++ program to write. In other articles I have written, I have described how to build the application up with a fair amount of detail, starting with describing how to deal with the Application Wizard. In this article, I will assume the reader knows how to create an application shell using the Application Wizard, and how to place controls on a form and attach variables to those controls, and how to make an application handler function for a button control. For those who aren't interested in the coding but would like to play with the calculator, the compiled application is available by clicking on the link above. It is easy to use. You enter your starting weights, training set, desired output, and value for eta, and press 'Go'. The program should stop running pretty quickly. The compiled program will run for 4.14 (3 votes) one hundred passes, which, as far as I can tell, is fully adequate. If you scroll down the output box until the numbers in column 'y' (the output) start to correspond with the numbers in column 'd' (the desired output), then the values shown in the output box under 'w0', 'w1', and 'w2' on that line are the correct weights for the network. You can test this by replacing the 'Starting weights' with these values and pressing 'Go' again. This time, every value in 'y' should equal the corresponding value in 'd' right from the start of the output. Background I have been looking at Phil Picton's book 'Neural Networks', having had the book for some time and not getting through it as fast as I would like. It has only been in the last few days, however, that I had the bright idea of working through the iterative process used for finding the weights on an ADALINE. I wrote out the numbers on a piece of paper, and did the sums in my head between serving customers at the supermarket where I work. I felt pleased because I felt that I had got the process more or less right. However, the process wasn't fast enough for me to get satisfactory results. People in shops can be very demanding, you know. Besides, I suck at mental arithmetic. So, on a Saturday night, I set to and wrote this rather undemanding little program. I am not going to try and describe the ADALINE here. I am only a learner. I will provide the appropriate Google search for the reader here - ADALINE Neural Network description. I am sure there is something useful out there. If not, then you could do worse than to buy Phil Picton's book. Using the code OK, very briefly, I set up an SDI using inheritance from CFormView. On the dialog, I added a number of text boxes for the input, and one for the output, plus one button to set everything going. All the code of any interest is in the event handler for the button CAdalineView::OnButtonGo() and one other member function I created called CAdalineView::adaline( ). void CAdalineView::OnButtonGo() { // TODO: Add your control notification handler code here float d1, d2, d3, d4, eta; float y1, y2, y3, y4; float w10, w11, w12; float w20, w21, w22; float w30, w31, w32; float w40, w41, w42; float x10, x11, x12; float x20, x21, x22; float x30, x31, x32; float x40, x41, x42; float net1, net2, net3, net4; float dw10, dw11, dw12; float dw20, dw21, dw22; float dw30, dw31, dw32; float dw40, dw41, dw42; eta = atof(m_strEta); d1 = atof(m_strD1); d2 = atof(m_strD2); d3 = atof(m_strD3); d4 = atof(m_strD4); CString str=""; m_strOutput=""; ////Starting data w10 = atof(m_strW0); w11 = atof(m_strW1); w12 = atof(m_strW2); int passes = 0; int matches = 0; x10 = atof(m_strX10); x11 = atof(m_strX11); x12 = atof(m_strX12); x20 = atof(m_strX20); x21 = atof(m_strX21); x22 = atof(m_strX22); x30 = atof(m_strX30); x31 = atof(m_strX31); x32 = atof(m_strX32); x40 = atof(m_strX40); x41 = atof(m_strX41); x42 = atof(m_strX42); while (passes <= 100){ ///// input pattern 1 adaline(x10, x11, x12, w10, w11, w12, d1, net1, y1, dw10, dw11, dw12, eta); //// change weights from pass1 to pass 2 w20 = w10+dw10; w21 = w11+dw11; w22 = w12+dw12; ///// input pattern 2 adaline(x20, x21, x22, w20, w21, w22, d2, net2, y2, dw20, dw21, dw22, eta); //// change weights from pass2 to pass 3 w30 = w20+dw20; w31 = w21+dw21; w32 = w22+dw22; ///// input pattern 3 adaline(x30, x31, x32, w30, w31, w32, d3, net3, y3, dw30, dw31, dw32, eta); //// change weights from pass3 to pass 4 w40 = w30+dw30; w41 = w31+dw31; w42 = w32+dw32; ///// input pattern 4 adaline(x40, x41, x42, w40, w41, w42, d4, net4, y4, dw40, dw41, dw42, eta); UpdateData(FALSE); //// change weights from pass4 to pass 1 w10 = w40+dw40; w11 = w41+dw41; w12 = w42+dw42; passes++; } } So, this function takes the data from the edit boxes and handles the iteration process, passing the data for each of the four input patterns to the adaline function in turn. The weights are adjusted here, also. However, the values by which the weights are adjusted are calculated in the adaline function. void CAdalineView::adaline(float x0, float x1, float x2, float w0, float w1, float w2, float d, float net, float y, float &dw0, float &dw1, float &dw2, float eta) { CString str = ""; //Calculate 'net' net = x0*w0 + x1*w1 + x2*w2; //Calculate output value by passing 'net' through //a hard-limiter if (net >= 0) y = 1; else y = -1; //Here the 'Delta Rule' is applied dw0 = eta*x0*(d - net); dw1 = eta*x1*(d - net); dw2 = eta*x2*(d - net); //And the output is shuffled off to the appropriate text box //inputs str.Format("%.2f\t%.2f\t%.2f\t",x0,x1,x2); m_strOutput += str; //weights str.Format("%.2f\t%.2f\t%.2f\t",w0,w1,w2); m_strOutput += str; //d, net, y str.Format("%.2f\t%.2f\t%.2f\t",d,net,y); m_strOutput += str; //weight adjustments str.Format("%.2f\t%.2f\t%.2f\t\r\n",dw0,dw1,dw2); m_strOutput += str; } Note that dw1, dw2, and dw3 are passed 'by reference' (indicated by the ampersand prefix). It is both easier and more efficient to do this, allowing the variables to be assigned new values in the called function and have those new values instantly available in the calling function (because both functions are looking at the same memory locations). This precludes the various overheads incurred by passing the variables 'by value'. The other parameters are left as 'by value' passes since they are not required to provide information back to the calling function. If you are looking for more efficiency and want to keep with the principle of minimum privilege (i.e., the function being called can't interfere with the variable in the calling function) that 'by value' passes provide, then you could have all of the variables passed 'by reference', and declare all of them, other than the ones you specifically want to manipulate, in the calling function (i.e., dw0, dw1, as dw2) as 'const'. The following snippet is where the Delta Rule is applied: //Here the 'Delta Rule' is applied dw0 = eta*x0*(d - net); dw1 = eta*x1*(d - net); dw2 = eta*x2*(d - net); and here the output 'y' is calculated by passing the value in 'net' through a 'hard-limiter': //Calculate output value by passing 'net' through //a hard-limiter if (net >= 0) y = 1; else y = -1; which, somehow, turns out to be a very friendly looking couple of lines of code to execute such a severe sounding function. Points of interest If I hadn't written this program, and carried on trying to do the calculations on a piece of paper, I would have been doing it till kingdom come, since I had been subtracting the weight changes (the dw values) from the weights (the w values). My program showed very efficiently that this led nowhere fast. After a moment or two of thinking about it, I changed the code (in the OnButtonGo function) to add the weight changes, and thankfully, it worked. w20 = w10+dw10; w21 = w11+dw11; w22 = w12+dw12; License This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below. A list of licenses authors might use can be found here About the Author Ben Aldhouse United Kingdom Member random and twisted Permalink | Advertise | Privacy | Mobile Web03 | 2.6.121031.1 | Last Updated 25 Jun 2007 Article Copyright 2007 by Ben Aldhouse Everything else Copyright © CodeProject, 1999-2012 Terms of Use Comments and Discussions 0 messages have been posted for this article Visit http://www.codeproject.com/Articles/19333/Calculating- the-weights-for-an-ADALINE-using-the-D to post and view comments on this article, or click here to get a print view with messages. . 199 9-2 012 Terms of Use Comments and Discussions 0 messages have been posted for this article Visit http://www .codeproject. com/Articles/19333 /Calculating- the- weights- for- an- ADALINE- using -the- D. set up an SDI using inheritance from CFormView. On the dialog, I added a number of text boxes for the input, and one for the output, plus one button to set everything going. All the code of any. this function takes the data from the edit boxes and handles the iteration process, passing the data for each of the four input patterns to the adaline function in turn. The weights are adjusted