Artificial Neural Network Identification And Control Of The Inverted Pendulum

Artificial Neural Network identification and control of the inverted pendulum Tim Callinan August 2003... Feedforward and Recurrent neural networks are used to model the inverted pendul

Artificial Neural Network identification and control

of the inverted pendulum

Tim Callinan August 2003

3

Abstract

This project takes the area of Artificial Neural Networks (ANN) and applies it to the inverted pendulum control problem The inverted pendulum is typically used to benchmark new control techniques, as it’s a highly non-linear unstable system Neural networks have unique characteristics, which enable them to control non-linear systems Feedforward and Recurrent neural networks are used to model the inverted pendulum Multi-output online identification was also researched A neuro-controller for the inverted pendulum was developed Traditional control methods were utilized to develop a control law to stabilize the inverted pendulum A feedforward network was trained to mimic the control law Tbe neuro-control shows that if a disturbance occurs in the system, the neural network learns to counteract this disturbance Finally the knowledge learned in identification and control was applied to the real time inverted pendulum rig An online adaptive neural network was developed to model the real time system

4

Table of Contents

1 Introduction 5

Outline of the document 6

2 Inverted Pendulum 7

3 Artificial Neural Networks 18

Advantages of ANN’s 19

Types of Learning 20

Neural network structures 20

Multi-layered perceptrons 22

4 System Identification 24

System identification procedure 25

Linear identification of the system 28

Non-Linear identification of the system 36

Non-linear Identification using neural networks 37

Multi-output identification 46

5 Neural control of the inverted pendulum 51

Neural-control in simulink: 58

6 Real-time identification and control 67

7 Conclusions 75

Summary 75

Scope for future work 79

8 Bibliography 80

5

The process used in this project is the inverted pendulum system The inverted pendulum is a highly nonlinear and open-loop unstable system This means that standard linear techniques cannot model the nonlinear dynamics of the system When the system is simulated the pendulum falls over quickly The characteristics of the inverted pendulum make identification and control more challenging There are two main aims of the project The first is to develop

an accurate model of the inverted pendulum system using neural networks The second aim is

to develop a neural network controller which determines the correct control action to stabilize the system, but can also learn from experience

System identification is the procedure that develops models of a dynamic system based on the input and output signals from the system The input and output data must show some of the dynamics of the process The parameters of the model are adjusted until the output from the model is similar to the output of the real system In order to develop an accurate model of the inverted pendulum, different methods (linear and nonlinear) of identification will be tested One of the problems encountered early in the project is collecting experimental data from the inverted pendulum system The output data from the unstable system does not show enough information or dynamics of the system Feedback controllers are developed which stabilize the system before identification can take place

Neural networks have shown great progress in identification of nonlinear systems There are certain characteristics in ANN which assist them in identifying complex nonlinear systems ANN are made up of many nonlinear elements and this gives them an advantage over linear techniques in modelling nonlinear systems ANN are trained by adaptive learning, the network

‘learns’ how to do tasks, perform functions based on the data given for training The knowledge learned during training is stored in the synaptic weights The standard ANN structures (feedforward and recurrent) are both used to model the inverted pendulum

6

The main task of this project is to design a neural network controller which keeps the pendulum system stabilized There are 3 main types of neural control – supervised, direct inverse and unsupervised

Supervised learning uses an existing controller or human feedback in training the neural network In order to train the neural network to imitate an existing controller a vector of inputs and control targets from the controller must be collected With supervised control, a neural network could be trained to imitate a robust controller The robust controller can operate correctly, if the process operates around a certain point The neuro-controller operates similarly to the robust controller but can also adapt if any disturbance occurs in the system Direct inverse control does not require an existing controller in training A neural network is trained to model the inverse of the process The neural network is cascaded with the process Theoretically if the inverse model is very accurate, the nonlinearities in the ANN will cancel out the nonlinearities in the process

Outl ne of the document

Chapter 2 details the research on the inverted pendulum system The dynamic system equations (linear and nonlinear) are derived The simulink models of the linear and nonlinear systems are developed The development of the feedback controllers to stabilize the system is also discussed Chapter 3 covers the theory, structure and operation of artificial neural networks Chapter 4 covers the whole area of system identification The procedure of system identification is discussed first Linear identification techniques are applied to the linear system Nonlinear identification using neural networks is then reported Chapter 5 details the development of the neuro-controller Chapter 6 discusses the real time identification and control using the inverted pendulum rig Finally Chapter 7 provides a summary of the work discussion of the results and scope for future work

7

The inverted pendulum system is a classic control problem that is used in universities around the world It is a suitable process to test prototype controllers due to its high non-linearities and lack of stability The system consists of an inverted pole hinged on a cart which is free to move in the x direction In this chapter, the dynamical equations of the system will be derived, the model will be developed in simulink and basic controllers will be developed The aim of developing an inverted pendulum in simulink is that the developed model will have the same characteristics as the actual process It will be possible to test each of the prototype controllers

in the simulink environment Before the inverted pendulum model can be developed in simulink, the system dynamical equations will be derived using ‘Lagrange Equations’ [1] The Lagrangian equations are one of many methods of determining the system equations Using this method it is possible to derive dynamical system equations for a complicated mechanical system such as the inverted pendulum Figure 1 is a free-bodied diagram of the pendulum system

The Lagrange equations use the kinetic and potential energy in the system to determine the dynamical equations of the cart-pole system

M – Mass of the cart

m – mass of the pole

l – length of the pole

f – control force

Fig 1: Free body diagram of the inverted pendulum system

8

The kinetic energy of the system is the sum of the kinetic energies of each mass The kinetic energy, T of the cart is 1

The pole can move in both the horizontal and vertical directions so the pole kinetic energy is

From the free body diagram y and 2 z are equal to 2

The total kinetic energy, T of the system is equal to

Equation 3 and 5 are inputted into equation 7 to give equation 8

The potential energy, V of the system is stored in the pendulum so

The Lagrangian function is

)(

2

2 2 2 2

•

•+

= m y z T

2

2 2 2 2

2

T T

2

1 2

1

θ θ

y y m y

M T

++

)(

2

mgl ml

y ml

y m M V

f ml

ml y

m

M + ) + cos θ θ − θ sin θ =

0 sin

sin

cos θ y − ml θ y θ + ml2θ − mgl θ =

ml & & & &

Some of the modelling and control techniques involved in the project are linear so these equations must be linearized It is possible to linearize these equations by approximating cosθ=1 and sinθ=0 It is assumed that θ is kept small The quadratic terms are also negligible Therefore the two linear system equations are

θ M

mg M

L L dt d

(M m y ml y

L

θ θ

sin

mgl L

(Eq.19)

(Eq.20) (Eq.12)

10

At this stage, a set of equations (linear & non-linear) describing the inverted pendulum have been developed The next stage is constructing a simulnk model of the inverted pendulum system There is no procedure for developing simulink models from dynamical state equations The diagram below is the linear pendulum model This model is constructed using integrators, gain blocks, etc The model (Fig 2) is simply a simulink representation of the linear state equations

The non-linear pendulum system (Fig 4) is shown in the next page The non-linear system, even though it is more complicated is developed in a similar manner Both models are large so

it is possible to encapsulate them in subsystem blocks shown below (Fig 3) Both the models are set-up using a mask The mask makes it possible to change the values of m, l, g, etc for different simulations The mass of the cart, M is set to 1.2 Kg, the mass of the pendulum is set

to 0.11 Kg and the length of the pendulum is 0.4 meters These figures are taken from the real time inverted pendulum rig

Fig 2 : Simulink model of the linear pendulum system

Fig 3 : Simulink blocks of the pendulum systems

11 The following simulink diagram is the non-linear pendulum model

Fig 4 : Simulink model of the nonlinear pendulum system

-100 -90 -80 -70 -60 -50 -40 -30 -20 -10 0

Fig 5: Open loop response of the inverted pendulum

Pendulum

angle, deg

Time

13

A full state feedback controller is developed to stabilize the linear pendulum system The linear system could have been stabilized using many different methods (PID,etc) The full-state feedback controller stabilizes the system by positioning the closed loop poles in the stable region The simulink model with controller is shown below (Fig.6)

The linear pendulum system is simulated, the angle of the pendulum is shown below (Fig.7) The stabilized system with controller keeps the pendulum angle stable The pendulum can be simulated for longer times The data is also of better quality for system identification purposes

-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8

Fig 6 : Simulink diagram of Linear Pendulum and controller

Fig 7: Closed loop response of the inverted pendulum with controller

Pendulum

angle, deg

Time

14

Developing a controller for the non-linear pendulum is more difficult Linear control techniques such as the PID, full-state feedback were tested but had no success in controlling the non-linear pendulum A feedback linearisation controller was developed to control the non-linear pendulum system Feedback linearisation cancels the non-linearities in the pendulum system so that the closed loop system is more linear

The following equations are a control law developed for the inverted pendulum controller The first four equations (Eq 21-24) are entered into the main equation The main equation (Eq 25) calculates the required force, U to keep the pendulum stable

For the simulations M, m, l, g are set to the values of the pendulum model The following numeric values are used: M = 1.2 Kg, m = 0.1 Kg, l = 0.4 m, g = 9.81 m/s, k = 25, 1 k = 10, 2

1

C = 1, C = 2.6 Also 2 x = 0 meters and d θ = 0 rad, which are the desired position of the cart d

and angle of the pendulum respectively For details on all the parameters see [4] A simulink model of the above control law was developed and is shown in Figure 8

θ

sin 4

4

3 1

m M f

1 2

1 2

1 1 2

2 h k ( ) k c ( x x ) c x f h

+

− +

=

•

•

θ θ

θ

(Eq.22)

(Eq.23)

(Eq.25) (Eq.21)

(Eq.24)

16

The following diagram (Fig.9) shows the set-up of the non-linear pendulum with control law

Figure 10 is the closed loop pendulum angle plotted by matlab The closed loop response is stable and shows that the control law is working

0 100 200 300 400 500 600 -5

-4 -3 -2 -1 0 1 2 3

4x 10

-3

Fig 9: Simulink diagram of the nonlinear system with control law

Fig 10: Closed loop response of the nonlinear pendulum with controller

Pendulum

angle, deg

Time

17

The linear and nonlinear models of the cart-pole system have been developed and simulated It was found that the system is open loop unstable For accurate system identification the process must be stable, because of this, standard feedback controllers were developed and tested The next chapter in the report discusses the theory and operation of artificial neural networks

18

The science of artificial neural networks is based on the neuron In order to understand the structure of artificial networks, the basic elements of the neuron should be understood Neurons are the fundamental elements in the central nervous system The diagram below (Fig 11) shows the components of a neuron [5]

A neuron is made up of 3 main parts -dendrites, cell body and axon The dendrites receive signals coming from the neighbouring neurons The dendrites send their signals to the body of the cell The cell body contains the nucleus of the neuron If the sum of the received signals is greater than a threshold value, the neuron fires by sending an electrical pulse along the axon to the next neuron The following model is based on the components of the biological neuron (Fig 12) The inputs X0-X3 represent the dendrites Each input is multiplied by weights W0-W3 The output of the neuron model, Y is a function, F of the summation of the input signals

Fig 11: The diagram shows the basic elements of a neuron

Fig 12: Diagram of neuron model

19

Advantages of ANN’s

1 The main advantage of neural networks is that it is possible to train a neural network to perform a particular function by adjusting the values of connections (weights) between elements For example, if we wanted to train a neuron model to approximate a specific function, the weights that multiply each input signal will be updated until the output from the neuron is similar to the function

2 Neural networks are composed of elements operating in parallel Parallel processing allows increased speed of calculation compared to slower sequential processing

3 Artificial neural networks (ANN) have memory The memory in neural networks corresponds to the weights in the neurons Neural networks can be trained offline and then transferred into a process where adaptive learning takes place In our case, a neural network controller could be trained to control an inverted pendulum system offline say

in the simulink environment After training, the network weights are set The ANN is placed in a feedback loop with the actual process The network will adapt the weights to improve performance as it controls the pendulum system

The main disadvantage of ANN is they operate as black boxes The rules of operation in neural networks are completely unknown It is not possible to convert the neural structure into known model structures such as ARMAX, etc Another disadvantage is the amount of time taken to train networks It can take considerable time to train an ANN for certain functions

Fig 13: Diagram shows the parallelism of neural networks

Neural network structures

There are 3 main types of ANN structures -single layer feedforward network, multi-layer feedforward network and recurrent networks [7] The most common type of single layer feedforward network is the perceptron Other types of single layer networks are based on the perceptron model The details of the perceptron are shown below (Fig 14)

Inputs to the perceptron are individually weighted and then summed The perceptron computes the output as a function F of the sum The activation function, F is needed to introduce non-linearities into the network This makes multi-layer networks powerful in representing nonlinear functions

Fig 14: Diagram of the perceptron model

21

There are 3 main types of activation function -tan-sigmoid, log-sigmoid and linear [8] Different activation functions affect the performance of an ANN

The output from the perceptron is

The weights are dynamically updated using the back propagation algorithm The difference between the target output and the actual output (error) is calculated

The errors are back propagated through the layers and the weight changes are made The formula for adjusting the weights is

Once the weights are adjusted, the feed-forward process is repeated The weights are adapted until the error between the target and actual output is low The approximation of the function improves as the error decreases Single-layer feedforward networks are useful when the data

to be trained is linearly separable If the data we are trying to model is not linearly separable or the function has complex mappings, the simple perceptron will have trouble trying to model the function adequately

Log-sigmoid function Tan-sigmoid function Linear function

(Eq.26)

(Eq.27)

(Eq.28)

]) [ ].

[ ( ]

] [ ] [ ]

] [ ].

[ ] [ ] 1

22

Multi-layered perceptrons

Neural networks can have several layers There are 2 main types of multi-layer networks- feedforward and recurrent In feedforward networks the direction of signals is from input to output, there is no feedback in the layers The diagram below (Fig 15) shows a 3-layered feedforward network

Increasing the number of neurons in the hidden layer or adding more hidden layers to the network allows the network to deal with more complex functions Cybenko’s theorem states that, “A feedforward neural network with a sufficiently large number of hidden neurons with continuous and differentiable transfer functions can approximate any continuous function over

a closed interval.” [9] The weights in MLP’s are updated using the backpropagation learning [10] There are two passes before the weights are updated

In the first pass (forward pass) the outputs of all neurons are calculated by multiplying the input vector by the weights The error is calculated for each of the output layer neurons

In the backward pass, the error is passed back through the network layer by layer The weights are adjusted according to the gradient decent rule, so that the actual output of the MLP moves closer to the desired output A momentum term could be added which increases the learning rate with stability

Fig 16: Diagram of a recurrent neural network

In ut a er

Hid en a er

Output a er

24

System identification is the process of developing a mathematical model of a dynamic system based on the input and output data from the actual process [12] This means it is possible to sample the input and output signals of a system and using this data generate a mathematical model An important stage in control system design is the development of a mathematical model of the system to be controlled In order to develop a controller, it must be possible to analyse the system to be controlled and this is done using a mathematical model Another advantage of system identification is evident if the process is changed or modified System identification allows the real system to be altered without having to calculate the dynamical equations and model the parameters again

System identification is concerned with developing models The diagram below (Fig 17) shows the inputs and output of a system

The mathematical model in this case is the black box, it describes the relationship between the input and output signals The inverted pendulum system is a non-linear process To adequately model it, non-linear methods using neural networks must be used Previous studies in system identification have demonstrated that neural networks are successful in modelling many non-linear systems [13] Before neural networks are investigated for identification, linear techniques such as auto regressive with exogenous input (ARX) and auto regressive moving average with exogenous input (ARMAX) will be applied to the linear inverted pendulum model

Fig 17: System showing input, disturbance and output signals

OUTPUT INPUT

DISTURBANCE

25

System dentification procedure

Basically system identification is achieved by adjusting the parameters of the model until the model output is similar to the output of the real system Below is a diagram (Fig 18) explaining the system identification procedure [14]

There are three main steps in the system identification procedure

1 The first step is to generate some experimental input/output data from the process we are trying to model In the case of the inverted pendulum system this would be the input force on the cart and the output pendulum angle

2 The next step is to choose a model structure to use For example the following model structure is the ARX

3 The parameters A and B will be adjusted until this model output is similar to the output

of the process In identification, there is no perfect model structure to use Models can

be developed using engineering intuition or a priori knowledge of the process we are

trying to model

) ( ) ( )

( y t Bu t e t

Fig 18: Diagram of the system identification procedure

(Eq.29)

26

The best solution in choosing a model structure is to pick a number of different models, test them all and use the model which yields the closest output to the process The standard linear models (ARX,ARMAX,etc) used in system identification were researched [15] Below is the diagram for the ARX model and the ARX model equation (Fig 19) The ARX is a simple linear difference equation, which describes the input-output relationship The input-output relationship is modelled using a transfer function block B/A It is assumed that the noise spectrum and the input-output model have the same characteristic dynamics This could attribute to some modelling error

The ARMAX model contains an extra C parameter in the noise spectrum model (Fig 20)

This gives the ARMAX model more accuracy than the ARX The input-output block and the noise spectrum block still have the same denominator

) ( )

( )

A

C t u A

B t

) (

1 ) ( )

A t

u A

B t

Noise spectrum model Input-output model

Fig 19: ARX model Eq.30: ARX equation

Fig 20: ARMAX model Eg.31 ARMAX equation

27

The next type of linear model is the output-error model (Fig.21) The main difference between this model and the previous models is that the output-error assumes that the disturbances are white noise so there is no noise spectrum model The input-output relationship is defined in the transfer block B/F

The last model used in linear system identification is called the Box-Jenkins model (Fig 22) This model has separate transfer functions for the input-output relationship and the noise spectrum This is an advantage compared to the ARMAX models where the noise model and input-output relationship have the same denominator

The next stage in the identification procedure is to generate the parameters of the model Matlab uses the least squares algorithm update the model parameters The least square algorithm takes the model structure and input/output data from the process and estimates the model parameters It also generates the residuals, which is the error between the model and the process If the residuals are too high, another model structure could be used or the experimental data might not show the true dynamics of the system to be modelled There are many different linear methods of estimating the parameters of the model Least squares is the main algorithm used in system identification The next section of the report details the linear identification experiments on the inverted pendulum system

) ( ) ( )

( u t e t

F

B t

) ( )

( )

D

C t

u F

B t

Fig 21: Output-error model Eq 32: OE equation

Fig 22: Box-Jenkins model Eq.33: BJ equation

28

Linear dentification of the system

In order to generate an ARX, ARMAX or Box-Jenkins model of the inverted pendulum system, the input-output data from the linear system must be collected The diagram below (Fig 23) shows the simulink model with the feedback control system The noise input is an excitation signal It is used to obtain a unique input-output response Using ‘to workspace’ blocks the data is exported to the Matlab console The data is split up into estimation and validation data Half of the generated data is used to generate the model and half will be used

to test the performance of the model

The diagram above shows that the pendulum block is a SIMO (single-input multi-output) system Previous linear models that were developed are SISO (single-input single-output) systems which model the pendulum angle θ The first type of models to be developed are

ARX

Fig 23: Linear Pendulum with feedback controller

29

The following code is an example of ARX estimation in matlab The input/output data is split

up into estimation and validation data The ARX function uses least-squares to estimate the parameters of the model The model can be converted into transfer function format using the command th2tf (tetha to transfer function)

The nn matrix defines the orders and delay of the ARX model

nn = [na nb nk]

na = number of parameters to be estimated in the denominator

nb = number of parameters in the numerator

nk = time delay in the model

To develop the best ARX model possible, different orders of na, nb will be tested Note that ARX221 means na=2, nb=2 nk=1 The following table (table 1) shows the different orders of na,nb used and the mean squared error between the model output and the target output The compare function is used to compare the model output with the validation data The models were tested with two sets of validation data The second validation data was generated using a different initial seed in the input signal The mean squared error increased slightly using the new data but this shows that the generated models can predict the output of the process The number of parameters in the models were increased It was expected that as the complexity of the models increased, the mean squared error would decrease This was not the case, from the results the models with the lowest mean squared error are ARX221 and ARX421

Table 1: Error associated with ARX models

The following two plots (Fig.24,25) show the model output with the actual process output

The mean squared error between the two is low Also the models seem to track the shape of the actual output These results for the ARX models are good but the best method to test the quality of a model is to generate a transfer function block of the model in simulink and input a noise signal with a different seed (Fig 26)

Blue: Model output, Black: Measured output Output # 1 Fit: 0.0051914

Fig 24: ARX 421 model output with validation data 1 Fig 25: ARX 421 model output with validation data 2

Pole

Angle

rad.

Pole Angle rad.

This is a controller that just keeps the pendulum stable but more of the process dynamics can

be seen A de-tuned PID controller is developed by adjusting the P, I, D parameters until the output becomes more unstable

Fig 26: Testing the linear models using transfer function block

-0.1 -0.08

-0.06

-0.04

-0.02

0 0.02 0.04 0.06 0.08 0.1

Red: PID Blue: Detuned PID

Fig 27: Closed-loop response with normal and de-tuned controllers

Fig 28: Simulink model using the system identification toolbox

Pendulum

Angle, rad

Time

33

The models developed using ARX and ARMAX couldn’t adequately predict the pendulum system output even using the de-tuned controller All orders of ARX, ARMAX models were tested The plot below is an example output from one of the ARX models (Fig 29)

Many of the journals on closed-loop identification indicate the best model structures to use are Box-Jenkins and Output-Error [16] ARX and ARMAX both make an assumption that the noise spectrum model and the input-output model have the same characteristic dynamics This explains why the ARX transfer functions were unable to model the linear inverted pendulum Models were generated using the BJ and OE simulink blocks These models were converted into transfer function blocks and simulated (Fig 30)

Fig 29: ARX model response compared to the real output

Fig 30: Testing the accuracy of the Box-Jenkins and Output-error models

Blue: Model output Black: Process output

Pendulum

Angle, rad

-9 -8 -7 -6 -5 -4 -3 -2 -1 0

1x 10

5 Output # 1 Fit: 140322.7567

34

The following tables show the results from the Box–Jenkins and Output-Error models The BJ/OE parameters were changed for each simulation to determine the best model The model BJ53331 has the lowest mean squared error The Output-Error models have a slightly higher error than the Box-Jenkins models These results indicate that

1 When trying to model an unstable system a feedback controller must be used to keep the system stable If the controller is de-tuned this will allow more of the pendulum dynamics to be seen This will make the model more accurate

2 The Box-jenkins/Output error models are the only structures that can adequately model the pendulum using the closed loop data

3 The best way to test the quality of a model is to construct a transfer block of the model and simulate it using a different initial input seed

Time (secs) Actual Output (Red Line) vs The Predicted Model output (Blue Line)

-5

0 5

x 10Actual Output (Red Line) vs The Predicted Model output (Blue Line)-3

Fig 31 : Box Jenkins Model [5 3 3 3 1] tested on validation data

Fig 32 : Output Error [2 2 1] tested on validation data

Pendulum

Angle, rad

Pendulum

Angle, rad

36

Non-Linear dentification of the system

The previous system identification is based on linear systems To achieve a better approximation of the inverted pendulum system, the non-linear model must be used Before neural networks are utilized in identification, linear methods of identification such as Box-Jenkins were applied to the nonlinear pendulum The diagram below shows the nonlinear pendulum with the feedback controller (Fig.33) The input and output signals from the system are directed to the BJ algorithm which develops the model

It is expected that linear methods such as BJ will not be able to capture the nonlinearitys of the pendulum, however as the diagram below shows (Fig.34) the Box-Jenkins model can actually model the nonlinear function This is because the process is in a closed loop The control law keeps the pendulum stable by removing some of the non-linearities of the pendulum system

Even using a detuned controller the Box-Jenkins can model the dynamics well

-0.4

-0.2

0 0.2

0.4

0.6

Time (secs) Actual Output (Red Line) vs The Predicted Model output (Blue Line)

Error In Predicted Model

Fig 33: Linear identification on the nonlinear system

Fig 34: The Box-Jenkins output Vs Process output

Pendulum

Angle, rad

37

Non-l near dentification using neural networks

This section discusses the different methods of identifying the pendulum process using neural networks The most common method of neural network identification is called forward modelling (Fig 35) [17] During training both the process and ANN receive the same input, the outputs from the ANN and process are compared, this error signal is used to update the weights in the ANN This is an example of supervised learning- the teacher (pendulum system) provides target values for the learner (the neural network)

There are 2 main types of networks (Feed-forward and Elman) that will be used for identification In order to provide a set of targets for the network to learn, the simulink model

with feedback control is used (Fig 36)

Fig 35: Neural network forward modelling method

Fig 36: The non-linear pendulum provides the neural targets

38

To emphasise the pendulum dynamics, the feedback controller was de-tuned The controller is based on a control law which basically cancels any non-linearities in the pendulum system The mass of the cart, M is set to 0.2 kg (In the control law) This makes the controller use less control force making the pendulum go more unstable More of the pendulum dynamics will be seen at the output (Fig 38)

Initially single-input single-output networks were developed, the input being the control force and the output pole angle The first type of neural network to be developed are feed-forward Using matlab it is possible to develop multi-layer perceptrons (MLP)

The following matlab code creates a feed-forward network

net = newff([-10 10],[4 1],{'tansig' 'purelin'},'trainlm');

Fig 37: Closed loop response with normal controller Fig 38: Closed loop response with detuned controller

Pole

Angle

rad

Pole Angle rad

39

The type of training used is also set here These networks use the back-propagation learning rule to update the weights The number of epochs for this example is set to 400 During training, the input vector will be passed through the neural network and the weights will be adjusted 400 times The learning rate of the network is also set The ‘train’ function adjusts the weights of the network so the output of the network will be similar to the non-linear pendulum

The following diagram shows the training of the MLP (Fig 39) At the start of the training, the error between the network and the pendulum is high As the number of epochs increases the mean squared error decreases As the curve begins to converge, very little learning is taking place Looking at the training diagram below it is possible to determine the correct amount of epochs for training

40

When the training is finished, the neural network is exported to simulink using the ‘gensim’ command The diagram below (Fig 40) shows the neural network in simulink Notice that the neural model and the process receive the same input signal To adequately test the quality of the model the initial seed of the input signal must be changed

The quality of the neural model is tested by calculating the MSE (mean squared error) The MSE gives a good indication of the accuracy of the model The MSE between the model and the process should be low A model could have a low MSE but not predict any of the dynamics of the pendulum system The output from the model and process is plotted to compare the dynamics Basically we want to see whether the model predicts the movement of the inverted pendulum Increasing the number of hidden layer neurons allows for more complex functions to be modelled During testing, neural networks with a range of hidden layer neurons were simulated It was expected that as the number of hidden neurons increased the more accurate the model would become The following graphs show the process output plotted against the model outputs (Fig 41-44) The initial seed is kept the same for all simulations to show the difference between the models

Fig 40: The quality of the model is tested in simulink