Dự án cuối cùng: Kiểm soát EEG thiết bị điện sử dụng thuật toán mạng thần kinh trên STM32,In the process to implement the project certain inevitable mistakes, we hope to receive your comments. Thank you parents, friends,etc. we have been the spiritual source of motivation for us to complete the subject. Finally, congratulations to the relationship between HCMUTE and RMUTL becomes better and better.
Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 Thanks Firstly, we would like to thank to teachers, everyone in Ho Chi Minh City University of Technical and Education (HCMUTE) has created best conditions for us the opportunity to exchange schooling at Rajamangala University of Technology Lanna (RMUTL) Thanks to the enthusiastic assistance of RMUTL teachers, especially Dr Nopadon Maneetien and many Thai friends They help us to complete successfully project “EEG Control Electrical Devices Using Neural Network Algorithm On STM32” In the process to implement the project certain inevitable mistakes, we hope to receive your comments Thank you parents, friends,etc we have been the spiritual source of motivation for us to complete the subject Finally, congratulations to the relationship between HCMUTE and RMUTL becomes better and better Comment ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… …………………………………………………………………………………………… Chiang Mai ……………………… Group of students: Âu Văn Bằng Nguyễn Đức Tài Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 Contents Chapter 1: Introduction 1.1 The origin and significance of the project 1.2 The purpose of the project 1.3 The scope of the project 1.4 The benefits of the system Chapter 2: The theory and research 2.1 The principle of project .3 2.2 Theory of project .3 2.2.1 Overview of ARM 2.2.2 EEG and Brainwaves .6 2.2.3 Neurosky mind wave mobile 2.2.4 Classification algorithms .9 2.2.5 LCD (Liquid Crystal Display) 16x2 10 2.2.6 Relay 13 2.2.7 OPTO PC817 .13 2.2.8 Module Bluetooth HC05 15 2.2.9 ESP 8266 .16 Chapter 3: Design 18 3.1 Introduction .18 3.2 Block diagram 18 3.3 Design 19 3.3.1 Brainwave block 19 3.3.2 Webserver block 19 3.3.3 Bluetooth block 20 3.3.4 ESP8266 Block 21 Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 3.3.5 Display Block 21 3.3.6 Control Device Block 22 3.3.7 Center processing block .24 3.3.8 Power block 26 Chapter 4: Implementation 27 4.1 Make PCB .27 4.1.1 Required components 27 4.1.2 Draw PCB 28 4.1.3 Completed circuit 29 4.2 Flow chart 29 4.3 KeilC 32 4.4 Web server interface 32 4.4.1 The requirements of the web server interface 32 4.4.2 Programs to build webserver .32 4.5 Test 35 Chapter 5: Results and Future Works 36 5.1 Results .36 5.2 Future Works 36 References 37 Addendum 38 Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 List of Figures Figure 2.1 Hardware block diagram Figure 2.2 Neurosky Mindwave mobile Figure 2.3 LCD 16x2 11 Figure 2.4 Timing chart LCD 12 Figure 2.5 Relay 5VDC 13 Figure 2.6 PC817 14 Figure 2.7 Module HC05 15 Figure 2.8 Module ESP8266 16 Figure 3.1 Diagram block .18 Figure 3.2 Bluetooth block .20 Figure 3.3 ESP8266 Block 21 Figure 3.4 Display Block 22 Figure 3.5 Control Device Block 22 Figure 3.6 Process block 25 Figure 4.1 PCB .28 Figure 4.2 3D 28 Figure 4.3 Completed circuit 29 Figure 4.4 Main flow chart .30 Figure 4.5 Neural network flow chart .31 Figure 4.6 Icon keilc 32 Figure 4.7 Start xampp for run web offline .33 Figure 4.8 Create new file .33 Figure 4.9 Selcet the PHP programming language 34 Figure 4.10 Save file path .34 Figure 4.11 Web interface completed 35 Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 List of Tables Table 2.1 Pin LCD 11 Table 2.2 Commands LCD 12 Table 2.3 Feature of PC817 .14 Table 4.1 Components 27 Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 Chapter 1: Introduction 1.1 The origin and significance of the project These days, there is a rapid increasing of the global economic development as well as technology explosion, people tend to enhance the standard of living based on hi-tech devices and smart inventions, this leads to scientists and technology specialists must propose projects as well as look for the new approaches for meeting people’s demand As a result, IoT-Internet of Things, which is commonly used a term in recent year, sets a signal for the 4th industrial revolution In fact, there is a vast range of invented smart systems which support the general public in many fields such as manufacturing, agriculture and health care Also, people are able to operate those systems in numerous ways and one of them is a Mind-Controlled method, which is a technology for manipulating objects via brainwaves We find it’s interesting between IoT and this one playing a crucial role in life quality, therefore, we choose the project called “Controlling internet-connected devices through brainwaves” “Controlling internet-connected devices through brainwaves” is capable of supporting users control electronics devices in the house without moving the body Especially, this is a light point for paralysis victims can take care themselves, even though nobody around them Implementing this tool include a lot of methodologies such as FPGA, ARM microcontrollers, etc In this project, we use ARM microcontrollers to create this one 1.2 The purpose of the project Understand how to operate the module to read the brainwaves and then handle signals in order to return Construct a web server to update the control signals as well as allow users to control the system by pushing buttons on the screen Research and build controlling internet-connected devices through brainwaves with modes-controlled, namely brainwaves, website and Android app Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 1.3 The scope of the project Control on/off two devices Using mind wave mobile Using power 3.3 and 5v 1.4 The benefits of the system Support those who is paralysis Control robot, computer and other devices by brainwaves Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 Chapter 2: The theory and research 2.1 The principle of project The principle of project are based on the brainwave-receive, analysis, classification, control and sending data to web server There are numerous classification algorithm for EEG such as Neural Networks, Nonlinear Bayesian classifiers, … However, Neural Networks are the most popular algorithm for its high performance 2.2 Theory of project 2.2.1 Overview of ARM Family of ARM These days, there is a vast range of type of ARM such as STM32F1, STM32F4, STM32F7, etc However, each version has distinctive standards about memory, speed analysis and so on Take STM32F103 as an example, which uses ARM 32-bit Cortex-M3 CPU Core, includes 64 or 128 Kbytes of Flash memory, 20 Kbytes of SRAM and 72 MHz maximum of the frequency Another instance is STM32F4, which uses 32-bit ARM Cortex-M4 with FPU core, includes 1-Mbyte Flash memory, 192-Kbyte RAM and 168 MHz maximum of the frequency In the contrast, as the requirement of the subject's brain waves handling neural network algorithms need an ARM kit with relative speed to run well and test Kit needs an ergonomic design with easy usage and low cost Therefore, we choose the Kit, namely STM32F407VGT6 for this subject Introduce STM32F407VGT6 The STM32F4DISCOVERY Discovery kit allows users to easily develop applications with the STM32F407 high performance microcontroller with ARM® Cortex®-M4 32-bit core It includes everything required either for beginners or for experienced users to get quickly started Based on the STM32F407VGT6, it includes an ST-LINK/V2 or ST-LINK/V2-A embedded debug tool, two ST MEMS digital accelerometers, a digital microphone, one audio DAC with integrated class D speaker driver, LEDs and push buttons and an USB Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 OTG micro-AB connector To expand the functionality of the STM32F4DISCOVERY Discovery kit with the Ethernet connectivity, LCD display and more, visit the www.st.com/stm32f4dis-expansion webpage The STM32F4DISCOVERY Discovery kit comes with the STM32 comprehensive software HAL library, together with various packaged software examples Features STM32F407VGT6 microcontroller featuring 32-bit ARM® Cortex®-M4 with FPU core, 1-Mbyte Flash memory, 192-Kbyte RAM in an LQFP100 package On-board ST-LINK/V2 on STM32F4DISCOVERY (old reference) or STLINK/V2-A on STM32F407G-DISC1 (new order code) USB ST-LINK with re-enumeration capability and three different interfaces: Debug port Virtual Com port (with new order code only) Mass storage (with new order code only) Board power supply: through USB bus or from an external 5V supply voltage External application power supply: 3V and 5V LIS302DL or LIS3DSH ST MEMS 3-axis accelerometer MP45DT02 ST-MEMS audio sensor omnidirectional digital microphone CS43L22 audio DAC with integrated class D speaker driver Eight LEDs: LD1 (red/green) for USB communication LD2 (red) for 3.3V power on Four user LEDs, LD3 (orange), LD4 (green), LD5 (red) and LD6 (blue) USB OTG LEDs LD7 (green) VBUS and LD8 (red) over-current Two push-buttons (user and reset) USB OTG FS with micro-AB connector Extension header for all LQFP100 I/O for Quick connection to prototyping board and easy probing Comprehensive free software including a variety of examples, part of STM32CubeF4 package or STSW-STM32068 to use legacy standard libraries Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 Hardware and layout The STM32F4DISCOVERY is designed around the STM32F407VGT6 microcontroller in a 100-pin LQFP package Figure 2.1 illustrates the connections between the STM32F407VGT6 and its peripherals (STLINK/V2 or ST-LINK/V2-A, pushbutton, LED, Audio DAC, USB, ST MEMS accelerometer, ST MEMS microphone, and connectors) Figure 2.1 Hardware block diagram Software and programming language There is a variety of different programming languages, the most popular programming language is C Thus, manufacturers have fabricated kit for ARM C language programming Additionally, the programming software also has a lot of choice such as EWARM, MDK-ARM, TrueSTUDIO, tasking, etc We decide to choose MDK-ARM because it gains a popularity of market as well as easy using and friendly interface with users 2.2.2 EEG and Brainwaves EEG Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 References [1] A.Prof: Nguyen Truong-Thinh, “Using Electrooculogram and Electromyogram for powered wheelchair”, Ho Chi Minh city University of Technical Education Ho Chi Minh City, Viet Nam [2] http://www.brainworksneurotherapy.com/what-are-brainwaves [3] A review of classification algorithms for EEG-based brain–computer interfaces - Fabien Lotte, Marco Congedo, Anatole L´ecuyer, Fabrice Lamarche, Bruno Ar [4] Prof Amr El-said, Prof Claudio Fornaro, Mostafa Ahmed Yousef and Mostafa Ezz EL-din Mohame “Brain computer interfaces system” [5] http://neurogadget.net/2012/12/20/neurosky-mindwave-mobile-review/6611 [6] HC-03/05 Embedded Bluetooth Serial Communication Module AT command set [7] https://www.youtube.com/watch?v=Y0y9kffW5wI [8] https://www.pantechsolutions.net/interfacing-mindwave-mobile-with-arduino 39 Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 Addendum Main program #include "main.h" #define Theshold_Eyeblink 120 #define EEG_AVG 70 #define DLED1 GPIO_Pin_6 #define DLED2 GPIO_Pin_8 #define DLED3 GPIO_Pin_12 char SSID_JOIN[]="Espresso"; char PASS_JOIN[]="espressormutl"; char SERVER_IP[] ="185.28.20.22"; char data[]="GET http://finalprojecteeg.890m.com/client.php?state="; char *state; const int PatternCount = 3; const int InputNodes = 4; const int HiddenNodes = 5; const int OutputNodes = 4; const float LearningRate = 0.3; const float Momentum = 0.9; const float InitialWeightMax = 0.5; const float Success = 0.0004; float Input[PatternCount][InputNodes] = { { 0.60, 0.70, 0.60,0.60 }, { 0.40, 0.40, 0.50, 0.30 }, { 0.12, 0.22, 0.18, 0.30 }, }; float Input1[PatternCount][InputNodes] ={ { 0, 0, 0, 0}, }; const char Target[PatternCount][OutputNodes] = { { 0, 0, 0, }, { 0, 0, 0, }, { 1, 1, 0, }, }; int i, j, p, q, r; int ReportEvery1000; int RandomizedIndex[PatternCount]; long TrainingCycle; float Rando; float Error; float Accum; 40 Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 float Accum1; float Hidden[HiddenNodes]; float Output[OutputNodes]; float HiddenWeights[InputNodes+1][HiddenNodes]; float OutputWeights[HiddenNodes+1][OutputNodes]; float HiddenDelta[HiddenNodes]; float OutputDelta[OutputNodes]; float ChangeHiddenWeights[InputNodes+1][HiddenNodes]; float ChangeOutputWeights[HiddenNodes+1][OutputNodes]; float Hidden1[HiddenNodes]; float Output1[OutputNodes]; char checksum=0,generatedchecksum=0; unsigned long payloadDataS[5] = {0}; unsigned long payloadDataB[32] = {0}; unsigned long Avg_Raw,Temp,Temp_Avg; unsigned int Raw_data, Raw_EEG, Poorquality, Attention, Plength, On_Flag=0, Off_Flag=1; unsigned int EyeBlink=0, EyeFlag1=0, EyeFlag2=0; unsigned int Attn_On_Flag_1=1, Attn_On_Flag_2=0, Attn_On_Flag_3=0, Sel_Device=0, dl_nn; uint16_t t=0,k=0,n=0; void toTerminal1(void); void toTerminal(void); void trainning(void); void Big_Packet(char data); unsigned long Small_Packet (char data) ; void Onesec_Rawval_Fun (unsigned long data); unsigned int Eye_Blink (unsigned int data) ; void Home_App_Fun(unsigned long data1,unsigned int data2); void Device_Control_fun(unsigned int data2); void Device_Select_Fun(unsigned int data1); void Attention_Fun(unsigned long data1,unsigned long data2); void Init(void); char USARTx_GetChar(USART_TypeDef* USARTx); void USARTx_SendString(USART_TypeDef* USARTx, char *Str); void Selection_Disp_Fun(void) ; uint8_t cc; u16 ND=0; void toTerminal(void); void toTerminal1(void); void trainning(void); int main() { 41 Final Project: EEG Control Electrical Devices Using Neural Network Algorithm on STM32 RCC_AHB2PeriphClockCmd(RCC_AHB2Periph_RNG, ENABLE); RNG_DeInit(); RNG_Cmd(ENABLE); trainning(); Init(); SystemInit(); TM_HD44780_Init(16, 2); TM_HD44780_Puts(0, 0,(char *)"AU VAN BANG"); esp_init(); join_AP(SSID_JOIN,PASS_JOIN); while(1) { cc=USARTx_GetChar(USART2); if(cc == 170) { cc=USARTx_GetChar(USART2); if(cc == 170) { Plength = USARTx_GetChar(USART2); if(Plength == 32) { Big_Packet (Plength); } else if(Plength == 4) { Raw_EEG = Small_Packet (Plength); if (t