(Đồ án hcmute) research and develop wireless imu based motion capture system

MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION CAPSTONE PROJECT MECHATRONICS ENGINEERING TECHNOLOGY RESEARCH AND DEVELOP WIRELESS IMU-BASED MOTION CAPTURE SYSTEM LECTURER: Assoc Prof NGUYEN TRUONG THINH STUDENT: PHAN GIA LUAN SKL 0 Ho Chi Minh City, July 2017 UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING DEPARTMENT OF MECHATRONICS TECHNOLOGY BACHELOR THESIS RESEARCH AND DEVELOP WIRELESS IMU-BASED MOTION CAPTURE SYSTEM ADVISOR: Assoc Prof NGUYEN TRUONG THINH STUDENT’S NAME: PHAN GIA LUAN STUDENT’S ID: 13146298 ACADEMIC BATCH: 2013 - 2017 MAJOR: MECHATRONICS Ho Chi Minh City, July 26th 2017 SOCIALIST REPUBLIC OF VIETNAM Independence - Freedom - Happiness *** Ho Chi Minh City, July 26th 2017 MISSION OF THESIS Student’s name: Phan Gia Luan Student’s ID: 13146298 Major: Mechatronics Class: 13146CLC Advisor: Assoc Prof Nguyen Truong Thinh Phone number: 0938 236 513 Delivery date: February 20th 2017 Submission date: July 21st 2017 Title of thesis: Research and develop wireless IMU-based motion capture system The original data, documents: - Researches and studies on orientation in 3D space, quaternions, MEMS sensors, orientation filters and wireless communications - Datasheets of electronic components - Motion capture developers: Xsens and Perception Neuron Main content: - Research on Madgwick orientation filter, wireless communication and game engine Unity3D - Design and perform motion capture system hardware - Implement orientation filter, calibration, simulation and wireless communication to the motion capture system software - Implement experiments to define errors and system performing statistics Product: - Complete basic motion capture system Dean of Department Advisor i SOCIALIST REPUBLIC OF VIETNAM Independence - Freedom - Happiness *** ADVISOR’S COMMENT Student’s name: Phan Gia Luan Student’s ID: 13146298 Major: Mechatronics Title of thesis: Research and develop wireless IMU-based motion capture system Advisor’s name: Nguyen Truong Thinh COMMENTS The content of thesis and the workload allocated: Advantages: Improvement point: Project defense approval: Yes  No  Evaluation remark: Grade: (in word: ) Ho Chi Minh City, July 2017 Advisor (Name, signature) ii SOCIALIST REPUBLIC OF VIETNAM Independence - Freedom - Happiness *** REVIEWER’S COMMENT Student’s name: Phan Gia Luan Student’s ID: 13146298 Major: Mechatronics Title of thesis: Research and develop wireless IMU-based motion capture system Reviewer’s name: Tuong Phuoc Tho COMMENTS The content of thesis and the workload allocated: Advantages: Improvement point: Project defense approval: Yes  No  Evaluation remark: Grade: (in word: ) Ho Chi Minh City, July 2017 Reviewer (Name, signature) iii ACKNOWLEDGEMENT All the way over here, I faced a lot of difficulties and obstacles I sometimes wonder if without the help of everybody, how I can come here today This thesis would be impossible without conscientious help of my advisor Assoc Prof Nguyen Truong Thinh I would like to send my sincere appreciation to my strict teacher as well as my advisor Assoc Prof Nguyen Truong Thinh for his advice and knowledge He provided us the professional working environment, necessary equipment, regular urge… so that we could focus on perfectly completing the project I want to send my thanks to Ph.D Vu Quang Huy and all teachers who unconditionally helped and motivated me, pointed out my project defects so that I could timely improve my thesis and essay optimally Thanks are not enough to extend my gratitude to my family especially my mother who is always beside and encourages me, has supported me everything in this project And I not forget to thanks to my friends who always by my side listened and shared difficulties with me even if they are not my teammates People on the internet well-timed helped me solve the problems that I can spend a lot of time solving alone without them Ho Chi Minh City, July 26th, 2017 Student: Phan Gia Luan iv ABSTRACT Talking about motion capture systems, we will think of a system that has a lot of white markers distributed on the suit worn on the human body, can record and simulate the motion of human or any other objects in the software However, these systems are worth a lot of money, can only operate in a wide and fixed space with many cameras attached around Therefore, only large animation filmmakers or graphic designers are capable of purchasing this type of system In this project, research and develop the low cost, moderate accuracy, high speed, portable and easy-to-use wireless IMU-based motion capture system is our main contribution We will focus on designing and building the full-featured hardware and the very simple operation program and control software by using the low cost components The presented system is based on a network of small inertial measurement units called “node” distributed on the human body Each node consist of the baseboard called “SHURIKEN”, battery and motor housed in the 3D-printed enclosure SHURIKEN is provided the MEMS sensors: tri-axis accelerometer, tri-axis gyroscopes and tri-axis magnetometer, 32-bit microcontroller unit integrated Wi-Fi and on-board power solution This board also has microUSB which supports UART interface or battery charging and microSD card socket acts as the external memory In essence, the MCU is the core of the board It collects measured data from the sensors, perform sensors fusion algorithm based on a quaternion-based Madgwick orientation filter and transfer data to host via Wi-Fi or store it in the memory for later use After that, Nodes’ processed data was simulated by the program called “SHURIKEN launcher” The nodes’ behavior also is controlled by this program All of these activities are incorporated into the operation of the system in this project The result of experiments on accuracy demonstrated the feasibility and advantages also a few shortcomings of the system The advantages and limitations of the system, hardware and software architecture in more detail will be discussed v CONTENTS MISSION OF THESIS i ADVISOR’S COMMENT ii REVIEWER’S COMMENT iii ACKNOWLEDGEMENT iv ABSTRACT v LIST OF ABBREVIATIONS ix LIST OF TABLES x LIST OF FIGURES AND CHART .xi CHAPTER INTRODUCTION 1.1 Motivation 1.2 Motion capture technology based on IMU 1.2.1 Advantage of the based-IMU mocap 1.2.2 Disadvantage of the based-IMU mocap 1.4 State-of-the-art IMU-based motion capture 1.4.1 Xsens – the leading innovator in 3D motion tracking technology 1.4.2 Perception Neuron 1.5 Research objectives CHAPTER 2: FUNDAMENTAL THEORY 2.1 Rotation in three-dimensional space 2.1.1 Coordinate frame 2.1.2 Quaternion 2.1.3 Conversion 10 2.1.4.1 Quaternions to Euler angles 11 2.1.4.2 Euler angles to quaternions 11 2.2 Mems-based inertial measurement unit 11 2.2.1 Gyroscope sensor 12 vi 2.2.2 Accelerometer 12 2.2.3 Magnetometer 12 2.3 Wireless data transmission 13 2.3.1 Wireless body sensor network 13 2.3.2 User datagram protocol 13 2.4 Unity3D game engine 14 CHAPTER 3: NODE ARCHITECTURE 15 3.1 System hardware diagram 15 3.2 SHURIKEN baseboard 17 3.2.1 MPU9255 – Nine-axis motion tracking sensor 17 3.2.2 ESP8285 – Highly integrated Wi-Fi solution 19 3.2.3 Power Solution 21 3.2.3.1 Battery 21 3.2.3.2 LDO Regulator 22 3.2.3.3 Charge manager 23 CHAPTER 4: ORIENTATION FILTER 24 4.0 Orientation filter overview 24 4.1 Data Calibration 24 4.1.1 Accelerometer zero-adjustment 24 4.1.2 Gyroscope bias drift 25 4.1.3 Hard iron and soft iron effects 26 4.2 Madgwick orientation filter 26 4.2.1 Orientation from angular rate 26 4.2.2 Orientation from vector observations 28 4.2.3 Filter fusion algorithm 31 CHAPTER 5: KINEMATICS 33 5.1 Kinematics in Unity3D 33 5.1.1 Orientation of device frame relatives to unity frame 34 vii 5.1.2 Orientation of device frame relatives to model frame 34 CHAPTER 6: COMMUNICATION AND IMPLEMENT 36 6.1 Node operation program 36 6.2 SHURIKEN launcher – emulator and nodes controller 37 6.2.1 Scenes controller 37 6.2.2 Nodes’ operation controller 37 6.2.3 Implement calibration 38 6.2.4 Assign the nodes’ ID to the model’s link 38 6.3 Nodes and user device communication 39 6.3.1 Wireless Body Sensor Network overview 39 6.3.2 Network management 40 CHAPTER 7: EXPERIMENT AND RESULT 42 7.1 Orientation filter experiments 42 7.1.1 Effect of calibration on the results 42 7.1.2 Effect of convergence coefficient on the results 45 7.2 Testing network performance 46 7.2.1 Latency - One way delay 46 7.2.2 Wireless update rates 46 7.3 Result of the project 47 CHAPTER 8: CONCLUSION 48 REFERENCES 49 viii 6.2 SHURIKEN launcher – emulator and nodes controller SHURIKEN launcher is the application that connects user and nodes via WiFi, receives data from nodes, reprocesses data and simulates data This software is completely built with game engine Unity3d version 5.6.1f1 This application uses a Maximo’s model which is Y-bot It has five main functions discussed in the subsections below 6.2.1 Scenes controller Figure 6.2 Scenes in the software In the figure 6.2, the program starts up at the main scene From here, you can choose one of three scenes where each scene simulates a different object "Simulate node scene" simulates a specific node, "Simulate arm scene" that simulates Y-bot's right arm movement and "Simulate Y-bot scene" emulates Y-bot's entire body motion 6.2.2 Nodes’ operation controller With UDP protocol, User can control the operation of the nodes As mentioned above, user sends a specific flag to a node (unicast) or to all nodes (broadcast) by using the buttons in the interface so that the nodes exactly perform a specific function After done the task, by returning the same flag from the nodes to user, software can perform a closed loop control – Detail descripted in figure 6.3 37 Figure 6.3 Block diagram represents closed loop control of user and node 6.2.3 Implement calibration To perform the calibration, user have to select the “node simulate scene” in main scene For once process, software just can request only one node perform calibrate that is slightly inconvenient but it has specific reason based on the way performing calibration Performing gyroscope and accelerometer calibration: To archive successful calibration, before calibration user need lie node on a flat surface Then user click the button “Calibrate MPU” and the process is started The calibration process will have done after a few seconds While performing the calibration process, external movements greatly affect the result of the calibration process Performing magnetometer calibration: It completely contrast to the performance of gyro and accelerometer calibration, the magnetometer calibration requires user rotates the node continuously in different directions over 10 seconds for the good results Because of difference from local magnetic flux, calibrate magnetometer at each working area is required 6.2.4 Assign the nodes’ ID to the model’s link As mentioned in the previous section, in order to obtain the motion of the links of the object, attaching the node to it is required When tying nodes to an object, the system does not require any order Therefore, for the software to know which node to emulate the link of the object, assign the node's ID to the corresponding model's link is indispensable This is a prerequisite step to simulations ID is the number to 38 distinguish the nodes which imprinted in the top surface of nodes’ enclosure Figure 6.4 figure with input field to assign the node’s ID As the figure 6.4, the figure’s humanoid model is presented This model has a total of 15 links, each link has an input field to enter the node’s ID that will measure its motion According to the figure, the node has an ID equals will measure the orientation of the left hand, so user will wear this node on their left hand If user didn’t not require any order then user could use “auto assign” 6.3 Nodes and user device communication 6.3.1 Wireless Body Sensor Network overview In this project, Wireless Body Sensor Network (BSN) consists of several nodes, end-devices and wireless router Nodes as the station is responsible for measuring and processing data and then sending data to end-devices via wireless router End-devices wait and accept data coming from wireless router then the obtained data is processed and used 39 Figure 6.5 Wireless Body Area Network overview 6.3.2 Network management In BSN, wireless router plays a central role It has three main tasks: broadcast WiFi signal, access control and forward data packets The router acts as a wireless AP, broadcast a hidden Wi-Fi signal Since this is an intranet and should be quarantined, this router has MAC address filter enabled It only allows devices which MAC address has been added to the filter to log DHCP in the wireless router will be provided with a client list (each client is identified by MAC address) in which each client is assigned a static IP This IP will be of the form: 192: 168: 0: 𝐼𝐷 ID is the number to distinguish the nodes which imprinted in the top surface of nodes’ enclosure Assuming system has 20 nodes, the nodes' IPs will be from 192.168.0.1 to 192.168.0.20 The last number of addresses of the end-devices which connects to the router must be greater than the number of nodes and different from the address of the router In this case, the router’s address is 192.168.0.99 The figure 6.6 gives an overview of the IP address of devices in the network: 40 Figure 6.6 IP address of devices in the network 41 CHAPTER 7: EXPERIMENT AND RESULT 7.1 Orientation filter experiments 7.1.1 Effect of calibration on the results Due to the limitations of measurement techniques, the measurement of the orientation error obtained from the node will be calculated by comparing the data with a step motor The orientation error has two components: static error and dynamic error The term static error or static accuracy refers to measurements made when the inertial unit is not moving and the on-board orientation algorithm has recovered from any sensor over-ranging This error is obtained by leaving node on the surface after k times which is recovered time from any sensor over-ranging Then measure the data over several seconds Because of Data measured in quaternion form, to make it easier to visualize the error, this data will be passed into the Euler form The below table shows parameters used in the experiment With ∆𝑡, the cycle takes data from the sensor, equals ms and µ equals 1, we get the result in figure 7.1 42 Figure 7.1 Static error obtained by leaving node on the surface The term dynamic error or accuracy refers to measurements made while the inertial unit is moving and not exceeding the measurement range of the individual on-board sensors This error is obtained by leaving node on the surface after k times Then measure the data over several seconds while the step motor rotate around specific axis with constant speed This experiment is completed with ∆𝑡 equals ms, μ equals 1, and the stepper rotates around the Zenith axis at 60RPM presented in figure 7.2 43 Figure 7.2 Dynamic error by rotating around Zenith with constant speed For calculating the errors of the uncalibrated sensor data and the calibrated sensor data, the root mean square of the deviations in the individual experiments are calculated [1] The errors of the uncalibrated and the calibrated are shown in table 7.1 Table 7.1 Root mean square error of uncalibrated and calibrated data Uncalibrated Calibrated RMS[roll] static error 2.1072 0.1861 RMS[pitch] static error 0.5347 0.1320 RMS[yaw] static error 64.2105 3.1474 RMS[roll] dynamic error 2.4393 0.6790 RMS[pitch] dynamic error 3.1434 0.9837 RMS[yaw] dynamic error 71.5253 3.0003 44 7.1.2 Effect of convergence coefficient on the results The results of an investigation into the effect of the filter adjustable parameter, µ, on the time response of output data represented by the fourth element of quaternion and steady-state error represented by the third element are summarized in figure 7.3 and 7.4 The experiment is based on the time that the algorithm achieves the node's current orientation when the node is stationary Figure 7.3 Time response of filter with different convergence coefficient Figure 7.4 Steady-state error with different convergence coefficient The results were based on three experiments with three different μ parameters The larger convergence coefficient, the smaller the settling time and the larger the steady-state error Based on empirical tests, μ equals gives an acceptable error 45 (table 7.1) and settling time is reasonable to ensure that able to capture motion with angular movement is less than the speed limit of the gyroscope sensor 7.2 Testing network performance 7.2.1 Latency - One way delay In real-time data transmission, network latency is a parameter that greatly influences the system's responsiveness Because the network in the project is primarily one-way traffic from nodes to the user, network latency corresponds to one-way delay One-way delay (OWD) refers to the time taken for a packet to be transmitted across a network from source to destination [19] Here is a simple method described in figure 7.5 to obtain OWD Assuming the UART transmission and processing time is negligible, using the stopwatch class, OWD is obtained Figure 7.5 Block diagram represents how to get OWD The experiment is done by using one node Repeated experiments yield different values The OWD value is equal to the maximum value of the experiments shown in table 8.2 7.2.2 Wireless update rates The wireless update rate, 𝑓, which is the number of data packages that are transmitted in a second, depend on the internal update rate and the time node takes to send data, 𝑡𝑡𝑓 Internal update rate equals inverse of the cycle collects and process data from the sensor, ∆𝑡 Experiment through the timer, ∆𝑡 is obtained We assume 𝑎 is the number of internal update in a second the wireless update rate is obtained by solving the equation (7.1) 46 𝑎 ∆𝑡 + 𝑓 𝑡𝑡𝑓 = (7.1) When the MCU operates at 80 MHz, mean of ∆t and 𝑡𝑡𝑓 approximately equals 1.7ms and 0.6ms respectively The higher 𝑓 is the lower 𝑎 To ensure both the number of times implement orientation filter and power consume by transferring, 𝑓 needs equals 30 Hz Therefore, 𝑎 approximately equals 482 Hz 7.3 Result of the project The Orientation filter constants and node specifications synthesized from the results of experiments and measurements shown in the table 7.1 and 7.2 Table 7.2: Orientation filter constants Constant parameters Value 92 Hz 𝑓𝑐 𝜇 0.0017 ms ∆𝑡 482 Hz 𝑎 ±500 o/s 𝐹𝑆𝑅𝜔 ±4 g 𝐹𝑆𝑅𝑎 ±4912 µT 𝐹𝑆𝑅𝑚 16-bit 𝐴𝐷𝐶 𝑤𝑜𝑟𝑑 𝑙𝑒𝑛𝑔𝑡ℎ Table 7.3: Node specifications Specification Value Dimensions Ø40 x 19.5 mm Resolution 1.8 degrees Internal sampling rate 500Hz Battery life (continuous use) 1.5 hours Charge time hour Wireless update rates 30Hz Wireless Range 25m Latency