Ho Chi Minh city, July, 2017 MINISTRY OF EDUCATION AND TRAINING HCMC UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF HIGH QUALITY TRAINING ADVISOR NGUYEN VAN THAI, PhD STUDENT NGUYEN HOANG HAI NGUYE[.]
MINISTRY OF EDUCATION AND TRAINING HCMC UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF HIGH QUALITY TRAINING CAPSTONE PROJECT AUTOMATION AND CONTROL ENGINEERING TECHNOLOGY RESEARCH, DESIGN AND CONSTRUCT TWO - WHEEL BALANCING ROBOT FOR OBJECT TRACKING ADVISOR: NGUYEN VAN THAI, PhD STUDENT: NGUYEN HOANG HAI NGUYEN DANG KHOA S K L0 0 Ho Chi Minh city, July, 2017 MINISTRY OF EDUCATION AND TRAINING HCMC UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF HIGH QUALITY TRAINING DEPARTMENT OF AUTOMATION CONTROL -*** *** - THESIS RESEARCH, DESIGN AND CONSTRUCT TWOWHEEL BALANCING ROBOT FOR OBJECT TRACKING AUTHOR: NGUYEN HOANG HAI NGUYEN DANG KHOA 13151026 13151043 2013 – 2017 AUTOMATION AND CONTROL ENGINEERING TECHNOLOGY ADVISOR: NGUYEN VAN THAI, PhD YEAR: MAJOR: HO CHI MINH city, July, 2017 HCMC UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF HIGH QUALITY TRAINING THESIS RESEARCH, DESIGN AND CONSTRUCT TWOWHEEL BALANCING ROBOT FOR OBJECT TRACKING AUTHOR: NGUYEN HOANG HAI NGUYEN DANG KHOA STUDENT ID: 13151026 STUDENT ID: 13151043 2013 – 2017 AUTOMATION AND CONTROL ENGINEERING TECHNOLOGY ADVISOR: NGUYEN VAN THAI, PhD YEAR: MAJOR: Ho Chi Minh city, July, 2017 HCMC UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF HIGH QUALITY TRAINING THESIS RESEARCH, DESIGN AND CONSTRUCT TWOWHEEL BALANCING ROBOT FOR OBJECT TRACKING AUTHOR: NGUYEN HOANG HAI NGUYEN DANG KHOA STUDENT ID: 13151026 STUDENT ID: 13151043 2013 – 2017 AUTOMATION AND CONTROL ENGINEERING TECHNOLOGY ADVISOR: NGUYEN VAN THAI, PhD YEAR: MAJOR: Ho Chi Minh city, July, 2017 SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom – Happiness ******* Ho Chi Minh city, July, 2017 TASKS OF THE THESIS Name of student: Student ID: Name of student: Student ID: Major: Class: Name of Advisor: Mobile phone: Name of the thesis: Initial figures and documents: Content of the thesis: Product: Ho Chi Minh city, July ,2017 Ho Chi Minh city, July ,2017 Dean of Faculty Advisor SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom – Happiness ******* ADVISOR’S COMMENT SHEET Name of student: Student ID: Name of student: Student ID: Major: Name of the thesis: Name of Advisor: COMMENT About the thesis’s content and workload: Advantage: Disadvantage: Propose defending thesis? Rating: Mark: (In writing: ) Ho Chi Minh city, July ,2017 Advisor SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom – Happiness ******* REVIEWER’S COMMENT SHEET Name of student: Student ID: Name of student: Student ID: Major: Name of the thesis: Name of Reviewer: COMMENT About the thesis’s content and workload: Advantage: Disadvantage: Propose defending thesis? Rating: Mark: (In writing: ) Ho Chi Minh city, July ,2017 Reviewer ACKNOWLEDGEMENTS It is a great pleasure to thank all of our teachers, friends that supported us in the process of doing our thesis We wish to express our gratitude towards Nguyen Van Thai, PhD for his time and effort in helping us to learn much more scientific concepts, basic and advance knowledge His excellent comments and enthusiastic approach helps us a lot when we get in stuck to find ideas We always remember the time when we first met him to discuss about the thesis Many thanks to our vice president of HCMC University of Technology and Education, Ngo Van Thuyen, PhD who is in charge of scientific research, international relations, publics relations, construction of ODA projects, deploy KPIs and quality assurance of our university He gave us the chances not only broaden our horizon but also strengthen our personality, soft skills I would like to thank Nguyen Minh Tam, PhD Dean of Faculty of Electrical and Electronics Engineering and Truong Dinh Nhon, PhD They never hesitate to help us when we encounter problems And we could not have done the research without all opportunities which Le My Ha, PhD and MS Nguyen Tran Minh Nguyet, from Department of Automation Control of Faculty of High Quality Training, created for us Especially, M.Eng Nguyen Tran Minh Nguyet was very closed to us, we learnt really a lot from our discussions and her encouragement as well as life lessons of Le My Ha, PhD Last but not least, our deepest gratitude goes to our family for their continuous presence and uncommon strength We are eternally grateful to our parents for their unflagging love and support throughout our life i TÓM TẮT Luận văn “Xe hai bánh tự cân theo dõi bám mục tiêu ứng dụng xử lý ảnh hệ điều hành Android" phát triển từ xe hai bánh tự cân dựa lý thuyết mơ hình lắc ngược Nhóm định phát triển đề tài từ hệ thống xe hai bánh tự cân dùng giải thuật PID sang sử dụng giải thuật LQR mô Matlab để nghiên cứu giải thuật LQR ứng dụng điều khiển robot hai bánh cân Bên cạnh thực xe hai bánh tự cân bằng, nhóm phát triển xe để ứng dụng xử lý ảnh hệ điều hành android nhằm theo dõi bám mục tiêu cụ thể bóng tennis Để robot bám theo bóng tennis trì khoảng cách mong muốn, cần kết hợp thêm điều khiển PID vào điều khiển LQR để robot vừa cân vừa bám theo trái banh Luận văn trình bày bao gồm phần cứng bao gồm: Arduino Mega 2560, cảm biến góc nghiêng MPU 6050, board nhóm thiết kế tích hợp mạch cầu H L298 max 50V-2A giảm áp LM2596 1.23-27V, module Bluetooth HC-05,… ii ABSTRACT The thesis “Two-wheel balancing robot monitoring and tracking the object using image processing on android operating system” is developed by our team from “Two-wheel balancing robot” based on “Inverted Pendubot” We decided to develop from PID controller to LQR controller and simulate on Matlab to research LQR and apply it to control Two-wheel balancing robot Besides making “Two-wheel balancing robot”, our team will upgrade one new function for this robot – monitoring and tracking the object by using image processing on android After several times of considerations, we have chosen a solution that smartphone identifies, calculates the distance between the robot and the object and sends data to Arduino Mega 2560 for processing through module Bluetooth HC-05 because of the convenience of using a smartphone instead of raspberry pi To make the robot track the tennis ball and maintain the desired distance from it requiring combination between PID controllers and LQR controller so the robot would be able to balance and track the ball at the same time We will present the hardware of system includes Arduino Mega 2560, sensor MPU 6050, board designed by our team integrated in H-bridge L298 50V-2A and LM2596 regulator 1.23-27V, module Bluetooth HC-05… iii 12V - converter has ExT value of 39.93 (V s) then the inductor region would be L39 5V – converter’s ExT value is of 28.26 (V s) , the inductor region would be L40 Look up the statistics table below, L39’s inductance value is 47 H and L40 is 33 H Based on the available inductor on the market, we chose both inductor value is of 33 H Table 3.5: Inductor Manufacturers Part Numbers Next step is to choose the catch diode, in this system, a 3A current rating is adequate and for a robust design, 1N5822 Schottky diode is selected Finally, to prevent large voltage transient from appearing at the input and for stable operation of the converter, an electrolytic capacitor is used between +Vin and ground pin GND 71 Since LM2596 is a forward-mode switching regulator with voltage mode control, its ope loop has 2-pole-1-zero frequency characteristic The loop stability is determined the output capacitor (capacitance, ESR) and inductance value Both input and output capacitor chosen in this case are 220 F electrolytic caps Figure 3.15: Step-down Converter Schematic 3.5 MODULE BLUETOOH: Figure 3.16: Module Bluetooth HC05 Bluetooth offers bandwidth up to 720 Kbps in data range from 10m to 100m Unlike infrared (IrDA), when the communication needs to surface the device close to each other, bluetooth uses multidirectional radio waves that allow transmission through non-metallic obstacles Bluetooth transmits at 2.4GHz frequency and uses continuous scalar broadcast technology 72 Table 3.6: Specification of module Bluetooth HC-05 Module Power Current Baud rate Operating frequency range Bluetooth protocol Dimension Bluetooth HC-05 3.3-5V 30mA when paring, after paring the normal transmission operation 8mA 1200, 2400, 4800, 9600, 19200, 38400, 57600, 115200 2.4GHz Bluetooth Specification v2.0 EDRo 28mm x 15mm x 2.35mm Setting default: Baud rate: 9600, N, 8, Pairing code: 1234 Figure 3.17: HC05 Schematic 73 3.6 MAINBOARD After selecting all the things needed, then we come to make the main circuit board for the system Below is the final overall schematic: Figure 3.18: Mainboard Schematic Figure 3.19: PCB Layout 74 CHAPTER 4: ALGORITHMS 4.1 GENERAL: The general diagram of the system as shown in figure 4.1: HC05 CAMERA ARDUINO BLUETOOTH ACTUATORS (DC SERVO) Figure 4.1: The general diagram of the system CAMERA IMAGE ACQUISITION EXTRACT PARAMETERS PREPROCESSING ANALYSIS Figure 4.2: The basic step of image processing Image acquisition: received image can be color image or binary image However, It is usually analog image Image quality depends on camera, environmental conditions (light, landscape) Pre-processing: after image acquisition, this process is an improvement of the image data that suppress undesired distortions or enhances some image features important for further processing However, it does not increase image information content Analysis: the object is described as the boundary or filled shape in the image If we represent the object shape based on the boundary, we need to trace and traverse the boundary Extract specification: in this thesis we extract the edges from an original image It simplifies the analysis of the image by dramatically reducing the amount of data to be processed while at the same time preserving useful information about the boundaries 75 4.2 GENERAL FLOWCHART OF IMAGE PROCESSING: START F OPEN CAMERA T F IMAGE ACQUISITION T F F IDENTIFY COLOR & MORPHOLOGICAL TRANSFORMATION T DETECT THE BALL T TRANSFER DATA Figure 4.3: Flowchart of image processing in this thesis 4.2.1 TRANSFER DATA: START F DATA AVAILABLE T F COM PORT CONNECTED COM PORT CONNECTED ERROR T TRANSFER/RECEIVE DATA Figure 4.4: The diagram of transfer data 76 4.2.2 IDENTIFY COLOR & MORPHOLOGICAL TRANSFORMATION: START CONVERT COLOR SPACE HSV COLOR FILTERD OpenCV support us function inputFrame.rgba() for capturing each frame and processing on that frame The output of this function is the RGB image which has three color channels: Red, Green, Blue However, rgba also consist of the fourth channel which provides us (a), light adjustment parameter We convert RGB to HSV color space by using Imgproc.cvtColor(inputArray src, OutputArray dst, int code, int dstCn) function: MORPHOLOGICAL TRANSFORMATION Figure 4.5: Color detection and morphological transformation src: input image: 8-bit unsigned, 16-bit unsigned or single-precision floating-point dst: output image of the same size and depth as src code: We use Improc.COLOR_RGB2HSV dstCn: number of channels in the destination image We use 3: H, S and V channels The output of converting RGB to HSV is filtered by Core.inRange(inputArray src, InputArray lowerb, inputArray upperb, OutputArray dst) function This function filters the image by scanning all matrices based on the range value Pixels in that range are made as 255 (maximum of bits image - white) On the other hand, they are made as (minimum of bits image – black) src: first input array lowerbs: inclusive lower boundary array or a scalar upperb: inclusive upper boundary array or a scalar dst: output array of the same size as src Next, in case we want to suppress small black holes and white noise We use Closing and Opening method The Imgproc.morphologyEx() 77 4.2.3 DETECT THE BALL: START EDGE DETECTION In edge detection process, we can use functions Imgproc.Canny(intputArray image, OutputArray edges, double threshold1, double threshold2) BLUR THE IMAGE CIRCLE DECTION Figure 4.6: Detect the ball image: single-channel 8-bit input image edges: output edge map, it has the same size and type as image threshold1: T1 threshold for hysteresis procedure threshold2: T2 threshold for hysteresis procedure As the image after processing still has a lot of noise and in order to avoid mistakes in the identifying process, the image should be blurred with function Imgproc.GaussianBlur(inputArray src, OutputArray dst, Size ksize, double sigmaX, double sigmaY) src: input image The image can have any number of channels dst: output image of the same size and type as src ksize: Gaussian kernel size with ksize.width and ksize.height sigmaX: Gaussian kernel standard deviation in X direction sigmaY: Gaussian kernel standard deviation in Y direction Finally, we need to detect the circle by using contour tracing method First, we need to find contours in binary image by using Imgproc.findContours(InputArray image, outputArray contours, OutputArray hierarchy, int mode, int method) Secondly, the function Imgproc.minEnclosingCircle(points, Point2f& center, float& radius), with points is the inputArray, helps us to find a circle of the minimum area enclosing a 2D point set 78 4.3 FLOWCHART OF ROBOT CONTROL: Begin Control signal F Tilt angle measurement and put on Kalman filter T T Value E Remote control mode F T Value A Run forward F T Value e Tracking ball mode T Value a Stop running F Measure the distance between the robot and the green ball F Distance value received from smartphone T Value B Run backward F T Take PWM values from LQR controller to make the robot balancing Take PWM values from LQR controller and PID controllers to make the robot balancing and tracking the ball T Value b Stop running T Value C Rotate clockwise F T Value c Stop rotating F T Value D Rotate counterclockwise F T Value d Stop rotating F Figure 4.7: Flowchart of controlling robot 79 CHAPTER 5: RESULTS 5.1 PRACTICAL MODEL: Robot can move forward, backward, turn left, turn right, balance and track the ball The practical model is shown in figure 5.1: Figure 5.1: The practical model Figure 5.2: Completed board 80 5.2 MATLAB SIMULATION: Figure 5.3 shows the Simulink of Robot: Figure 5.3: Simulation of Robot using LQR controller The input parameters when balancing are (teta=0, psi=0, phi=0) We can choose different input parameters to perceive the change of system: = 0.001 (rad), = 0.0012 (rad/s), = 0.002 (rad), = 0.002 (rad/s), = 0.02 (rad), = 0.0014 (rad/s) We also choose the weighting matrix: 0 0 100000 10000 0 0 0 0 1 Q ; R 0 0 0 0 0 100000 0 0 10000 By using the function lqr(A, B, Q, R), we can find out the K parameter: 1.0e+04 * K -0.0224 0.0156 -1.0551 -0.2208 -0.0224 -0.0226 -0.0224 0.0156 -1.0551 -0.2208 0.0224 0.0226 The result of Matlab simulation is shown in figure 5.4: 81 Figure 5.4: Output response 5.2.1 COMPUTER INTERFACE: We use Zedgraph to track the angle by drawing angle on Zedgraph Figure 5.5: Communicating interface 82 5.3 EXPERIMENTAL RESULT: 5.3.1 EXPERIMENTAL RESULT OF MANUAL MODE: Experimental Psi Error < Psi Error > times degrees degrees Stand still 30 29 Forward 30 29 Backward 30 29 Turn Left 30 30 Turn Right 30 30 Figure 5.6: Experimental result of manual mode Efficiency 97% 97% 97% 100% 100% 5.3.2 EXPERIMENTAL RESULT OF TRACKING MODE: Experimental Distance Distance Efficiency times Error < 1cm Error > cm Green 40 38 95% Ball Figure 5.7: Experimental result of tracking mode 83 CHAPTER 6: CONCLUSION AND DEVELOPING ORIENTATION 6.1 CONCLUSION: After months working together, our team has finished the thesis “Two-wheel balancing robot tracking the ball” with several key goals has been achieved: The robot is compact in size, flexible in operation Robot works effectively and achieves the required standard Transfer data by communicating to computer and smartphone via Bluetooth platform Robot can track the object based on the setting distance Read angle correctly by using Kalman filter Robot can balance, move forward, backward, turn left, turn right on even terrain The android software works well and is easy to use Besides that, this project also has some limitations such as there is very small shake when the robot stand still and not be able to track a too quickly moving ball 6.2 DEVELOPING ORIENTATION: Based on the successful research of the thesis “Research, Design and Construct Two-Wheel Balancing Robot for Object Tracking”, we can develop the project by using a better smartphone to improve image processing It also could be utilized as a warehouse robot with the main task is tracking item status and update new information to a server system through a wireless connection Firefighting robot is also a good example of application 84