Research, design and construct an autonomous golf cart based on multi task model and gps navigation

MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION THESIS AUTOMATION AND CONTROL ENGINEERING TECHNOLOGY RESEARCH, DESIGN AND CONSTRUCT AN AUTONOMOUS GOLF-CART BASED ON MULTI-TASK MODEL AND GPS NAVIGATION ADVISOR : ASSOC PROF LE MY HA STUDENTS: TA TRAN NHAT MINH SKL011633 Ho Chi Minh City, June 2023 MINISTRY OF EDUCATION AND TRAINING HCMC UNIVERSITY OF TECHNOLOGY AND EDUCATION Faculty of Electrical and Electronics Engineering    FINAL THESIS RESEARCH, DESIGN AND CONSTRUCT AN AUTONOMOUS GOLF-CART BASED ON MULTITASK MODEL AND GPS NAVIGATION Major: Automation and Control Engineering Technology Student: Ta Tran Nhat Minh Student ID: 19151034 Acad Year: 2019 – 2023 Advisor: Assoc Prof Le My Ha Ho Chi Minh city, June 2023 BIÊN BẢN CHỈNH SỬA ĐATN HK2 NH 2022-2023 NGÀNH CNKT ĐK-TĐH Tên đề tài: Research, design and construct an autonomous golf-cart based on multi-task model and GPS navigation Tên SV: Tạ Trần Nhật Minh MSSV: 19151034 Tên GVHD: PGS.TS Lê Mỹ Hà Nội dung chỉnh sửa STT Yêu cầu chỉnh sửa (GVPB Nội dung chỉnh sửa (trình bày nhóm HĐ) chỉnh sửa gì, mục nào, trang bao nhiêu) Chỉnh sửa lại kích thước hình Đã chỉnh sửa kích thước hình 3.36, mục 3.2.11, trang 56 Chỉnh sửa lại lưu đồ kết hợp GPS Đã thêm phương pháp đánh lái vào lưu đồ cho Camera luồng sử dụng GPS Hình chỉnh sửa hình 4.21, mục 4.4, trang 80 Xác nhận GVHĐ (Chèn chữ kí ghi rõ họ tên) Tp HCM, ngày 26 tháng năm 2023 (Chèn chữ kí ghi rõ họ tên) i HCMC University of Technology and Education SOCIALIST REPUBLIC OF VIETNAM Faculty of Electrical and Electronics Engineering Independence – Freedom– Happiness -o0o - Department of automatic control THESIS ASSIGNMENTS Advisor: Assoc Prof Le My Ha Student: Ta Tran Nhat Minh Student id: 19151034 Class: 191512A Major: Automation and Control Engineering Technology Project title Research, design and construct an autonomous golf-cart based on multi-task model and GPS navigation Project’s content • Research documents, papers, books to find out the direction for solving thesis problems • Fix the mechanical and electrical problems from my prototype model • Design and construct the steering control board system with many electronic components • Calculate steering strategy for control wheeling • Collect and label data for training and testing model • Build a multi-task network with deep learning approach • Collect and build a campus road map with GPS data • Synchronize multitask model and GPS navigation with processor board • Collect and evaluate the experiment results • Write the thesis Assignment date: February 15th, 2023 Complete date: June 30th, 2023 CHAIR OF PROGRAM (Sign with full name) ADVISOR (Sign with full name) ii HCMC University of Technology and Education SOCIALIST REPUBLIC OF VIETNAM Faculty of Electrical and Electronics Engineering Independence – Freedom– Happiness -o0o - Department of automatic control ADVISOR’S COMMENT SHEET Student: Ta Tran Nhat Minh Student id: 19151034 Major: Automation and Control Engineering Technology Project: Research, design and construct an autonomous golf-cart based on multitask model and GPS navigation Advisor Assoc Prof Le My Ha Class: 191512A COMMENT Thesis’s content: This thesis consists of six chapters and 100 pages The proposed system is the combination of multiple sensors The real system is successfully completed the objectives in the proposal Advantages: The system has the ability to aid UTE students and lecturers in their mobility within the campus Successfully build a multitask model with the control technique and mechanism design for guiding self-driving cars All the sensors used for this project are low-cost devices The execution time for autonomous mode is suitable for real-time applications Disadvantage: The system is unable to function autonomously without the presence of an operator due to the need for upgrades and improvements in various aspects The accuracy and stability of the system stops at an acceptable level in basic outdoor environments Propose defending thesis: Approved Overall evaluation: Excellent Mark: 10 (In words:) Ten Ho Chi Minh, June 2023 ADVISOR (Sign with full name) iii HCMC University of Technology and Education SOCIALIST REPUBLIC OF VIETNAM Faculty of Electrical and Electronics Engineering Independence – Freedom– Happiness -o0o - Department of automatic control REVIEWER’S COMMENT SHEET Student: Ta Tran Nhat Minh Student id: 19151034 Major: Automation and Control Engineering Technology Project: Research, design and construct an autonomous golf-cart based on multitask model and GPS navigation Advisor Assoc Prof Le My Ha Class: 191512A COMMENT Thesis’s content: The content of this thesis is good and novel Advantage: The results of this thesis have been deployed in the real car and it can operate successfully Disadvantage: Propose defending thesis: Approved Overall evaluation: Very good Mark: 9.5 (In words:) Nine point five Question: Please explain the method to detect the moving line at each corner of the road Ho Chi Minh, June 2023 REVIEWER (Sign with full name) iv A GUARANTEE This thesis is the result of my study and implementation, which I hereby formally proclaim I did not plagiarize from a published article without author acceptance I will take full responsibility for any violations that may have occurred Ho Chi Minh, June 2023 AUTHOR (Sign with full name) Ta Tran Nhat Minh v ACKNOWLEDGMENT I would like to express my deepest grateful towards Assoc Prof Le My Ha for his inspiration, guidance, and instruction to support me for completing my graduation thesis During the whole progress, he not only helped me in finding solutions but also motivated and encouraged me to be able to successfully complete this project Furthermore, I would like to express my sincere thanks to all lectures in Department of automatic control due to the accumulation of basic knowledge and experience they gave me in the four years of university Especially, I also spent my honor to Faculty of Electrical and Electronics Engineering, for providing me the golf cart hardware for the research In addition, I would like to thank Intelligent System Laboratory (ISLab) of Faculty of Electrical and Electronics Engineering and ISLab’s leader Phan Thanh Danh with ISLab’s car team, for the support of facilities and sharing their valuable experiences with me I also would like to express my gratitude to all ISLab members who strongly supported me when I dealt with this thesis Finally, I take this opportunity to respect my parents and family for the encouragement, support, and attention throughout the project Sincere thanks for everything! Ho Chi Minh, June 2023 AUTHOR (Sign with full name) Ta Tran Nhat Minh vi ABSTRACT In this thesis, an autonomous car utilizes a fusion between Multitask model, which includes scene parsing head and lane detection head, and multi-sensors data to manage a robust self-operating in the HCMUTE campus The proposed multi-sensors comprise a Camera, GPS, and IMU (Inertial Measurement Unit) to operate my system In more detail, there are two major tasks for my project: Designing a controller board for steering control and enhancing self-driving car’s perception To begin with, the microprocessor used as the centre element is STM32 and many onboard electronic components The main functions of the mentioned board are controlling the steering angle and the car’s speed Move to perception tasks, in this thesis, a fast and accurate multi-task approach for road line detection was proposed Despite the effectiveness of lane detection, the road line algorithm still suffers from hurdles on the street such as cars or people To this end, I proposed an additional segmentation-based scene parsing module to work in parallel with the road line algorithm The scene parsing acts as external guidance on where to make lane predictions on for the road line algorithm The fusion of two methods was evaluated on a small driving scene dataset collected from a prototype autonomous car The experiment results illustrate that my system performs well on HCMUTE campus roads, with the accuracy of lane line detection is 93.50% and the mIOU value for Segmentation head is 0.81 As for the GPS navigation, the street map was first built up from three different GPS sensors data After that, the was applied to reduce the noises from the environment and enhance the real-time performance of GPS modules My work was implemented on my laptop with NVIDIA GTX 1050Ti graphic card as the primary processing unit Moreover, to reduce the execution burden, I inserted Jetson TX2 as a support processing unit for the mission of dealing with GPS and IMU data Experimental results reveal that the whole system works well on several campus roads The proposed method was applied to a prototype golf-based autonomous car, achieving accurate navigation results on campus road lanes, with about 34 FPS respectively This result satisfies real-time practical applications vii CONTENTS BIÊN BẢN CHỈNH SỬA ĐATN HK2 NH 2022-2023 i THESIS ASSIGNMENTS ii ADVISOR’S COMMENT SHEET iii REVIEWER’S COMMENT SHEET iv A GUARANTEE .v ACKNOWLEDGMENT vi ABSTRACT vii CONTENTS viii LIST OF FIGURES xii LIST OF TABLES xv LIST OF ABBREVIATIONS xvi CHAPTER 1: OVERVIEW 1.1 Introduction 1.1.1 International research .1 1.1.2 Domestic research 1.2 Objectives 1.3 Limitations: 1.4 Research contents: 1.5 Thesis summary .5 CHAPTER 2: LITERATURE REVIEW .6 2.1 Self-driving car technologies 2.1.1 Camera .6 2.1.2 GPS .6 2.1.3 IMU system 2.2 Overview about Artificial Intelligence 2.3 Machine Learning 2.4 Deep Learning 12 viii