(Đồ án HCMUTE) intelligent parking support system

MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING ELECTRONICS AND TELECOMMUNICATION ENGINEERING TECHNOLOGY INTELLIGENT PARKING SUPPORT SYSTEM LECTURER: PHAM VAN KHOA STUDENT : LA GIA KIET NGUYEN DAC THANG SKL 0 9 Ho Chi Minh City, August, 2020 i HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION THESIS MAJOR: ELECTRONICS AND COMMUNICATION ENGINEERING TECHNOLOGY INTELLIGENT PARKING SUPPORT SYSTEM STUDENT: PHAM DUC HOANG NGUYEN DAC THANG ADVISOR: Ph D PHAM VAN KHOA Ho Chi Minh City, July 2022 i - 18161012 - 18161037 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION THESIS MAJOR: ELECTRONICS AND COMMUNICATION ENGINEERING TECHNOLOGY INTELLIGENT PARKING SUPPORT SYSTEM STUDENT: PHAM DUC HOANG NGUYEN DAC THANG ADVISOR: Ph D PHAM VAN KHOA Ho Chi Minh City, July 2022 i - 18161012 - 18161037 THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -Ho Chi Minh City, July 28th, 2022 GRADUATION PROJECT ASSIGNMENT Student name: Pham Duc Hoang Student ID: 18161012 Student name: Nguyen Dac Thang Student ID: 18161037 Major: Electronics and Communication Engineering Technology Class: 18161CLA Advisor: Ph D Pham Van Khoa Phone number: Date of assignment: Date of submission: Thesis title: Intelligent Parking Support System Initial materials provided by the advisor Content of the thesis: - To determine the thesis' development course, consult the document, read it, and make a summary - Design and explanation of flow charts - Running the program and confirming successful completion - Prepare presentations and write a report CHAIR OF THE PROGRAM ADVISOR (Sign with full name) (Sign with full name) Ph D Pham Van Khoa i THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -Ho Chi Minh City, July 28th, 2022 ADVISOR’S EVALUATION SHEET Student name: Pham Duc Hoang Student ID: 18161012 Student name: Nguyen Dac Thang Student ID: 18161037 Major: Electronics and Communication Engineering Technology Thesis title: Intelligent Parking Support System Advisor: Ph D Pham Van Khoa Phone number: EVALUATION Content of the project: Strengths: Weaknesses: Approval for oral defense? (Approved or denied) Overall evaluation: (Excellent, Good, Fair, Poor) Ho Chi Minh City, July 28th, 2022 ADVISOR (Sign with full name) Ph D Pham Van Khoa i THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -Ho Chi Minh City, July 28th, 2022 PRE-DEFENSE EVALUATION SHEET Student name: Pham Duc Hoang Student ID: 18161012 Student name: Nguyen Dac Thang Student ID: 18161037 Major: Electronics and Communication Engineering Technology Thesis title: Intelligent Parking Support System Name of Reviewer: Ph.D Do Duy Tan Phone number: EVALUATION Content and workload of the project: Strengths: Weaknesses: Approval for oral defense? (Approved or denied) Reviewer questions for project valuation Mask: (in words:………………………………………………….) Ho Chi Minh City, July 28th, 2022 REVIEWER (Sign with full name) Ph.D Do Duy Tan i DISCLAIMER Thesis title: Intelligent Parking Support System Advisor: PHAM VAN KHOA, Ph D Student 1: PHAM DUC HOANG Student’s ID: 18161012 Class: 18161CLA1 Email: 18161012@student.hcmute.edu.vn Student 2: NGUYEN DAC THANG Student’s ID: 18161037 Class: 18161CLA2 Email: 18161037@student.hcmute.edu.vn “We commit not to copy or reuse the results of other people's work as part of the graduation project There is a comprehensive list of references available.” Ho Chi Minh City, July 28th, 2022 GROUP MEMBER (Sign with full name) Pham Duc Hoang Nguyen Dac Thang i i ACKNOWLEDGE  The most crucial phase of every student's life is the process of finishing their graduation thesis Our graduation thesis serves to provide us with knowledge we will need in the future, as well as research techniques We would first like to express our gratitude to Ho Chi Minh City University of Technology and Education The Faculty of High-Quality Training teachers, in particular during our time in the lecture hall, enthusiastically lectured and equipped us with essential knowledge, laying the foundation for our capacity to complete the thesis We want to express our gratitude to Ph.D Pham Van Khoa for his passionate assistance and guidance in terms of scientific thinking and effort Those are some extremely valuable suggestions, both for the development of this thesis and as a foundation for our subsequent coursework and professional growth If something is missing from the project, our team is hopeful that Ph.D Pham Van Khoa and the professors can sympathize and provide suggestions on how to improve Ho Chi Minh City, July 28th, 2022 ii i ABSTRACT Currently, the number of vehicles participating in traffic on the road is very large, leading to consuming a lot of human and material resources for the management of personal vehicles in the parking lot Without a convenient tool, the management of personal vehicles is very time consuming, easy to cause confusion, damage to service users at parking lots To reduce the load on tasks such as collecting money, insuring cars, finding vehicles in parking lots, the world has developed automatic monitoring technology for vehicles, thanks to the individuality of the sea The number of vehicles that has become the main object used for research and development in this technology Therefore, our group want to choose this topic as a basic step in understanding more powerful monitoring tools such as vehicle control on the road or facial recognition which are being paid great attention by the world at the moment iii i TABLE OF CONTENTS DISCLAIMER i ACKNOWLEDGE ii ABSTRACT iii TABLE OF CONTENTS iv LIST OF FIGURES vi ABBREVIATIONS viii CHAPTER 1: INTRODUCTION 1.1 Topic Overview 1.2 Aim and scope 1.3 Methodology 1.4 Thesis summary CHAPTER 2: LITERATURE REVIEW 2.1 Optical Character recognition 2.1.1 OCR 2.1.2 Machine learning OCR with Tesseract 2.2 Methods in image processing 2.2.1 Grayscale Image 2.2.2 Noise reduction with a Gaussian filter 2.2.3 Binary with dynamic threshold 2.2.4 Canny Edge Detection 2.2.5 Filter number plate with contours 2.3 YOLO 10 2.3.1 About Yolo 10 2.3.2 How yolo works 10 2.3.3 Yolo structure 11 2.3.4 Loss Function 12 2.3.5 The network 12 CHAPTER 3: SYSTEM DESIGN 13 3.1 Overview 13 iv i Figure 8: Flowchart overview when entering 20 i Figure 9: Detailed flowchart upon entering The system's processing stages when a car enters the parking lot are shown in detail in Figure 3.9 The information on the car will initially be identified by the system after reading the RFID tag code The first step will be to read the car's license plate using the rear camera, the second step will be to determine the vehicle's color using the front camera when the vehicle enters, and the third step will be to determine the vehicle type using the parallel camera with the oncoming vehicle The system will continue encoding that data after fully reading the vehicle's information from the cameras installed surrounding the entry area before saving it in the system's database The system is in the form of a text file, whose name will be the RFID tag code that is initially read and whose content is the information that has been captured by the system's camera The barrier will then be opened for the vehicle to enter after the data has been saved 3.4 Flowchart of the system when going out 21 i When the vehicle leaves the parking lot, the system will process according to the diagrams below, Figure 3.10 is an overview of how the system will handle when the car comes out, and Figure 3.11 will describe the operation steps in detail system dynamics Figure 10: Flowchart overview when going out 22 i Figure 11: Detailed flowchart upon going out Figure 3.11 illustrates in detail the system's processing stages when a car pulls out of a parking space The information on the car will initially be identified by the system after reading the RFID tag code The car's license plate will be read using the rear camera first, followed by the color of the vehicle being determined using the front camera when the vehicle enters and finally the type of vehicle being determined using the side camera parallel to the path taken by the car The system will take the read code of the RFID first in order to be able to retrieve the data of that code from the system database after thoroughly reading the vehicle's information through the cameras positioned surrounding the departing area The system will then compare the information from that card code with the information retrieved from the vehicle as it exits the parking lot If those two data match, the vehicle will be allowed to leave, otherwise, the vehicle will not be allowed to leave the parking lot 23 i CHAPTER 4: EXPERIMENT RESULTS 4.1 Simulation We can assess the system through simulation to determine whether it satisfies the basic requirements and to better understand how the proposed system functions or not A block with the role of reading RFID card codes, a block with the purpose of processing photos of license plates, and a block with the function of color recognition will all be included in the simulation of the parking lot system's five components, as illustrated in Figure 4.1 One block will be responsible for identifying the vehicle, and the final block will be responsible for acting when the system has done processing the aforementioned data The simulation model is shown below: Figure 1: Function simulation blocks The Test RFID block will have the function of recognizing the card code, while the Test License Plate block will have the function of recognizing the number plate, the Test Vehicle Color block will perform the function of recognizing the color of the vehicle and the Test Vehicle Properties block will vehicle type identifier After the blocks have performed their functions, they will send the data to the Processing block for processing Finally, after the processing is complete, the vehicle has a signal that is sent to the Execution block to the next things, such as opening the barrier so that the car can go in 4.1.1 Simulate receiving RFID card code Figure 4.2 below will describe a real-life RFID card identification system The card code will be read by inserting the card into the reader then the card reader will receive the card code and transmit it to the computer 24 i Figure 2: Realistic model of RFID tag reading However, in this topic, it is carried out by simulation method, so the way to get the RFID card code will be done by manually entering the card code That process will be shown in Figure 4.3 Figure 3: Test Read RFID Tag 4.1.2 Simulate number plate recognition For license plate recognition, the camera will use the camera to get data and then put that data into the system for identification and processing Therefore, in order to simulate how the system will recognize the number plate, it will be done by putting a photo 25 i of the license plate into the system for the system to recognize and output license plate data Figure 4.4 simulates how license plates are recognized Figure 4: Demo of license plate recognition system However, the system's ability to recognize still has some limitations, due to misidentification of characters, influence of brightness, etc Figure 4.5 below shows that the times the number plate recognition system is wrong, the system mistakenly recognizes the letter "D" as the number "0" Figure 5: Wrong number plate recognition system 26 i 4.1.3 Simulate vehicle paint color recognition As for the car's paint color recognition, it will get data from the camera located in the front of the car of the parking system Therefore, to simulate the car's paint color recognition is done by taking a photo of the front of the car, which is usually the main color on the car Figure 4.6 below will show that more clearly: Figure 6: Demo of motorcycle color recognition However, since this is manual color recognition, there will be many errors in recognition Figure 4.7 below represents that misidentification: Figure 7: Wrong color recognition 27 i 4.1.4 Model of vehicle identification This is the last information that the system receives to enhance the security of the parking system However, this is the most important information in improving this security system Figure 4.8 and figure 4.9 show the result of vehicle type recognition of the system: Figure 8: Scooter identification Figure 9: Identification of motorcycles 28 i Besides, because the model training data is not many and lacks diversity, there are many cases where it will be wrong Figure 4.10 below will show that: Figure 10: Wrong vehicle identification 4.2 Model of the entire system After performing separate demos for each part of the system, those parts will be combined together to form a complete system A complete system will consist of two parts The first part will be the handling part of the entering lane and the rest will be the handling part of the system in the exit lane Figure 4.11 below shows how the system works when the vehicle is in 29 i Figure 11: Process of the system when the vehicle enters After processing, the system will save the information into a txt file with the file name being the tag code, and the content of the file is the information that the system has encrypted, and that encrypted string has the length is 97 bits In which, there are 72 bits coding for the number plate, 24 bits coding for the paint color of the car and the last bit coding for the vehicle type Figure 4.12 shows how the system stores the information after processing Figure 12: Information storage system Next, it will come to the processing part of the system when a certain vehicle comes out Figure 4.13 below shows how the system behaves 30 i Figure 13: The system's processing process when a vehicle comes out Figure 4.14 below shows how the system will recognize when the data read from the camera does not match the data stored in the database Figure 14: The system recognizes when the outgoing data is different from the card's stored data 31 i CHAPTER 5: CONCLUSION AND FURTHER WORK 5.1 Conclusion After researching and implementing the topic, the group achieved the following results: - Gain a better understanding of how a semi-automated parking system works - Improve some weaknesses of the old car park system, thereby improving service quality and ensuring safety for vehicles parked in the parking lot - Simulating the built system, from the process of the vehicle entering to the exiting process With the help of the aforementioned results, the team concluded as follows: "The simulation system generally served the group's intended aim." Despite having a low accuracy rate, the new system somewhat makes up for some of the old system's shortcomings Only the car color identification process remains somewhat manually developed with the system the team created, and this is the area where it influences the system's accuracy In addition to the vehicle's paint color, the classification of the vehicle is done automatically, but because there is little diversity in the data, it also lowers the system's recognition rate 5.2 Further work Through the weaknesses that the team has recognized, in order to have the most complete system, the team will need to focus on automating the most accurate identification of the vehicle's paint color, and data enhancement to be able to create a model that can best identify the vehicle type Besides, it is also necessary to create many encryption methods to prevent crooks from easily infiltrating and stealing data information Using a dedicated camera for license plate recognition because it is resistant to fog, dark, bright, Use other image processing algorithms to better determine license plate position such as Hough transform method for line recognition, color identification, algorithms that limit image movement when the vehicle is moving 32 i REFERENCE [1] Dang Xuan Truong, Kim Eung Tae - Robust Motorbike License Plate Detection and Recognition using Image Warping based on YOLOv2 [2] Adrian Rosebrock - OpenCV: Automatic License / Number Plate Recognition (ANPR) with Python [3] M.H.ter Brugge; J.H.Stevens; recognition using DTCNNs J.A.G.Nijhuis; L.Spaanenburg - License plate [4] Asep Haryono; Sahid Bismantoko; Gilang Mantara Putra; Tri Widodo - Accuracy in Object Detection based on Image Processing at the Implementation of Motorbike Parking on The Street [5] Damitha S.B Tilakaratna and Ukrit Watchareeruetai - Image Analysis Algorithms for Vehicle Color Recognition [6] Chirag Patel, Atul Patel PhD, Dharmendra Patel - Optical Character Recognition by Open-Source OCR Tool Tesseract: A Case Study - October 2012 [7] Nguyen Viet Anh – Image filtering - September 29, 2018 [8] Abeer George Ghuneim – Contour tracing – 2000 [9] Intel Corporation, Willow Garage, Itseez - Contour Features - https://docs.opencv.org [10] Redmon, J., Divvala, S., Girshick, R., Farhadi, A - You only look once: real-time object detection In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 779–788 (2016) 33 i S i K L 0