Anti theft system design using opencv on raspberry pi 4

MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING CAPSTONE PROJECT ELECTRONIC AND COMMUNICATION ANTI-THEFT SYSTEM DESIGN USING OPENCV ON RASPBERRY PI LECTURER: Ph.D DO DUY TAN STUDENT: NGUYEN XUAN THANG LUU DINH PHUONG NGUYEN SKL 0 Ho Chi Minh City, 2022 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING - - GRADUATION PROJECT ANTI-THEFT SYSTEM DESIGN USING OPENCV ON RASPBERRY PI NGUYEN XUAN THANG Student ID: 18161038 LUU DINH PHUONG NGUYEN Student ID: 18161027 Major: ELECTRONIC AND COMMUNICATION ENGINEERING Advisor: Ph.D DO DUY TAN Ho Chi Minh City, July 2022 THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness *** -Ho Chi Minh City, July 29th, 2021 ADVISOR’S EVALUATION SHEET Student name: Nguyen Xuan Thang Student name: Luu Dinh Phuong Nguyen Student ID: 18161038 Student ID: 18161027 Major: Electronics and Communication Engineering Project title: Anti-theft system design using OpenCV on Raspberry Pi Advisor: Ph.D Do Duy Tan EVALUATION Content of the project: The content and workload of the project meets the initial requirements of the project Strengths: The topic of the project is practical and applicable Extensive experimental results are conducted to evaluate the performance of the proposed design Weaknesses: − There are many grammatical mistakes and typos in the report − The performance of the proposed design might be unstable in environments that light intensity is low Approval for oral defense? (Approved or denied) Approved Overall evaluation: (Excellent, Good, Fair, Poor) Good Mark: 8.0 (in words: Eight ) Ho Chi Minh City, July 29th, 2022 ADVISOR (Sign with full name) iii THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -Ho Chi Minh City, July 29th, 2022 PRE-DEFENSE EVALUATION SHEET Student name: Nguyen Xuan Thang Student ID: 18161038 Student name: Luu Dinh Phuong Nguyen Student ID: 18161027 Major: Electronics and Communication Engineering Project title: Anti-theft system design using OpenCV on Raspberry Pi Name of Reviewer: EVALUATION Content and workload of the project: Strengths: Weaknesses: Approval for oral defense? (Approved or denied): Overall evaluation: (Excellent, Good, Fair, Poor): Mark: ………… (in works:…….………….….………….……………) REVIEWER (Sign with full name) iv THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -Ho Chi Minh City, July 29th, 2022 EVALUATION SHEET OF DEFENSE COMMITTEE MEMBER Student name: Nguyen Xuan Thang Student ID: 18161038 Student name: Luu Dinh Phuong Nguyen Student ID: 18161027 Major: Electronics and Communication Engineering Project title: Anti-theft system design using OpenCV on Raspberry Pi Advisor: Ph.D Do Duy Tan Name of Defense Committee Member: EVALUATION Content and workload of the project: Strengths: Weaknesses: Overall evaluation: (Excellent, Good, Fair, Poor): Mark: ………… (in works:…….………….….………….……………) Ho Chi Minh City, day… month… year COMMITTEE MEMBER (Sign with full name) v HCMC University of Technology and Education Faculty for High Quality training Socialist Republic of Vietnam Independence - Liberty – Happiness Ho Chi Minh City, July 29th, 2021 PROJECT ASSIGNMENT Name of Members: Nguyen Xuan Thang Student ID: 18161038 Luu Dinh Phuong Nguyen Student ID: 18161027 Training system: Regular university Major: Electronics and communication engineering Class: 18161CLA I NAME OF PROJECT: ANTI-THEFT SYSTEM DESIGN USING OPENCV ON RASPBERRY PI II ASSIGNMENT The original figures − Recognize the urgency and importance of the project − Conducting research, studying materials, current models related to the topic for comparison and improvement Content of implementation Task 1: Brainstorm and research related topics Task 2: Research and select components Task 3: Draw algorithm flowchart Task 3: Design and build a complete system model Run tests and calibrate the system Task 4: Write a report III DATE OF RECEIVING TASK: 14/03/2021 IV DATE OF TASK COMPLETION: 27/06/2021 V ADVISOR: Ph.D Do Duy Tan CHAIR OF THE PROGRAM (Sign with full name) ADVISOR (Sign with full name) i HCMC University of Technology and Education Faculty for High Quality training Socialist Republic of Vietnam Independence - Liberty – Happiness Ho Chi Minh City, July 29th, 2022 PROJECT IMPLEMENT SCHEDULE Name of Members: Nguyen Xuan Thang Student ID: 18161038 Luu Dinh Phuong Nguyen Student ID: 18161027 Class: 18161CLA Name of project: Anti-theft system design using OpenCV on Raspberry Pi Week/Date Content 1st - Meet the Instructor to disseminate regulations: (14/3-20/3) choose the topic and working time - Instructor conducts a review of the topic 2st (21/3-27/3) - Write the plan and detailed outlines for the topic - Write a summary of the requirements of the selected topic 3nd (28/3-4/4) 4rd (5/4-11/4) 5th (12/4-18/4) 6th (19/4-25/4) - Learn some topics similar to the selected topic - Learn the components used in the selected topic - Design block diagram - Design the principal diagram and explain the function of each block ii Lecture Confirm 7th (26/4-2/5) 8th Ss - Write a program to detect human face in motion (3/5-9/5) and test 9th (10/5-16/5) 10th - Write a program to alert users via Gmail (17/5-23/5) 11th (24/5-30/5) account and web server - Model construction and design 12th (31/5-6/6) 13th - Implement, test and fix the project - Write the report (7/6-13/6) - Complete the report and send it to Instructor to 14th (14/6-20/6) 15th (21/6-27/6) review and comment for the last time before printing the report - Submit the report and slide PowerPoint ADVISOR (Sign with full name) STUDENTS (Sign with full name) iii DISCLAIMER We guarantee that this project is our research, under Ph.D Do Duy Tan's guidance The results published in this project are honest and are not replicated from any other work Topic: ANTI-THEFT SYSTEM DESIGN USING OPENCV ON RASPBERRY PI Instructor: Ph.D DO DUY TAN Student 1: NGUYEN XUAN THANG Student 1’s ID: 18161038 Class: 18161CLA Email: 18161038@student.hcmute.edu.vn Student 2: LUU DINH PHUONG NGUYEN Student 2’s ID: 18161027 Class: 18161CLA Email: 18161027@student.hcmute.edu.vn STUDENTS Luu Dinh Phuong Nguyen iv Nguyen Xuan Thang ACKNOWLEDGMENTS After the process of learning, researching, implementing capstone project and completing on schedule, we sincerely thank our teachers, family and my friends who wholeheartedly help, give advice and encouragement in difficult times We would like to thank Mr Do Duy Tan, a lecturer at HCMC University of Technology and Education, for standing side by side as well as creating all conditions for our group to be able to perform well on their topic And we would like to extend our gratitude to the teachers in the Electrical and Electronic Faculty of the University of Technology and Education, Ho Chi Minh City who are dedicated to teach us the basic knowledge and give us a solid knowledge base to complete this topic Family is always a source of motivation and inspiration for us to continually strive and complete the topic, thank you for the birth that always encouraged us in the most frustrating times And finally, we would like to thank the brothers, sisters and friends in the class for their sincere advice and suggestions so that our topic can be more complete Once again, we would like to thank everyone for always being by our side and helping us, creating the motivation for us to perfect this topic Sincerely! STUDENTS Luu Dinh Phuong Nguyen v Nguyen Xuan Thang Figure 4.12 Object detection system looking to the left Figure 4.13 The system captures the face of the subject looking to the left Similarly, in cases of right tilt, the facial recognition system leans to the right at an angle of about 30° as shown in Figure 4.14 40 Figure 4.14 The system captures the subject's face looking to the right If the face is tilted too much, the system will not be able to identify the subject and cannot recognize the face The face is too tilted in Figure 4.15, the subject's sitting posture in Figure 4.16 is affected, and the image taken in bad conditions in Figure 4.17 causes the system to fail to recognize the face Figure 4.15 The face is tilted so much that the system does not recognize the object 41 Figure 4.16 The object is not recognized by the system Figure 4.17 Unrecognizable case due to sitting position In case two people appear in front of the camera lens, the system will detect and recognize both object at the same time when the light is strong enough as shown in Figure 4.18 and only detect one object when the light is not good enough as shown in Figure 4.19 42 Figure 4.18 Two object recognition system Figure 4.19 The system recognizes only one object In the case of multiple people appearing in front of the camera lens, the system will detect and recognize objects when lighting conditions are good as shown in Figure 4.20 43 Figure 4.20 Multiple object recognition system In the test case with masks, our team implemented our model with a single person case However, the system only detects the object standing in front of the lens but does not identify the object's face as shown in Figure 4.21 Figure 4.21 One person with mask The data will be used to test the system's real-time human face recognition capability with the 5MP module camera in the case of students without masks for face recognition in Table 4.1 No Table 4.1 Testing conducted with students without mask Sample Testing Distance Time Results Accuracy (meters) 44 Xuan Thang Phuong Nguyen Hoan Hao 7h30 Detected 9h 10h30 Detected Detected 15h 17h Detected Detected 18h30 Detected 8h Detected 9h30 11h Detected Detected 15h30 17h30 Detected Detected 18h30 8h30 10h Not Detected Detected Detected 11h30 16h 18h 18h30 Detected Detected Not Detected Detected 100% 83,33% 83,33% Statistical Table 4.1 presents the algorithm that our team built and used quite simply but still achieved high performance compared to the previous requirement In the human motion detection test in the case of without a mask, the system used a 5MP camera module using the OpenCV algorithm library and combined the PIR sensor module with 100% detection results people passing by During the high-intensity sunlight exposure test period from 8am to 4pm, the combination of the OpenCV algorithm, the Dlib algorithm, and the PIR sensor had a detection rate and accuracy of up to 100% and the rate will gradually decrease over time as it starts to get dark and there is not enough light The results from the experimental table show that the OpenCV library's human face recognition feature is not affected too much by the effects of light control The PIR motion sensor module, on the other hand, depends on the heat output to sense changes in motion, potentially triggering intermittently when exposed to sunlight Because IR emitted from the sun can create an anomalous situation leading to false alarm results Haar-Cascade algorithm is implemented to detect human and pet faces as they move through the system In summary, when conducting the experiment, the detection rate of motion and human face exceeded the expected result with a recognition rate of over 88% 45 4.2 REVIEW AND EVALUATION 4.2.1 Review The construction model implemented by the team has achieved the goal about 85% set by ourselves The detection rate of the subject's face when moving is quite high, about 95%, depending on the viewing angle of the subject in front of the camera lens The correct detection rate of a human face also depends relatively little on viewing angles and light intensity The model can detect many faces at the same time, but only compared with a folder of photos, so it can only recognize one person at the same time The system sends alerts about the user's Gmail account and updates the image and time information to the web server, accounting for almost 98% of the time When starting the system, the face detection system is still quite long with minute 4.2.2 Evaluation Objective assessment of image processing speed as well as sending warning information to users via Gmail account and web server of the system achieved a fairly accurate rate However, the actual operation model needs fast face detection with high accuracy, but the operation model is still unstable because our team only uses a 5MP camera module, the configuration of the Raspberry module has not met the needs and programs The code is not optimized Due to time constraints, as well as cost limitations, the implementation team has not been able to solve the arising problems or improve the features to make the system more efficient to meet actual needs The model of the construction team only stops at the level of compact and simple construction and solves the basic goal of face detection combined with additional features to warn users by Gmail application and send information warn users to go to the web server with the knowledge of image processing and machine learning learned in school classrooms 46 CHAPTER 5: CONCLUSION 5.1 CONCLUSION After a period of research and building a complete model, the project " Anti-theft system design using OpenCV on Raspberry Pi 4" has completed the goal as our team suggestion The initial purpose of the topic includes the use of knowledge of programming, image processing, computer vision to apply it to solve difficult problems of society, namely the problem of social problems security issues, property theft Our products are created with high applicability for monitoring small personal areas to reduce or prevent theft, suitable for installation in locations such as homes, personal offices, small workshops, and entrance to the parking loTs Due to limited time and limited knowledge, our group still has certain limitations as we not have a backup power source to supply the system to operate when the main power is lost The motion detection and face detection systems are still not working at their best, especially in conditions that are too dark or too bright The whole system operates relatively stable, although sometimes it is still affected by the impact response of the hardware system The warning and communication system works well when alerting users around, sending email notifications and updating the webserver continuously in real-time and with high accuracy With the results set out by our team, anti-theft security systems using IoT can be implemented in places where it is necessary to ensure problems of order, security or property values to help people's lives people much easier, comfortable and secure The entire system can be monitored from a remote location and can be controlled using internetenabled devices anywhere Therefore, IoT can help us improve the quality of life from simple to complex tasks, creating maximum convenience and comfort 5.2 DISCUSSION For future work, our team has several improvements that can be made Design additional backup power using solar cells and batteries so that when the main power is lost, the system can still operate normally Upgrade web server interface with fast update speed and user-friendly The camera module can be upgraded to an infrared camera with better image quality and is not affected by lighting conditions because it uses infrared technology to take pictures The facial recognition algorithm can still be improved because the results are still far from perfect 47 APPENDIX Program Codes: • Code to take a picture and send it to Gmail import RPi.GPIO as GPIO import datetime import time from picamera import PiCamera import os import yagmail from face_detection import face_detection from face_detection import update_photo_log_file_face_detection PIR_PIN = LED_PIN = 17 LOG_FILE_NAME = "/home/pi/camera/photo_logs.txt" LOG_FACE_DETECTION_NAME = "/home/pi/camera/photo_logs_face_detection.txt" def take_photo(camera): file_name = "/home/pi/camera/img_" + ' ' + str(datetime.datetime.now()) + ".jpg" camera.capture(file_name) return file_name def update_photo_log_file(photo_file_name): with open(LOG_FILE_NAME, "a") as f: f.write(photo_file_name) f.write("\n") def send_email_with_photo(yagmail_client, file_name): yagmail_client.send(to="xuanthangphuongthao3004@gmail.com", subject="Movement detected!", contents="Here's a photo taken by your Raspberry Pi", attachments=file_name) # Setup camera camera = PiCamera() camera.resolution = (1280, 720) #camera.rotation = 180 print("Waiting seconds to init the camera ") time.sleep(2) print("Camera setup OK.") # Remove log file if os.path.exists(LOG_FILE_NAME): os.remove(LOG_FILE_NAME) print("Log file removed.") if os.path.exists(LOG_FACE_DETECTION_NAME): 48 os.remove(LOG_FACE_DETECTION_NAME) print("Log file removed.") # Setup yagmail password = "" with open("/home/pi/.local/share/.email_password", "r") as f: password = f.read() yag = yagmail.SMTP("xuanthangphuongthao311@gmail.com", password) print("Email sender setup OK.") # Setup GPIOs GPIO.setmode(GPIO.BCM) GPIO.setup(PIR_PIN, GPIO.IN) GPIO.setup(LED_PIN, GPIO.OUT) GPIO.output(LED_PIN, GPIO.LOW) print("GPIOs setup OK.") MOV_DETECT_TRESHOLD = 3.0 last_pir_state = GPIO.input(PIR_PIN) movement_timer = time.time() MIN_DURATION_BETWEEN_2_PHOTOS = 60.0 last_time_photo_taken = print("Everything has been setup.") try: while True: time.sleep(0.01) pir_state = GPIO.input(PIR_PIN) if pir_state == GPIO.HIGH: GPIO.output(LED_PIN, GPIO.HIGH) time.sleep(0.5) GPIO.output(LED_PIN, GPIO.LOW) time.sleep(0.5) else: GPIO.output(LED_PIN, GPIO.LOW) if last_pir_state == GPIO.LOW and pir_state == GPIO.HIGH: movement_timer = time.time() if last_pir_state == GPIO.HIGH and pir_state == GPIO.HIGH: if time.time() - movement_timer > MOV_DETECT_TRESHOLD: if time.time() last_time_photo_taken MIN_DURATION_BETWEEN_2_PHOTOS: print("Take photo and send it by email") photo_file_name = take_photo(camera) face_detection_file_name = face_detection(photo_file_name) update_photo_log_file(face_detection_file_name) update_photo_log_file_face_detection(face_detection_file_name) # send_email_with_photo(yag, photo_file_name) last_time_photo_taken = time.time() 49 > last_pir_state = pir_state except KeyboardInterrupt: GPIO.cleanup() • Web server interface design from flask import Flask, render_template import os CAMERA_FOLDER_PATH = "/home/pi/camera" LOG_FACE_DETECTION_NAME = CAMERA_FOLDER_PATH "/photo_logs_face_detection.txt" + photo_counter = app = Flask( name , static_url_path=CAMERA_FOLDER_PATH, static_folder=CAMERA_FOLDER_PATH) @app.route("/") def index(): return render_template('index.html') @app.route("/check-movement") def check_movement(): message = "" line_counter = last_photo_file_name = "" if os.path.exists(LOG_FACE_DETECTION_NAME): with open(LOG_FACE_DETECTION_NAME, "r") as f: for line in f: line_counter += last_photo_file_name = line global photo_counter difference = line_counter - photo_counter message = str(difference) + " photos were taken since you last checked " message += "Last photo: " + last_photo_file_name + "" message += "" photo_counter = line_counter else: message = "Nothing new" return message app.run(host="0.0.0.0", port=8500) • facial recognition program code import cv2 50 import dlib import time import datetime LOG_FACE_DETECTION_NAME = "/home/pi/camera/photo_logs_face_detection.txt" def face_detection(img): # read the image img = cv2.imread(img) # conver img to grayscale: 3D -> 2D gray = cv2.cvtColor(src=img, code=cv2.COLOR_BGR2GRAY) # dlib: Load Face Recognition Detector face_detector = dlib.get_frontal_face_detector() # use detector to find face landmarks faces = face_detector(gray) for face in faces: x1 = face.left() # left Point y1 = face.top() # top Point x2 = face.right() # right Point y2 = face.bottom() # bottom Point # Draw a rectangle cv2.rectangle(img=img, pt1=(x1, y1), pt2=( x2, y2), color=(0, 255, 0), thickness=3) file_name_face_detection = "/home/pi/camera/img_face_detetion" str(datetime.datetime.now()) + ".jpg" cv2.imwrite(file_name_face_detection, img) return file_name_face_detection def update_photo_log_file_face_detection(file_name_face_detection): with open(LOG_FACE_DETECTION_NAME, "a") as f: f.write(file_name_face_detection) f.write("\n") # # show the image # cv2.imshow(winname="Face Recognition App", mat=img) # # # wait for a key press to exit # cv2.waitKey(delay=0) # # # Close All windows # cv2.destroyAllWindows() 51 + " " + REFFERENCES [1] Zhang, Z.; Yi, D.; Lei, Z.; Li, S.Z Regularized Transfer Boosting for Face Detection Across Spectrum IEEE Signal Process Lett 2012, 19, 131–134 [2] Bộ Công an công bố số liệu thống kê năm 2021 http://bocongan.gov.vn/tin-tuc-su-kien/bo-cong-an-cong-bo-so-lieu-thong-ke-nam-2021d17-t30879.html [3] Tăng Nguyễn Công Thiên, “Ứng dụng xử lý ảnh thiết kế thi công mạch chống trộm thông minh”, Đồ án tốt nghiệp, Trường Đại học Sư phạm Kĩ thuật, Tp.HCM, 2018 [4] Dương Chí Hùng, “Thiết kế mạch chống trộm”, Đồ án tốt nghiệp, Trường Đại học BÀ RỊA VŨNG TÀU, 2020 [5] Trần Quốc Huy, Trần Ngọc Vinh, “Hệ thống báo trộm báo cháy thông minh”, Đồ án tốt nghiệp, Trường Đại học Sư phạm Kĩ thuật, Tp.HCM, 2017 [6] Prince Patel, Shoban Babu, Priyadharshini V, Raspberry pi projects – study of best fit wireless prototypes, February 2021 [7] Khalid M Hosny, Mohamed M Darwish, Kenli Li, Ahmad Salah, COVID-19 diagnosis from CT scans and chest X-ray images using low-cost Raspberry Pi, 2021 [8] Raspberry Pi Computer Model B, June 2019 [9] Raspberry Pi Model B Datasheet https://datasheets.raspberry.com/rpi4/raspbrtty-pi-4-datasheet.pdf [10] Saidi Ramadan Siregar, Pristiwanto, Build Data Backup with Nextcloud Based Infrastucture as A Service (IAAS) Concept on Budi Darma University, March 2021 [11] 5MP Camera User Manual https://docs.rs-online.com/b064/A700000006917308.pdf [12] P.Dharani Devi and C A Pavithra, PIR Sensor based Blaze Barricade using Raspberry Pi, 2021 [13] PIR Motion Sensor Datasheet https://cdn-learn.adafruit.com/downloads/pdf/pir-passive-infrared-proximity-motionsensor.pdf [14] Nguyễn Thanh Tuấn, "Deep Learning bản", 2019 [15] Md Zahangir Alom, Tarek M Taha, Chris Yakopcic, Stefan Westberg, Paheding Sidike, Mst Shamima Nasrin, Mahmudul Hasan, Brian C Van Essen, Abdul A S Awwal and Vijayan K Asari, A State-of-the-Art Survey on Deep Learning Theory and Architectures, 2019 [16] Facundo Brea, Juan M Gimenez, Vıctor D Fachinotti, Prediction of wind pressure coefficients on building surfaces using Artificial Neural Networks, 2017 [17] Nguyen Quang Hoan, Xử lý ảnh, Học viện công nghệ bưu viễn thơng, 2006 [18] P Kenda, A Witanti, Sistem Presensi Berbasis Wajah Dengan Metode Haar Cascade, 2021 52 [19] P.V.P Sai Kiran, Sujith Reddy Vangala, Mamoon Rashid, Efficient Approach for Measuring Heart Pulse from Real Time Facial RGB Color Videos, 2021 [20] Kavita Khobragade, A Comparative study of Converting Coloured Image to Grayscale Image using Different Technologies, 2017 [21] Patrick Kennedy, Deep Dive into Flask's Application and Request Contexts, March 1st, 2022, https://testdriven.io/blog/flask-contexts-advanced/#what-is-a-context 53 S K L 0