(Đồ án hcmute) research, design and implementation of switch in network device connecting with flashing devices and multiple ecus

MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION THESIS MECHANICAL ENGINEERING TECHONOLOGY RESEARCH, DESIGN AND IMPLEMENTATION OF SWITCH-IN NETWORK DEVICE CONNECTING WITH FLASHING DEVICES AND MULTIPLE ECUS ADVISOR: MS NGUYEN MINH TRIET STUDENT: NGUYEN VIET HIEP NGUYEN DINH QUANG KHAI SKL Ho Chi Minh City, Febuary, 2023 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF HIGH QUALITY TRAINING GRADUATION PROJECT RESEARCH, DESIGN AND IMPLEMENTATION OF SWITCH-IN NETWORK DEVICE CONNECTING WITH FLASHING DEVICES AND MULTIPLE ECUS STUDENT’S NAME AND ID: NGUYEN VIET HIEP - 18146017 NGUYEN DINH QUANG KHAI - 18146031 MAJOR: MECHATRONICS ENGINEERING INSTRUCTOR: MS NGUYEN MINH TRIET HO CHI MINH CITY, 2/2023 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING Capstone Project Proposal Student name: Nguyễn Đinh Quang Khải Student ID: 18146031 Student name: Nguyễn Việt Hiệp Student ID: 18146017 Advisor: Nguyễn Minh Triết ✆: 0338739318 ✆: 0776262656 ✆: 0915719984 Title of Capstone Project: RESEARCH, DESIGN AND IMPLEMENTATION OF SWITCH-IN NETWORK DEVICE CONNECTING WITH FLASHING/DEBUG DEVICES AND MULTIPLE ECUS USING FOR LABORATORIES Initial materials: Microcontroller: Raspberry Pi Size and tolerance of device The number of devices and ECU connected with the switch box Using step motor for choosing communication port Main Content: Microcontroller: Raspberry Pi Size and tolerance of device The number of devices and ECU connected with the switch box Using step motor for choosing communication port Expected Results Real prototype of switch device Lab booking app Project report Delivery date: Submission date: Language: Report: Oral presentation: Dean English English Head of Department • • Vietnamese • Vietnamese • Advisor THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -Ho Chi Minh City, Day Month……., Year… … MECHANTRONICS PROJECT ASSIGNMENT ADVISOR’S EVALUATION SHEET Student name: Nguyen Viet Hiep Student ID: 18146017 Student name: Nguyen Dinh Quang Khai Student ID: 18146031 Major: Mechatronics Engineering Project title: Research, Design And Implementation Of Switch-In Network Device Connecting With Flashing Devices And Multiple Ecus Advisor: MS Nguyen Minh Triet Defender Teacher: Ph.D Vu Quang Huy EVALUATION Content of the project: Strengths: Weaknesses: Approval for oral defense? (Approved or denied) Overall evaluation: (Excellent, Good, Fair, Poor) Mark:……………….(in ) words: Ho Chi Minh City, Day Month… , Year…… ADVISOR (Sign with full name) ACKNOWLEDGEMENT We would like to express our great gratitude to our honorable teacher, MS Nguyen Minh Triet, who supported us with proper guidance, sincere comments and advice, and led us in the right way to conduct our graduation thesis We are grateful to Ho Chi Minh City University of Technology and Education, The Faculty For High Quality Training and The Faculty Of Mechanical Engineering for giving us an intensive learning environment so that we could equip ourselves with both practical experience and necessary skills to encounter realistic working conditions We also would love to thank all of my respected teachers who heartedly impart to us the knowledge during four years At last, we are thankful to our family and friends for their help during our learning period Their faith in me has kept my spirits and motivation up throughout this process We would like to express our heartfelt appreciation and gratitude to our colleagues at Bosch Global Software Technologies Your encouragement and guidance have enabled us to complete our process in this thesis Your guidance on both research and my career has been invaluable We will strive to use gained skills and knowledge in the best possible way in order to attain desired career objectives Sincerely, TABLE OF CONTENTS Contents ACKNOWLEDGEMENT TABLE OF CONTENTS LIST OF FIGURE LIST OF TABLES ABSTRACT CHAPTER 1: INTRODUCTION 10 1.1 MOTIVATION FOR THE RESEARCH 11 1.2 REQUIREMENT FOR THE DEVICE 12 1.2.1 Scope And Objective of Project 12 1.2.2 Input Requirements 12 1.2.3 Output Requirements 12 1.3 METHOD TO SWITCH IN THE NETWORK 13 1.3.1 Messages Switching Method 13 1.3.2 Packet Switching Method 14 1.3.3 Circuit switching method 14 CHAPTER 2: LITERATURE REVIEW 16 2.1 DEVICE IN THE SYSTEM SPECIFICATIONS REVIEW 17 2.1.1 IC5000 17 2.1.2 Vector VX1000 series 19 2.2 JTAG Interface Of ECU 20 2.3 NETWORK SWITCHING REVIEW 21 2.3.1 Network topology 21 2.3.2 Internet of Things (IoT) 22 2.3.3 Architecture of a network 23 2.3.4 Database 24 2.4 MECHANISM ANALYSIS 26 2.4.1 Circuit switching method phase 26 2.4.2 Mechanical selection 26 2.4.3 Motor selection 32 2.5 MACHINE VISION 35 2.5.1 General About Image Processing 35 2.5.2 Features Haar – Like 37 2.5.3 AdaBoost Algorithm 39 2.5.4 Multi-tasking Cascaded Convolutional Networks (MTCNN) 39 CHAPTER 3: DESIGN AND IMPLEMENTATION 41 3.1 BLOCK DIAGRAM 42 3.1.1 Block Listing 42 3.1.2 Block Diagram 43 3.1.3 Mechanical Block Diagram 44 3.2 OPERATION DESCRIPTION 46 3.3 FLOWCHART DESIGN 47 3.3.1 Controlling Flowchart 47 3.3.2 Switch Box Controller And Function Flwchart 48 3.3.3 Machine Vision Method Flowchart 50 3.4 PACKAGING DESIGN 51 3.5 ELECTRICAL DESIGN 55 3.5.1 The Slection Of Electric Components 55 3.5.2 Schematic Diagram 63 3.6 MECHANICAL DESIGN 65 3.6.1 Mechanical Drive Design 65 3.6.2 Mechanical Guide Design 67 3.7 FACE RECOGNITION 70 3.8 SWITCH BOX TOOL DESIGN 73 CHAPTER 4: DESIGN ANALYSIS 79 4.1 ESTABLISH TARGET SPECIFICATIONS 80 4.1.1 Qualitative Statements 80 4.1.2 List of metrics 81 4.1.3 Proposed specifications and experimental methods 82 4.2 EXPERIMENTS 82 4.2.4.1 Static Tests 82 4.2.2 Dynamic test: 82 CHAPTER 5: CONCLUSIONS 99 5.1 ACHIEVEMENTS 100 5.2 PROBLEMS AND SOLUTION 101 5.3 FUTURE WORK 101 REFERENCES 102 APPENDIX 1: 103 APPENDIX 2: 117 LIST OF FIGURE Figure 1: Rack setup for switching device 12 Figure 2: Message switching method diagram 13 Figure 3: Packet switching diagram 14 Figure 4: Circuit switching diagram 15 Figure 5: IC500 specification table 17 Figure 6: Bluebox device IC5000 18 Figure 7: 14-pin 2.54mm MPC5xxx Debug Adapter 18 Figure 8: VX1060 Base module specsification 19 Figure 9: VX1060 Base module 19 Figure 10: Bosch USS ECU 21 Figure 11: Types of network topology 22 Figure 12: SQLite Database 25 Figure 13: LAN shared folders 25 Figure 14: Lead screw – nut drive 26 Figure 15: Lead scew structure 27 Figure 16: Gear bell drive 28 Figure 17: Rack and pinion gear drive 29 Figure 18: Linear Motion Bearing 31 Figure 19: Stepper Motor 32 Figure 20: Principle diagram of DC servo motor with feedback 33 Figure 21: DC Motor 34 Figure 22: Step in image proccessing diagram 35 Figure 23: Some feature of Haar-Like 37 Figure 24: How to calculate the Integral Image of an image 38 Figure 25: How to calculate the total pixel value of the area to be calculated 38 Figure 26: Block diagram of AdaBoost face detection stage 39 Figure 27: P-Net network architecture 40 Figure 28: R-Net network architecture 40 Figure 29:O-Net network architecture 40 Figure 30: Block diagram 43 Figure 31: Mechanical BLock Diagram Of Switch Box 44 Figure 32: Controlling Flowchart 47 Figure 33: Controlling Flowchart 48 Figure 34: Machine Vision Method Flowchart 50 Figure 35: Switch Box Body 3D design 51 Figure 36: Operating Header 3D Design 52 Figure 37: Bearing holder 3D design 53 Figure 38: Track Holder 3D Design 53 Figure 39: 8mm bearing 3D design 54 os.popen(path_hs_display) except: if not os.path.exists(Path_HS): Write_txt(Path_HS,"Error") def app(): win_GUI_1 = Tk() # This is to create a basic windo xw win_GUI_1.geometry("1020x500") # this is for the size of the window win_GUI_1.resizable(0, 0) # this is to prevent from resizing the window win_GUI_1.title("SWITCH BOX CONTROLLER") label1_GUI_1 = tk.Label(win_GUI_1, text='Choose LabCar') label1_GUI_1.place(x=20,y=20) # # def lc_app(): llc_app = Tk() llc_app.geometry("240x200") # lc_table = Frame(llc_app) # lc_table.place(x=200,y=30) lc_scroll = Scrollbar(llc_app) lc_scroll.pack(side=RIGHT, fill=Y) # lc_scroll = Scrollbar(lc_table,orient='horizontal') # lc_scroll.pack(side= BOTTOM,fill=X) lc = ttk.Treeview(llc_app,yscrollcommand=lc_scroll.set) lc.pack() lc_scroll.config(command=lc.yview) # lc_scroll.config(command=lc.xview) lc_columns = 'LabCar','State' lc['columns']= (lc_columns) lc.column("#0", width=0, stretch=NO) lc.column("LabCar",anchor=CENTER, width=80) lc.column("State",anchor=CENTER, width=130) lc.heading("#0",text="",anchor=CENTER) lc.heading("LabCar",text="LabCar",anchor=CENTER) lc.heading("State",text="State",anchor=CENTER) with open(rlab_path) as f: reader = csv.DictReader(f) for row in reader: LabCar = row['LabCar'] State_2 = row['State'] lc.insert("", 'end', values=(LabCar,State_2)) llc_app.mainloop() options = [ "Select LC", "HC_C-000RA", "HC_UT30025C", "HC_UT31212C", "HC_UT31215C", "HC_UT31218C", "HC_UT40485C", "HC_UT40592C", "HC_UT40608C" ] # datatype of menu text clicked = StringVar(win_GUI_1) 108 # initial menu text clicked.set(options[0]) lab_id = OptionMenu(win_GUI_1, clicked, *options) lab_id.place(x=20,y=50) # for text, value, a in LabCar: # Gradio_1 = tk.Radiobutton(win_GUI_1, text=text, variable=Labb, value=value) # Gradio_1.place(x= a,y= 50) with open(rlab_path) as f: reader = csv.DictReader(f) for row in reader: LabCar = row['LabCar'] State_2 = row['State'] label1 = tk.Label(win_GUI_1, text='Please choose your LabCar') label1.place(x=80,y=83) # label_lc = tk.Label(win_GUI_1, text='Connect State: ' + clicked.get() + ' ' + State_2) # label_lc.place(x=80,y=123) def click1(value): label1.config(text='You choose ' + value) label1.place(x=80,y=83) def lc_state(): with open (rlab_path,'a') as f: fieldnames = ['0','LabCar','State'] writer = csv.DictWriter(f,fieldnames=fieldnames) writer.writerow({'0':'0','LabCar': clicked.get(), 'State': 'Connected'}) lc_app() # lc.insert('','end',values=(clicked.get(),"Connected")) def lc_state_2(): with open (rlab_path,'a') as f: fieldnames = ['0','LabCar','State'] writer = csv.DictWriter(f,fieldnames=fieldnames) writer.writerow({'0':'0','LabCar': clicked.get(), 'State': 'Disonnected'}) lc_app() # lc.insert('','end',values=(clicked.get(),'Connected')) # ser = serial.Serial('/dev/ttyACM0', 9600) # time.sleep(5) # ser.write('1') GButton_508=tk.Button(win_GUI_1, text = "Confirm",command=lambda: click1(clicked.get())) GButton_508.place(x=20,y=80) # # # # label3 = tk.Label(win_GUI_1,text='Connect State: ' + clicked.get() + ' ' + State_2) # label3.place(x=80,y=143) def click3(): label3 = tk.Label(text='') label3.place(x=80,y=123) ser = serial.Serial('/dev/ttyACM0', 9600) time.sleep(5) ser.write('1') 109 GButton_308=tk.Button(win_GUI_1, text = "Connect",command=lambda: lc_state(), bg='Green') GButton_308.place(x=20,y=120) def click4(): label4 = tk.Label(text='Start connecting ') label4.place(x=80,y=163) GButton_208=tk.Button(win_GUI_1, text = "Disconnect", command=lambda: lc_state_2()) GButton_208.place(x=20,y=160) # # lock_table = Frame(win_GUI_1) lock_table.place(x=450,y=30) lock_scroll = Scrollbar(lock_table) lock_scroll.pack(side=RIGHT, fill=Y) lock_scroll = Scrollbar(lock_table,orient='horizontal') lock_scroll.pack(side= BOTTOM,fill=X) status = ttk.Treeview(lock_table,yscrollcommand=lock_scroll.set, xscrollcommand =lock_scroll.set) status.pack() lock_scroll.config(command=status.yview) lock_scroll.config(command=status.xview) lock_columns = 'Manv','Lab','Status','Time' status['columns']= (lock_columns) status.column("#0", width=0, stretch=NO) status.column("Manv",anchor=CENTER, width=80) status.column("Lab",anchor=CENTER, width=130) status.column("Status",anchor=CENTER, width=160) status.column("Time",anchor=CENTER, width=180) status.heading("#0",text="",anchor=CENTER) status.heading("Manv",text="Manv",anchor=CENTER) status.heading("Lab",text="Lab",anchor=CENTER) status.heading("Status",text="Status",anchor=CENTER) status.heading("Time",text="Time",anchor=CENTER) # def labinfo(): with open(status_path) as f: reader = csv.DictReader(f) for row in reader: Manv = row['Manv'] Lab = row['Lab'] Status = row['Status'] Time = row['Time'] status.insert("", 'end', values=(Manv, Lab, Status, Time)) # # game_frame = Frame(win_GUI_1) game_frame.place(x=10,y=250) #scrollbar game_scroll = Scrollbar(game_frame) game_scroll.pack(side=RIGHT, fill=Y) game_scroll = Scrollbar(game_frame,orient='horizontal') 110 game_scroll.pack(side= BOTTOM,fill=X) set = ttk.Treeview(game_frame,yscrollcommand=game_scroll.set, xscrollcommand =game_scroll.set) set.pack() game_scroll.config(command=set.yview) game_scroll.config(command=set.xview) set_columns = 'id', 'Computer','ECU','State','User','IP' set['columns']= (set_columns) set.column("#0", width=0, stretch=NO) set.column("id",anchor=CENTER, width=50) set.column("Computer",anchor=CENTER, width=130) set.column("ECU",anchor=CENTER, width=160) set.column("State",anchor=CENTER, width=160) set.column("User",anchor=CENTER, width=350) set.column("IP",anchor=CENTER, width= 140) set.heading("#0",text="",anchor=CENTER) set.heading("id",text="ID",anchor=CENTER) set.heading("Computer",text="Computer",anchor=CENTER) set.heading("ECU",text="ECU",anchor=CENTER) set.heading("State",text="State",anchor=CENTER) set.heading("User",text="User",anchor=CENTER) set.heading("IP",text="IP",anchor=CENTER) def set_data_fn(): global set_data,Name_Lab set_data = [] set_data.clear() Group_Data() for i in range(len(data)): temp = i,Name_Lab[i],data[i][0],data[i][1],data[i][2],data[i][3] set_data.append(temp) return set_data set_data = set_data_fn() for i in range(len(set_data)): set.insert(parent='',index='end',iid=i,text='',values=(set_data[i])) def Refresh(): global set_data selected = [] set_data = set_data_fn() for i in range(8): selected.append(i) #save new data for i in range(len(set_data)): set.item(int(selected[i]),text="",values=(set_data[i])) messagebox.showinfo(title="Information", message=" You have refreshed !!!") def access(): selected=set.focus() path_ac_data() if selected != "": os.popen(Path_AC[int(selected)]) time.sleep(0.5) 111 pyautogui.press('left') pyautogui.press('enter') else: messagebox.showinfo(title="Information", message=" You need to choose Lab !!!") def Messages(): global Name_Lab selected=set.focus() Path_Message() if selected != "": App_MS(Path_MS[int(selected)],Name_Lab[int(selected)]) else: messagebox.showinfo(title="Information", message=" You need to choose Lab !!!") def History(): global Name_Lab selected=set.focus() Path_history() if selected != "": Fn_history(Path_HS[int(selected)]) else: messagebox.showinfo(title="Information", message=" You need to choose Lab !!!") tk.Button(text='Access',command = access,bg='#739aae').place(x=20,y=200) tk.Button(text='Refresh',command = Refresh,bg='#87a9ba').place(x=100,y=200) tk.Button(text='Message',command = Messages,bg='#9bb7c5').place(x=180,y=200) tk.Button(text='History',command = History,bg='#afc5d1').place(x=260,y=200) def Close(): win_GUI_1.destroy() def lock_lab(): win2 = Tk() win2.geometry("240x200") name_id = tk.Text(win2, height = 1, width = 20) # lab_id = tk.Text(win2, height = 1, width = 20) options = [ "HC_C-000RA", "HC_UT30025C", "HC_UT31212C", "HC_UT31215C", "HC_UT31218C", "HC_UT40485C", "HC_UT40592C", "HC_UT40608C" ] # datatype of menu text clicked = StringVar(win2) # initial menu text clicked.set(options[0]) lab_id = OptionMenu(win2, clicked, *options) name_id.pack() lab_id.pack() 112 tk.Button(win2, text='Check ID', command=lambda: checkid()).pack(side=BOTTOM) def checkid(): inp = name_id.get(1.0, "end-1c") lab_inp = clicked.get() date = datetime.now() current_time = datetime.timestamp(date) current_date = datetime.fromtimestamp(current_time) if (inp == text): with open(status_path,'a') as f: fieldnames = ['0','Manv','Lab','Status','Time'] writer = csv.DictWriter(f,fieldnames=fieldnames) writer.writerow({'0':'0','Manv': text, 'Lab': lab_inp, 'Status': 'Locked', 'Time': current_date}) status.insert('','end',values=(text,lab_inp,'Locked',current_date) ) messagebox.showinfo(title="Information", message="Matched ID") win2.destroy() else: messagebox.showinfo(title="Information", message="Please check ID again") win2.destroy() train_root = Path('dataset/') face_detect = cv2.CascadeClassifier() face_detect.load('model_run/test.xml') # model = tf.keras.models.load_model('model_run/saved_model.pbtxt') model = models.load_model('mymodel_4') cam = cv2.VideoCapture(0) while True: OK, frame = cam.read() # gray = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB) faces = face_detect.detectMultiScale(frame, 1.2, 5) for (x,y,w,h) in faces: # img1 = image.load_img('dataset/1 (100).jpg',target_size=(300,300)) roi = cv2.resize(frame[y: y+h, x: x+w], (300,300)) X = image.img_to_array(roi.reshape(180,180,3)) X = np.expand_dims(X,axis=0) images = np.vstack([X]) val = Model.predict(model,images) cv2.rectangle(frame, (x,y), (x+w, y+h), (255,100,200), 3) # result = np.argmax(Model.predict(model,roi.reshape((1,300,300,3)))) if val[0,0] == 1: text = 'nih7hc' 113 elif val[0,1] == 1: text ='nky7hc' else: text='unknown' cv2.putText(frame, text, (x+50,y-20), cv2.FONT_HERSHEY_DUPLEX, 0.6, (20,255,255),2) # tk.Label(win_GUI_1, text=text + ' is using').place(x=420,y=200) cv2.imshow('FaceDetect',frame) # cv2.namedWindow('FaceDetect', cv2.WINDOW_KEEPRATIO) if cv2.waitKey(1) & 0xFF == ord('q'): break cam.release() cv2.destroyAllWindows() win2.mainloop() def Unlock(): win2 = Tk() win2.geometry("240x200") name_id = tk.Text(win2, height = 1, width = 20) # lab_id = tk.Text(win2, height = 1, width = 20) options = [ "HC_C-000RA", "HC_UT30025C", "HC_UT31212C", "HC_UT31215C", "HC_UT31218C", "HC_UT40485C", "HC_UT40592C", "HC_UT40608C" ] # datatype of menu text clicked = StringVar(win2) # initial menu text clicked.set(options[0]) lab_id = OptionMenu(win2, clicked, *options) name_id.pack() lab_id.pack() tk.Button(win2, text='Check ID', command=lambda: checkid()).pack(side=BOTTOM) def checkid(): inp = name_id.get(1.0, "end-1c") lab_inp = clicked.get() date = datetime.now() current_time = datetime.timestamp(date) current_date = datetime.fromtimestamp(current_time) if (inp == text): with open(status_path,'a') as f: fieldnames = ['0','Manv','Lab','Status','Time'] writer = csv.DictWriter(f,fieldnames=fieldnames) writer.writerow({'0':'0','Manv': text, 'Lab': lab_inp, 'Status': 'Unocked', 'Time': current_date}) 114 status.insert('','end',values=(text,lab_inp,'Unlocked',current_dat e)) messagebox.showinfo(title="Information", message="Matched ID") win2.destroy() else: messagebox.showinfo(title="Information", message="Please check ID again") win2.destroy() train_root = Path('dataset/') face_detect = cv2.CascadeClassifier() face_detect.load('model_run/test.xml') # model = tf.keras.models.load_model('model_run/saved_model.pbtxt') model = models.load_model('mymodel') cam = cv2.VideoCapture(0) while True: OK, frame = cam.read() gray = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB) faces = face_detect.detectMultiScale(frame, 1.2, 5) for (x,y,w,h) in faces: # img1 = image.load_img('dataset/1 (100).jpg',target_size=(300,300)) roi = cv2.resize(frame[y: y+h, x: x+w], (300,300)) X = image.img_to_array(roi.reshape(300,300,3)) X = np.expand_dims(X,axis=0) images = np.vstack([X]) val = Model.predict(model,images) cv2.rectangle(frame, (x,y), (x+w, y+h), (255,100,200), 3) # result = np.argmax(Model.predict(model,roi.reshape((1,300,300,3)))) if val[0,0] == 1: text = 'nih7hc' elif val[0,1] == 1: text ='nky7hc' else: text='unknown' cv2.putText(frame, text, (x+50,y-20), cv2.FONT_HERSHEY_DUPLEX, 0.6, (20,255,255),2) # tk.Label(win_GUI_1, text=text + ' is using').place(x=420,y=200) cv2.imshow('',frame) if cv2.waitKey(1) & 0xFF == ord('q'): break cam.release() 115 cv2.destroyAllWindows() win2.mainloop() tk.Button(win_GUI_1, text="Lock", command=lambda: lock_lab(),bg='cyan').place(x=340,y=160) tk.Button(win_GUI_1, text="Unlock", command=lambda: Unlock(),bg='cyan').place(x=340,y=200) # Button for closing tk.Button(win_GUI_1, text="Cancel", command=Close,bg='orange').pack(side=BOTTOM) win_GUI_1.mainloop() # # def App_MS(Path,Name_Lab): global name ms = Tk() ms.geometry("350x300") ms.title(" Message to "+ Name_Lab) def Take_input(): global name Mess = '' Mess = inputtxt.get("1.0", "end-1c") Content = "From "+ name + Mess Write_txt(Path,Content) inputtxt.delete("1.0","end") def Quit(): ms.quit() inputtxt = Text(ms, height = 10,width = 25,bg = "light yellow") Button(ms,text ="Send",command = lambda:Take_input()).place(x=120, y=210) Button(ms,text ="Quit",command = lambda:Quit()).place(x=170, y=210) inputtxt.pack(pady = 20) ms.resizable(FALSE,FALSE) mainloop() ms.destroy() try: threadlist = [] a = threadlist.append(Thread(target=dir_Name_Lab())) b = threadlist.append(Thread(target=Group_Data())) d = threadlist.append(Thread(target=app)) c = threadlist.append(Thread(target=check_Name(path_name))) e = threadlist.append(Thread(target=write_data(path_database,path_name))) for t in threadlist: t.start() for t in threadlist: t.join() except Exception as bug: Write_txt(path_sys,bug) 116 APPENDIX 2: Machine vision (CNN and Cascade) model training: from google.colab import drive drive.mount('/content/gdrive') !ls gdrive/MyDrive/test !unzip gdrive/MyDrive/test/data5.zip import numpy as np import os from PIL import Image import tensorflow as tf from tensorflow.python import keras from tensorflow.keras import layers import numpy as np from tensorflow.keras.preprocessing.image import img_to_array from tensorflow.python.keras import models from tensorflow.python.keras import Model from tensorflow.keras.layers.experimental.preprocessing import Normalization, Resizing, RandomFlip, RandomRotation, RandomZoom from tensorflow.python.keras.layers import * from tensorflow.python.keras.models import * import glob import cv2 from pathlib import Path import matplotlib.pyplot as plt from tensorflow.keras.preprocessing.image import ImageDataGenerator train_root = Path('data3/') batch_size = 32 img_height = 180 img_width = 180 dir1 = train_root / 'VH' dir2 = train_root / 'QK' 117 dir3 = train_root / 'Khanh' tf.io.gfile.mkdir(dir1) tf.io.gfile.mkdir(dir2) tf.io.gfile.mkdir(dir3) for item in train_root.glob("*.jpg"): cur_dir = train_root / item.name if item.name[0] == '1': des_dir = dir1 / item.name elif item.name[0] == '2': des_dir = dir2 / item.name elif item.name[0] == '3': des_dir = dir3 / item.name tf.io.gfile.copy(cur_dir, des_dir, overwrite=True) tf.io.gfile.remove(cur_dir) !ls data3/VH | wc -l train_ds = tf.keras.utils.image_dataset_from_directory( train_root, validation_split=0.15, subset="training", seed=123, image_size=(img_height, img_width), batch_size = batch_size ) val_ds = tf.keras.utils.image_dataset_from_directory( train_root, validation_split=0.15, subset="validation", seed=123, image_size=(img_height, img_width), batch_size = batch_size ) class_names = train_ds.class_names print(class_names) class_names2 = val_ds.class_names print(class_names2) #create dataset from list filelist_ds = tf.data.Dataset.list_files(str(train_root/'*/*')) for file in filelist_ds.take(9): print(file) 118 def get_label(file_path): parts = tf.strings.split(file_path, '/') # print(file_path) if parts[1] == "VH": labels = [1,0,0,0] elif parts[1] == "QK": labels = [0,1,0,0] elif parts[1] == "Khanh": labels = [0,0,1,0] else: labels = [0,0,0,1] return tf.convert_to_tensor(labels) # for file in filelist_ds.take(9): # # print('filename:', file.numpy().decode('utf-8')) print('label:', get_label(file).numpy()) def preprocess(file_path): #read file img = tf.io.read_file(file_path) img = tf.image.decode_jpeg(img, channels=3) #transform #convert from uint8 to float32 and normalize values to [0,1] img = tf.image.convert_image_dtype(img, tf.float32) #resize image img = tf.image.resize(img, [img_width,img_height]) #get image label tag = get_label(file_path) #return return img,tag train_ds train_ds train_ds train_ds = = = = filelist_ds.shuffle(300) train_ds.map(preprocess) train_ds.batch(batch_size) train_ds.prefetch(1) # # #create model using Functional API input = tf.keras.Input(shape=(180,180,3)) # x x # x #Augmentation = layers.experimental.preprocessing.RandomRotation(0.1)(input) = layers.experimental.preprocessing.RandomFlip('horizontal')(x) x = layers.experimental.preprocessing.RandomFlip('vertical')(x) = layers.experimental.preprocessing.RandomZoom(0.3)(x) 119 x = layers.experimental.preprocessing.RandomHeight(0.2)(x) x = layers.Conv2D(10,5, strides=(1,1), padding='valid', activation='relu')(input) # filters 10, kernel size 5, padding valid, activation relu x = layers.MaxPool2D(pool_size=(2,2), strides=2)(x) # pool size (2,2) stride = x = layers.Conv2D(20,5, strides=(1,1), padding='valid', activation='relu')(x) # filters 20, kernel size 5, padding valid, activation relu x = layers.MaxPool2D(pool_size=(2,2), strides=2)(x) # pool size (2,2) stride = x = layers.Dropout(0.25)(x) # bo 25% x = layers.Flatten()(x) x = layers.Dense(100, activation='relu')(x) output = layers.Dense(4, activation='softmax')(x) model = tf.keras.Model(input,output) # #compile model model.compile( optimizer='Adam', loss='categorical_crossentropy', metrics=["accuracy"] ) tf.keras.utils.plot_model(model, 'model.png', show_shapes=True) # #train model train1 = model.fit(train_ds, epochs=10) # # # model.save('/gdrive/MyDrive/test/face_functional') # # # summarize history for accuracy plt.plot(train1.history['accuracy']) # plt.plot(train1.history['val_accuracy']) plt.title('model accuracy') plt.ylabel('accuracy') plt.xlabel('epoch') plt.legend(['Train', 'Validation'], loc='upper left') plt.show() # summarize history for loss plt.plot(train1.history['loss']) # plt.plot(train1.history['val_loss']) 120 plt.title('model loss') plt.ylabel('loss') plt.xlabel('epoch') plt.legend(['Train', 'Validation'], loc='upper left') plt.show() for file in filelist_ds.take(9): print('filename:', file.numpy().decode('utf-8')) print('label:', get_label(file).numpy()) scores = model.evaluate(val_ds) print(scores) test1= model.predict(val_ds) print(test1[2]) tf.keras.utils.plot_model(model, 'model.png', show_shapes=True) model.summary() model.save('gdrive/MyDrive/test/mymodel_5') from google.colab.patches import cv2_imshow from tensorflow.keras.preprocessing import image face_detect = cv2.CascadeClassifier() face_detect.load('/content/gdrive/MyDrive/test/test.xml') img_path = '*.jpg' frame = cv2.imread(img_path) faces = face_detect.detectMultiScale(frame, 1.1, 3) for (x,y,w,h) in faces: img1 = image.load_img(img_path,target_size=(180,180)) X = image.img_to_array(img1) X = np.expand_dims(X,axis=0) images = np.vstack([X]) val = model.predict(images) print(val) if val[0,0] == 1: text = 'VH' elif val[0,1] == 1: text ='QK' elif val[0,2] == 1: text ='Khanh' else: text ='none' cv2.rectangle(frame,(x,y),(x+w,y+h),(0,255,0),3) cv2.putText(frame, text, (x,y+10), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (20,255,255),2) # print(text) cv2_imshow(frame) 121 S K L 0