So sánh nhận diện khuôn mặt sử dụng giải thuật k gần nhất với mạng nơ ron tự cấu trúc

VIETNAM NATIONAL UNIVERSITY OF HO CHI MINH CITY HO CHI MINH UNIVERSITY OF TECHNOLOGY -o0o - NGUYỄN ĐỨC MINH FACE RECOGNITION PERFORMANCE COMPARISON BETWEEN K-NEAREST NEIGHBORS ALGORITHM AND SELF-ORGANIZED MAP SO SÁNH NHẬN DIỆN KHUÔN MẶT SỬ DỤNG GIẢI THUẬT K GẦN NHẤT VỚI MẠNG NƠ-RON TỰ CẤU TRÚC Department: Control Engineering & Automation Department ID: 60520216 MASTER THESIS HO CHI MINH CITY, September 2020 CƠNG TRÌNH ĐƯỢC HOÀN THÀNH TẠI TRƯỜNG ĐẠI HỌC BÁCH KHOA – ĐHQG – HCM Cán hướng dẫn khoa học: GS.TS Hồ Phạm Huy Ánh Cán chấm nhận xét 1: PGS.TS Huỳnh Thái Hoàng Cán chấm nhận xét 2: PGS.TS Nguyễn Tấn Lũy Luận văn thạc sĩ bảo vệ Trường Đại học Bách Khoa, ĐHQG TP HCM Ngày 04 tháng 09 năm 2020 Thành phần Hội đồng đánh giá luận văn thạc sĩ gồm: Chủ tịch: PGS.TS Nguyễn Thanh Phương Thư kí: TS Trần Ngọc Huy Phản biện 1: PGS.TS Huỳnh Thái Hoàng Phản biện 2: PGS.TS Nguyễn Tấn Lũy Ủy viên: TS Nguyễn Hoàng Giáp Xác nhận Chủ tịch Hội đồng đánh giá LV Trưởng Khoa quản lý chuyên ngành sau luận văn sửa chữa (nếu có) CHỦ TỊCH HỘI ĐỒNG PGS.TS NGUYỄN THANH PHƯƠNG TRƯỞNG KHOA ĐIỆN-ĐIỆN TỬ TS HUỲNH PHÚ MINH CƯỜNG ĐẠI HỌC QUỐC GIA TP.HCM TRƯỜNG ĐẠI HỌC BÁCH KHOA CỘNG HÒA XÃ HỘI CHỦ NGHĨA VIỆT NAM Độc lập - Tự - Hạnh phúc NHIỆM VỤ LUẬN VĂN THẠC SĨ Họ tên học viên: NGUYỄN ĐỨC MINH MSHV: 1770217 Ngày, tháng, năm sinh: 01/11/1994 Nơi sinh: TP.HCM Chuyên ngành: Kỹ thuật Điều Khiển Tự Động Hóa Mã số : 60520216 I TÊN ĐỀ TÀI: So Sánh Nhận Diện Khuôn Mặt Sử Dụng Giải Thuật K Gần Nhất với Mạng Nơ-ron Tự Cấu Trúc Face Recognition Performance Comparison between K-Nearest Neighbors Algorithm and Self-Organized Map II NHIỆM VỤ VÀ NỘI DUNG: Xây dựng hai hệ thống nhận diện khuôn mặt khác phương pháp Giải thuật K gần Mạng Nơ-ron tự cấu trúc Từ tiến hành so sánh mặt lý thuyết ứng dụng, độ xác việc nhận diện khuôn mặt, ưu khuyết điểm hai phương pháp III NGÀY GIAO NHIỆM VỤ : 19/08/2019 IV NGÀY HOÀN THÀNH NHIỆM VỤ: 03/08/2020 V CÁN BỘ HƯỚNG DẪN (Ghi rõ học hàm, học vị, họ, tên): GS.TS Hồ Phạm Huy Ánh Tp HCM, ngày 23 tháng 09 năm 2020 CÁN BỘ HƯỚNG DẪN (Họ tên chữ ký) GS.TS Hồ Phạm Huy Ánh CHỦ NHIỆM BỘ MÔN ĐÀO TẠO (Họ tên chữ ký) TS Nguyễn Vĩnh Hảo TRƯỞNG KHOA….……… (Họ tên chữ ký) TS Huỳnh Phú Minh Cường ACKNOWLEDGMENTS First and foremost, I would like to express our sincere gratitude and respect to my senior project supervisor, Assoc Prof Dr Ho Pham Huy Anh for his guidance, advice, supervision and patience His enthusiastic support and encouragement gives me motivation to research in this field Also, I would like to thank my lecturers at Ho Chi Minh City University of Technology (HCMUT) who imparted valuable knowledge as well as shared their experiences and advice for me in the past years These things are very meaningful for my further studying and following jobs in the future Besides, I also want to gives a profound thanks to my parents for their understanding, encouragement and support during the study period at HCMUT It always motivate me to strive on my learning path Last but not least, I would like to thank all my friends who also has an important role in my studying with their supports and encouragement throughout our time studying together in the Ho Chi Minh City University of Technology Ho Chi Minh City, August, 2020 Student NGUYEN DUC MINH i ABSTRACT In recent years, automatic subject and object recognition system is not only a new trend but also a challenging technology that attracts lots of attention due to its various applications in different fields Face recognition is one of those functions Currently there are many techniques that can provide a robust solution to various situations of this technology, which can adapt through environmental conditions and factors that affect the recognition ability Nowadays, automating the face recognition process is a very practical task due to its wide range of applications including surveillance, human to machine interaction, security system, video compression, video indexing of large databases and a whole of other multimedia applications Therefore, many designs and developments of a face recognition system that can apply for at least one of the possibility above can be found anywhere, from mobile phone camera to security surveillance Yet performing a detailed comparison between methods still haven’t come to interest of researchers, as until now not many articles mention about this This problem might restrain newcomers of face recognition field to get an overview on current advantages and disadvantages of technologies, also experienced researchers might be affected by too focusing in one field without noticed about different methods Therefore, a performance comparison between two applicable face recognition method is our main goal In this document, we describe the work completed for our senior project and provide the design of an efficient high-speed face recognition system As a further step from my university thesis [1], this project includes a research about some of the existing methods for face recognition, develop two algorithms for two face recognition systems that formulates both image-based and feature-based approach, using the 29 levels Residual Neural Network (ResNet-29) for encoding facial features in a normal picture, following by k-Nearest Neighbors (KNN) Algorithm for training and recognition The method is implement in Python language and thus applicable for various operating system, with a user friendly GUI and tested many times in different working conditions to prove that the method has good success rate in real life applications From that we perform a performance comparison between this new method and the result of my previous work, which was using Illumination Normalization (IN), 2D-DCT and Self-Organized Map (SOM) written in MATLAB environment Due to different algorithm and environment, we will focus on final result of the output application ii T Ó M T Ắ T L UẬ N VĂ N Những năm gần đây, ứng dụng nhận dạng vật thể trở nên thông dụng thu hút nhiều ý từ nhà nghiên cứu, nhờ vào tiềm rộng lớn mà đem lại Nhận diện khuôn mặt số ứng dụng Cho đến tại, có khơng phương pháp nhận diện khuôn mặt cho kết tốt, điều kiện hình ảnh ánh sáng khắc nghiệt - yếu tố ảnh hưởng xấu tới kết nhận dạng phương pháp truyền thống Ngày nay, ứng dụng nhận diện khuôn mặt trở nên quen thuộc Trong kể đến ứng dụng mở khóa khn mặt điện thoại di động, hệ thống an ninh, tách lọc thông tin nhận dạng từ video, nhiều ứng dụng khác Để đạt khả ứng dụng lớn vậy, nhiều phương pháp nhận diện khuôn mặt khác đề xuất Tuy nhiên việc so sánh thuật toán kết phương pháp lại nhận ý nhà nghiên cứu Hiện tại, khơng có nhiều báo khoa học đề cập đến việc Hệ người bắt đầu tìm hiểu ứng dụng nhận diện khn mặt khó tìm ưu khut điểm phương pháp, cịn người có kinh nghiệm lại tập trung vào vài phương pháp mà họ biết bỏ qua tiềm thuật tốn nhận diện khác Vì lý đó, mục tiêu luận văn đưa so sánh thuật tốn độ xác hai phương pháp nhận diện khuôn mặt thông dụng Trong viết này, mô tả việc thực để xây dựng so sánh hai hệ thống nhận diện khuôn mặt khác Được phát triển từ luận văn tốt nghiệp đại học [1], luận văn thạc sĩ bao gồm cơng việc: tìm hiểu nghiên cứu phương pháp nhận diện khuôn mặt thông dụng nay, xây dựng hai hệ thống nhận diện khuôn mặt dựa vào hai thuật toán chọn lọc, ứng dụng nhận diện dựa vào hình ảnh dựa vào đặc trưng Hệ thống thứ sử dụng mạng nơ-ron ResNet 29 lớp (ResNet-29) để nén đặc trưng hình ảnh, kết hợp với thuật tốn K gần (KNN) cho q trình huấn luyện nhận dạng Ngơn ngữ lập trình Python chọn để ứng dụng cho nhiều hệ điều hành khác nhau, với GUI dễ sử dụng kiểm thử nhiều lần nhiều điều kiện ánh sáng khác nhằm chứng minh khả ứng dụng hệ thống Sau đó, tơi tiến hành so sánh với hệ thống thứ hai lấy từ luận văn đại học tơi, sử dụng xử lý hình ảnh thuật tốn IN 2D-DCT, với nhận diện mạng nơ-ron tự cấu trúc (SOM) mơi trường MATLAB Vì thuật tốn ngơn ngữ khác nhau, ta tập trung vào so sánh kết đầu hai phương pháp ứng dụng thực tiễn iii D E C L A R AT I O N I declare that this thesis is an original report of my research, has been written by me and has not been submitted anywhere else The experimental work is almost entirely my own work The collaborative contributions have been indicated clearly and acknowledged Due references have been provided on all supporting literatures and resources I declare that this thesis was composed by myself, that the work contained herein is my own except where explicitly stated otherwise in the text, and that this work has not been submitted for any other publication or professional qualification Ho Chi Minh City, August, 2020 Student NGUYEN DUC MINH iii CONTENTS I N T RO D U C T I O N O V E RV I E W 1.1 Pattern Recognition 1.2 Face Recognition I A B O U T T H I S P RO J E C T 2.1 Project Overview 2.2 Problem Statement 2.3 Project Objective 2.4 Project Methodology 2.4.1 Study and Research 2.4.2 Design and Implementation 2.4.3 Performance Comparison 3 7 R E L AT E D T H E O RY M AC H I N E L E A R N I N G A N D A RT I F I C I A L RAL NETWORK 3.1 Introduction 3.2 Historical Background 3.2.1 Origins of Machine Learning 3.2.2 Origins of Neural Networks 3.3 Machine Learning Algorithms 3.3.1 An overview 3.3.2 Machine Learning Models II 10 10 12 14 14 15 16 16 18 19 21 21 22 3.4.1 KNN Algorithms 22 3.3.2.1 3.3.2.2 3.3.2.3 3.3.2.4 3.3.2.5 3.3.2.6 3.4 NEU- k-Nearest Artificial Neural Networks Decision Trees Linear Regression Support Vector Machine k-Nearest Neighbors Bayesian Networks Neighbors iv CONTENTS 3.4.1.1 Determine value of K 3.4.1.2 Distance calculation 3.4.1.3 Output class measurement 3.4.2 Application of KNN 3.5 Neural Network Algorithms 3.5.1 3.5.2 3.5.3 3.5.4 3.5.5 3.5.6 23 23 23 24 24 Biological and Artificial Neurons 25 3.5.1.1 Biological Neurons 3.5.1.2 Artificial Neurons 3.5.1.2.1 Firing Rules 3.5.1.2.2 Simple Artificial Neuron 3.5.1.2.3 Complicated Artificial Neuron Architecture of Neural Networks 3.5.2.1 Feed-forward Networks 3.5.2.1.1 Single-Layer Perceptron 3.5.2.1.2 Multi-layer Perceptron 3.5.2.1.3 ADALINE 3.5.2.1.4 Radial Basis Function Network 3.5.2.1.5 Convolutional Neural Network (CNN) 3.5.2.1.6 Residual Neural Network (ResNet) 3.5.2.1.7 Kohonen Self-Organizing Map (SOM) 3.5.2.2 Recurrent Networks 3.5.2.2.1 Simple Recurrent Network 3.5.2.2.2 Hopfield Network 3.5.2.2.3 Echo State Network 3.5.2.2.4 Long Short-term Memory Network 3.5.2.3 Stochastic Neural Networks 3.5.2.4 Botlzmann Machine 3.5.2.5 Modular Neural Networks Neural Network Training 3.5.3.1 Definition of Training 3.5.3.2 Selection of Cost Function 3.5.3.3 Memorization of Inputs 3.5.3.3.1 Associative Mapping 3.5.3.3.1.1 Auto-association 3.5.3.3.1.2 Hetero-association 3.5.3.3.2 Regularity Detection 3.5.3.4 Determination of Weight Learning Paradigms 3.5.4.1 Supervised Learning 3.5.4.2 Unsupervised Learning 3.5.4.3 Reinforcement Learning 3.5.4.4 Training Function Learning Algorithm Employing Artificial Neural Networks 25 26 26 27 28 28 29 30 31 32 33 34 35 36 37 37 38 38 38 39 39 39 39 39 40 41 41 41 42 42 42 42 43 43 43 44 44 45 v CONTENTS 3.5.6.1 Selection of Model 3.5.6.2 Selection of Learning Algorithm 3.5.6.3 Robustness 3.5.7 Applications of Artificial Neural Networks 3.6 Kohonen Self-Organizing Map 45 45 45 45 46 3.6.1 SOM Network Architecture 3.6.2 Training Process of SOM 3.6.2.1 The Competitive Process 3.6.2.2 The Cooperative Process 3.6.2.3 The Adaptive Process 3.6.2.4 Ordering and Convergence 3.6.3 SOM Applications 3.7 Conclusion 46 47 47 48 48 49 49 50 3.7.1 Different between ML and ANN 3.7.2 Applying ML and ANN in this project 50 51 52 I M AG E C O M P R E S S I O N 4.1 Discrete Cosine Transform 52 4.1.1 Introduction 4.1.2 Properties of DCT 4.1.3 Definition of DCT 4.1.3.1 Overview 4.1.3.2 One-dimensional type-2 DCT 4.1.3.3 Two-Dimensional Type-2 DCT 4.1.3.4 2D-DCT in Image compression 4.1.3.4.1 2D-DCT basis functions 4.1.3.4.2 DCT coefficients matrix 4.1.4 DCT Image Compression in JPEG format 4.1.4.1 Overview 4.1.4.2 Example with detailed process 4.1.5 Other Applications of DCT 4.1.6 Conclusion 4.2 Illumination Normalization 52 53 54 54 54 56 57 57 58 60 60 61 63 63 63 4.2.1 An Engineer Approach 4.2.2 IN Techniques 4.2.2.1 Introduction 4.2.2.2 Finding least error IN technique 4.2.2.3 Applying and testing AS with DCT in compression 4.2.3 Conclusion 4.2.3.1 Research Analysis 4.2.3.2 Disadvantages of the Proposed Method 4.3 Residual Network for Image Data Encoding 63 66 66 67 68 73 73 74 74 image vi 11 P E R F O R M A N C E C O M PA R I S O N In this chapter, we will finalize the two completed systems and bring it to the competition about facial recognition performance, as well as each system’s advantages Motivation and objective of them are the same, but the design of each system are completely different Despite they are based on the same metric to perform learning: Euclidean distance, but they might excel in different situations Therefore, we created a table 11 to conclude their general properties and from it we can have an overview about the two systems Training time and accuracy were based on our training sets from previous section, which has 30 training data featuring persons and positive test data From the table, we can see that SOM system perform training much faster than KNN system The reason behind it is because of pre-processing algorithm The image-based process requires impressively less resources and computation time, compare to the featurebased approach deep neural network ResNet-29 While 2D-DCT and Anisotropic diffusion only applies a few mathematical algorithms to each pixel’s value for one picture, ResNet perform convolution computation for many layers with × and × kernels, also just for one picture As a result, despite KNN algorithm is much simpler than SOM in term of classification, the main problem of the whole KNN system is it requires more bruce-force calculations for preparation It makes the total training time for KNN is almost unreasonable for real applications Fortunately, the designed KNN system boasts significant higher accuracy in final prediction Therefore both systems are worth consideration while searching for an efficient high speed face recognition system 124 P E R F O R M A N C E C O M PA R I S O N Item SOM-based System Feature Extraction Methodology Image-based Frequency Extraction Classification Methodology Pre-Processing Input Type Designed Image Channel Pre-Processing Methodology Pre-Processing Output Type Feature Classification Methodology Whole System Training Time (s) Whole System Prediction Time (s) Expected Prediction Accuracy (%) Feed-Forward Neural Network Squared JPEG, optimal 512 × 512 Squared JPEG, required 512 × 512 Grayscale RGB 2D-DCT and Anisotropic diffusion Pre-trained ResNet-29 64 × 1, integer to 255 128 × 1, float −1 to Self-Organized Map K-Nearest Neighbors ∼ 25 ∼ 65 ∼ 0.7 ∼ 2.0 95.6 97.4 KNN-based System Feature-based Feed-Forward Neural Network Machine Learning Table 11: General system deviation between the designed SOM and KNN 125 Part V R E C O M M E N D AT I O N S A N D CONCLUSION 12 CONCLUSION 12.1 S I G N I F I C A N C E O F T H E P RO J E C T After performing the simulations in MATLAB and Python, we can see that our project could describe two successful designs of efficient high-speed face recognition systems, especially in surveillance application with large amount of individuals’ data Based on the results we got, both systems met all main objectives that it can recognize correctly nearly 95% of the persons inside the database For the first system, thanks to using 2D-DCT and SOM Neural Network techniques, our face database can be saved after discard redundant data to make it become very light (about 200KB for a database with 30 images of persons, from an originally more than 10MB data) and easier for storage as well as transfer with generally leads to fast training time Furthermore, its excellent working speed is consider to be another noticeable plus: in particular, it can gives you the result within less than second In addition, we also provide the additional image preprocess method ASDCT to help removing unwanted illumination features from the input images so that it can further improves the correct recognition rate in some cases For the second system, powered by a more advanced feature extraction techniques named ResNet, combined with a fast and simple data classifier K-Nearest Neighbors The processed image data can also be much more compact, not only comparable to 2D-DCT, but also make the surveillance application become feasible Despite the encoding method requires more resources as well as working speed is a bit slower than the first system, the average accuracy is significantly higher and it can compensate to the disadvantages By the time we finished our project objective, we have been familiarizing ourselves with MATLAB’s and Python’s commands and supporting tools like ANN Toolbox, Image Processing, Image Acquisition Toolbox, Open library Scikit-Learn, Face Recognition Toolbox, MATLAB and Python GUI We also acquired a lot of knowledge about the Machine Learning, ANN and its potential capability in modern industry development as well as high level technology applications 127 12.2 R E M A I N I N G L I M I TAT I O N S Hence, in brief, the 2D-DCT, ResNet-29, SOM neural network and K-Nearest Neighbors are the core algorithms for the design and implementation of our efficient high-speed face recognition systems Simulated using MATLAB and Python, it has proven to be highly accurate for recognizing a variety of face images with different facial expressions on plain backgrounds under slightly changes in lighting conditions, as well as open the door to other platform implementation, thanks to the OS-friendly language Python 12.2 R E M A I N I N G L I M I TAT I O N S Despite appearing to be very efficient in face recognition, both systems still remains some weaknesses Nevertheless, most of these problems are not from the system itself but caused by the image quality for input and database (e.g too complex background, different light exposure in image using for database, blurred image, inhomogeneous face position and more) The most serious issue about our systems is: both of them cannot precisely point out an individual that is not inside the database The system sometimes tends to result in someone inside the database instead of confirms that it can’t recognize the input image and return "unknown" value However, consider this system is used for surveillance purpose, where the authority possesses all of their citizens’ personal identity information included their face image, the weakness mentioned above is acceptable Last but not least, is the limitation in our resources and project scale Currently, our tests are only focus on a limited interval when it comes to the parameter optimization and data scale, because the operating system we’re using is just a personal computer The processor of our test hardware is not specialized for this particular task Therefore, it may affect the performance During our test, we recorded some anomaly in the statistical test If we can possess a more efficient system, we’ll have more chance to apply testing with larger scale and could improve the peak performance for our programs 128 13 R E C O M M E N D AT I O N S About SOM system, upon extensive study and research, recommendations for improvement and enhancement of the face recognition system program are concluded as follows: • Applying different image processing methods besides DCT, such method that is superior and has improved algorithm for image compression which requires less processing time than DCT but still maintain good compression capability • Further SOM neural network efficiency testing based on the following factors: – Selecting the optimal number of DCT coefficients to use for face image compression, which will lead to less DCT processing time and increase the program execution time In this project, currently we are using 61 DCT coefficients for image compression – Selecting other optimal parameters for our neural network rather than using the default one Although most of the default parameters are recommended by universal experiments from MATLAB main website and its community, it seems that for a particular task, these parameters should be tuned so as to find out the most suitable values – Extending our range of testing to find better solutions This can be done by expanding the system scale, database scale and parameters scale • Complete a fully developed GUI program with the ability to be converted into a stand-alone application (.exe) so that it can run freely on any machine without having MATLAB installed as a pre-requisite About KNN system, expected to be a more advanced method compare to SOM system and escaped from an OS-limited MATLAB language, still have many open points for improvement which are listed below: • Optimize parameters in deep neural network for feature extraction step in order to perform faster and also maintain the final result performance 129 R E C O M M E N D AT I O N S • Further KNN efficiency testing based on the following factors: – Can be applied as a regression learning method, to optimize and adapt its parameters (number of k, bounded distance, etc ) based on different input conditions, rather than using a default value – Use a more advanced nearest neighbors searching method to boost the output speed Some of the method can be mentioned are KDTree or BallTree – Extending our range of testing to find better solutions This can be done by expanding the system scale, database scale and parameters scale • The Python based GUI were designed from pure code by Tkinter library, which is not user-friendly, especially for inexperienced developers We also tried to generate execution file (.exe) from python file (.py), but the result is still not work Therefore the ability to run the application freely on any machine without having Python installed is still an open point for improvement All the above mentioned factors will result to achieve the shortest amount of time for program execution while maintaining the maximum possible accuracy to produce an efficient high-speed face recognition system, such system that can be implemented for a practical task in real life application 130 Part VI REFERENCES BIBLIOGRAPHY [1] N D Minh and T L M Tam, “Face recognition system using discrete cosine transform and self-organized map,” 2016 [2] Smart Parenting for Smart Kids Jossey-Bass, 2011 [3] J Nagi, “Design of development an efficient high-speed face recognition system in a matlab/simulink environment,” 2007 [4] R Wang, “Transform coding and jpeg image compression,” 15 November 2007, accessed time June 2020 http://fourier.eng.hmc.edu/e101/lectures/Image_ Processing/node14.html [5] R C Narayanan Ramanathan and A K R Chowdhury, “Facial similarity across age, disguise, illumination and pose,” 2016 [6] Z Sufyanu, I Member, F S Mohamad, A A Yusuf, and M B Mamat, “Enhanced face recognition using discrete cosine transform,” Engineering Letters, 2016 [7] X He, Kaiming; Zhang, “Deep residual learning for image recognition,” Computer Vision and Pattern Recognition (cs.CV), 2015 [8] F Azirar, Facial Expression Recognition PhD thesis, School of Electronics and Physical Sciences, Departmentof Electronic Engineering, University of Surrey, 2004 [9] P Brimblecombe, Face Detection Using Neural Networks PhD thesis, School of Electronics and Physical Sciences, Departmentof Electronic Engineering, University of Surrey, 2005 [10] Data-driven Technology for Engineering System Health Management: Design Approach, Feature Construction, Fault Diagnosis, Prognostics, Fusion and Decisions Springer, 2016 [11] Y Chen, S Jiang, and A Abraham, Face Recognition using Discrete Cosine Transform and Hybrid Flexible Neural Tree PhD thesis, School of Information Science and Engineering, Jinan University, 2005 [12] V M P Jun-Cheng Chen, “Unconstrained face verification using deep cnn features,” 2019 132 BIBLIOGRAPHY [13] C Clabaugh, D Myszewski, and J Pang, “Neural networks,” 2000, accessed time May 2020 https://cs.stanford.edu/people/eroberts/courses/soco/ projects/neural-networks/index.html [14] A Mohri, Mehryar; Talwalkar, “Foundations of machine learning,” USA, Massachusetts: MIT Press, 2012 [15] Wikipedia, “Machine learning,” retrieved June 2020, accessed time June 2020 https: //en.wikipedia.org/wiki/Machine_learning [16] T H N Kumar, “Why machine learning models often fail to learn,” retrieved April 2017, accessed time May 2020 https://web.archive.org/web/20170320225010/ https://www.bloomberg.com/news/articles/2016-11-10/ why-machine-learning-models-often-fail-to-learn-quicktake-q-a [17] Wikipedia, “Decision tree learning,” retrieved June 2020, accessed time June 2020 https://en.wikipedia.org/wiki/Decision_tree_learning [18] O Rokach, L.; Maimon, “Top-down induction of decision trees classifiers-a survey,” IEEE Transactions on Systems, Man, and Cybernetics - Part C: Applications and Reviews 35 (4): p.476–487, 2005 [19] Wikipedia, “Linear regression,” retrieved June 2020, accessed time June 2020 https: //en.wikipedia.org/wiki/Linear_regression [20] L E Peterson, “K-nearest neighbor,” 2009, accessed time June 2020 http://www.scholarpedia.org/article/K-nearest_neighbor#:~:text= In%20an%20unpublished%20US%20Air,Fix%20%26%20Hodges%2C%201951) [21] S Sun, “An adaptive k-nearest neighbor algorithm,” 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery (FSKD 2010), 2010 [22] A.D.Dongare, R.R.Kharde, and A D.Kachare, “Introduction to artificial neural network,” International Journal of Engineering and Innovative Technology (IJEIT), 2012 [23] Wikipedia, “Neuron,” retrieved October 2015, accessed time June 2020 https: //en.wikipedia.org/wiki/Neuron [24] L Downs, E Santana, and A Berta-Antalics, “Before retinal cells die, they regenerate, penn vet blindness study finds,” 2016 [25] Wade and Nicholas, “Brain may grow new cells daily,” The New York Times, 1999 [26] C Stergiou and D Siganos, “Artificial neural network,” Retrieved 2015 133 BIBLIOGRAPHY [27] Face Recognition Based on Kernel Radial Basis Function Networks Computer Science Department, Beijing Institute of Technology, Beijing, China, 2004 [28] Wikipedia, “Feedforward neural network,” retrieved October 2015, accessed time May 2020 https://en.wikipedia.org/wiki/Feedforward_neural_network [29] Multilayer Feedforward Networks are Universal Approximators Pergamon Press plc, 1989 [30] B Widrow, “An adaptive adaline neuron using chemical "memistors",” tech rep., Office of Naval Research, 1960 [31] Impact of Climate Change on Natural Resource Management Springer Science and Business Media, 2010 [32] P L Smith, “An introduction to neural networks,” 1996, retrieved March 2016, accessed time June 2020 http://www.cs.stir.ac.uk/~lss/NNIntro/InvSlides html [33] Software Reliability Assessment with OR Applications Springer Science and Business Media, 2013 [34] Quantum Particle Swarm Optimization for Elman Recurrent Network Universiti Teknologi Malaysia, 2010 [35] Wikipedia, “Boltzmann machine,” retrieved March 2016, accessed time June 2020 https://en.wikipedia.org/wiki/Boltzmann_machine [36] The Nature of Code: Simulating Natural Systems with Processing Free Software Foundation, 2012 [37] M Niknafs, “Neural network optimization,” retrieved March 2016, accessed time May 2020 http://courses.mai.liu.se/FU/MAI0083/Report_Mina_Nikanfs.pdf [38] Wikipedia, “Artificial neural network,” retrieved March 2016, accessed time May 2020 https://en.wikipedia.org/wiki/Artificial_neural_network [39] Y Fan, “Face recognition system based on self-organizing maps and multilayer perception networks,” 2002 [40] Wikipedia, “Self-organizing map,” retrieved March 2016, accessed time May 2020 https://en.wikipedia.org/wiki/Self-organizing_map [41] F Schroff and D K from Google Inc., “Facenet: A unified embedding for face recognition and clustering,” Computer Vision Foundation, 2016 134 BIBLIOGRAPHY [42] N Ahmed, T Natarajan, and K Rao, “Discrete cosine transform,” IEEE Trans on Computers volume 23, no 1, pp 90-93, 1974 [43] S Vikas and J P Gupta, “Collusion attack resistant watermarking scheme for colored images using dct,” 2007 [44] K Rao and P Yip, “Discrete cosine transform, algorithms, advantages, applications,” 1990 [45] D Marshall, “The discrete transform (dct),” 10 April 2010, accessed time June 2020 https://www.cs.cf.ac.uk/Dave/Multimedia/node231.html [46] B K Ahmed, “Dct image compression by run-length and shift coding techniques,” 2011 [47] New Technique for DCT-PCA Based Face Recognition 2011 [48] Z Sufyanu, F S Mohamad, and A A Yusuf, “A new discrete cosine transform on face recognition through histograms for an optimized compression,” 2015 [49] Wikipedia, “Discrete cosine transform,” 18 November 2005, accessed time May 2020 https://en.wikipedia.org/wiki/Discrete_cosine_transform [50] “The 2d discrete cosine transform and image compression,” 17 February 2007, accessed time May 2020 http://dmr.ath.cx/gfx/dct/ [51] J Nagi, “Design of an efficient high-speed face recognition system,” 2006 [52] B A Wandell, “Foundations of vision, chapter 8: Multiresolution image representations,” 2016 [53] Z Sufyanu, F S Mohamad, and A S Ben-Musa, “Choice of illumination normalization algorithm for preprocessing efficiency of discrete cosine transform,” International Journal of Applied Engineering, vol 10, no pp 6341-6351, 2015 [54] M Lee and C H Park, “An efficient image normalization method for face recognition under varying illuminations,” The 1st ACM international conference on multimedia information retrieval, pp 128-133, 2008 [55] V Struc and N Pavesic, “Gabor-based kernel partial-least-squares discrimination features for face recognition,” 2009 [56] S Du and R K Ward, “Adaptive region-based image enhancement method for robust face recognition under variable illumination conditions,” IEEE transactions on circuits and systems for video technology, vol 20, no 9, pp 1165-1175, 2010 135 BIBLIOGRAPHY [57] V Struc, “The inface toolbox v2.0: The matlab toolbox for illumination invariant face recognition,” 2011 [58] V Struc and N Pavesic, “Photometric normalization techniques for illumination invariance,” 2011 [59] N Science and T C (NSTC), “Biometrics testing and statistics,” August 2006, accessed time July 2020 http://www.biometrics.gov/documents/ biotestingandstats.pdf [60] C Chi-Tsong, “Digital signal processing: spectral computation and filter design,” 2001 [61] P.-C Su, H.-J M Wang, and C.-C J Kuo, “Digital watermarking on ebcot compressed images,” SPIE’s International Symposium on Optical Science, Engineering, and Instrumentation, International Society for Optics and Photonics, 1999 [62] J K Anil, R P W Duin, and J Mao, “Statistical pattern recognition: A review,” Pattern Analysis and Machine Intelligence, pp 4-37, IEEE Transactions, 2000 [63] B Er, “Microsoft presents : Deep residual networks,” retrieved August 2016, accessed time June 2020 https://medium.com/@bakiiii/ microsoft-presents-deep-residual-networks-d0ebd3fe5887 [64] “The database of faces,” retrieved March 2016, accessed time July 2020 http: //www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html [65] E Aybar, TopolojikKenar slecleri PhD thesis, Anadolu Üniversitesi, 2003 [66] A Abdallah, M A El-Nasr, and A L Abbott, “A new face detection technique using 2d-dct and self organizing feature map,” International Journal of Computer, Electrical, Automation, Control and Information Engineering, 2007 [67] “Neural network toolbox glossary,” retrieved March 2016, accessed time June 2020 http://www.mathworks.com/help/nnet/gs/_bsr3kgi-1.html#bsr3kgi-33 [68] A B Hassanat, “Solving the problem of the k parameter in the knn classifier using an ensemble learning approach,” International Journal of Computer Science and Information Security, Vol 12, No 8, August 2014 [69] K Bache and M Lichman, “Uci machine learning repository,” 2013, accessed time 2020 http://archive.ics.uci.edu/ml 136 CURRICULUM VITAE Student name : NGUYEN DUC MINH Date of birth : 01/11/1994 Origin : Ho Chi Minh City Address : 173, Ngo Tat To Street, Ward 22, Binh Thanh district, Ho Chi Minh City EDUCATION BACKGROUND I University : Bach Khoa University (BKU) (9/2012-9/2016) Student ID : ILI12075 Advanced Program Department : Control Engineering and Automation Faculty : Electrical and Electronics Engineering II Post-Graduate : Bach Khoa University (BKU) (3/2017-9/2020) Student ID : 1770217 Department : Control Engineering and Automation Faculty : Electrical and Electronics Engineering PROFESSIONAL EXPERIENCE HELLA Vietnam Co Ltd (9/2016 – current) Position : Advanced SW Test Engineer PHẦN LÝ LỊCH TRÍCH NGANG Họ tên học viên: NGUYỄN ĐỨC MINH Ngày, tháng, năm sinh: 01/11/1994 Nơi sinh: TP.HCM Địa liên lạc: 173, đường Ngô Tất Tố, phường 22, quận Bình Thạnh, TP.HCM Q TRÌNH ĐÀO TẠO I Đại học : Đại học Bách Khoa TP.HCM (9/2012-9/2016) MSHV : ILI12075 Chương trình Tiên Tiến Chuyên ngành : Kỹ thuật Điều khiển Tự động hóa Khoa : Điện – Điện Tử II Cao học : Đại học Bách Khoa TP.HCM (3/2017-9/2020) MSHV : 1770217 Chuyên ngành : Kỹ thuật Điều khiển Tự động hóa Khoa : Điện – Điện Tử Q TRÌNH CƠNG TÁC Cơng ty TNHH HELLA Việt Nam Chức vụ : Kỹ sư kiểm thử phần mềm Thời gian công tác : 9/2016 đến ... TP.HCM Chuyên ngành: K? ?? thuật Điều Khiển Tự Động Hóa Mã số : 60520216 I TÊN ĐỀ TÀI: So Sánh Nhận Diện Khuôn Mặt Sử Dụng Giải Thuật K Gần Nhất với Mạng Nơ-ron Tự Cấu Trúc Face... pháp nhận diện khuôn mặt thông dụng nay, xây dựng hai hệ thống nhận diện khuôn mặt dựa vào hai thuật toán chọn lọc, ứng dụng nhận diện dựa vào hình ảnh dựa vào đặc trưng Hệ thống thứ sử dụng mạng. .. đại học tơi, sử dụng xử lý hình ảnh thuật toán IN 2D-DCT, với nhận diện mạng nơ-ron tự cấu trúc (SOM) môi trường MATLAB Vì thuật tốn ngơn ngữ khác nhau, ta tập trung vào so sánh k? ??t đầu hai phương