A multifunctional embedded system based on deep learning for assisting the cognition of visually impaired people

資 逢甲大學 訊 工 程 學 博士論文 系 基於深度學習來輔助視覺障礙者認知 之多功能嵌入式系統 A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People 指導教授： 竇其仁 林峰正 研 究 生：吳友輝 中 華 民 國 一 百 一 十 年 一 月 Tai ngay!!! Ban co the xoa dong chu nay!!! A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People Acknowledgement First and foremost, I would like to express my sincere gratitude to my advisor, Prof Chyi-Ren Dow, for his motivation, extensive experience, and immense knowledge I am very grateful for all his ideas, time, and funding contributions that laid the foundation for my research experience The passion and enthusiasm he has for the research were inspired and motivational to me, especially during tough periods in the Ph.D pursuit I am also thankful for the excellent pieces of advice he has offered as an outstanding professor These pieces of advice are valuable lessons for me in all the time of research and the future It is an honor for me to be one of his Ph.D students Again, I would like to convey my heartfelt gratitude to him I would like to express my sincere gratitude to my co-advisor, Prof Feng-Cheng Lin, for his scientific advice, knowledge, and valuable guidance I am very grateful for all his ideas, time, and the supported devices He always encourages and helps me to promote strengths in my research, especially appreciates our research results I am also thankful for his excellent pieces of advice It is an honor for me to be his first Ph.D student From the bottom of my heart, I would like to express my sincere gratitude to him again I would like to thank my dissertation committee: Prof Hsiao-Hsi Wang, Prof Tsung-Chuan Huang, Prof Lin-Huang Chang, Prof Cheng-Min Lin, and Prof Hsi-Min Chen, for their meaningful suggestions, which help me continue to improve and develop my research In addition, I am indebted to Szu-Yi Ho (Toni), who gave me numerous insightful discussions and suggestions She supported and shared with me to resolve my faced challenges in related research issues, especially publishing papers i FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People I would always remember my fellow labmates at the Mobile Computing lab for the inspiring discussions, unconditional supports, friendship, and for all the fun-time we have had in the last four years In particular, my gratitude goes to Ms Yu-Yun Chang (Amber) and Mr Kuan-Chieh Wang (Rich) for providing essential local supports during the years Last but not least, I am grateful to my family members for all their encouragements and faith in me They gave me enough moral support, encouragement, and motivation to accomplish the personal goals And most of all for my parents, who raised me up with unconditional love and gave me unlimited support in every decision I have made ii FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People 摘要 視力障礙的人在生活中面臨許多困難，例如，無輔助導航，獲取訊息和情境 感知。儘管許多智慧型裝置可用來幫助視障人士，但大多數只在提供導航幫助和 避障。在本研究中，我們專注於情境感知和周遭物件辨識。與大多數主從式架構 或是單台桌機運算所進行的研究不同，我們提出了一種基於深度學習的多功能嵌 入式系統，以幫助視覺障礙者對周遭環境的認知。我們提出的系統還克服了使用 上的區域限制，並增強了導航任務的能力。我們使用嵌入式設備（NVIDIA Jetson AGX Xavier）作為主要的處理器模組，並連接到其他外部周邊設備（像是網路鏡 頭、藍芽喇叭、螢幕、滑鼠和藍芽音訊配對器）。它幾乎可以執行所有主機應有 的系統功能，包括影像蒐集，影像處理和結果呈現。首先，系統的網路鏡頭用於 擷取使用者當前場景。然後，透過遙控器執行所選取的功能來處理該圖像。最後， 系統將當前場景的結果描述，從文字描述轉為語音，並由藍芽喇叭將其傳達給使 用者。該系統的三個主要功能，包括臉部辨識和情緒分類感知（第一個功能）， 年齡和性別分類（第二個功能）以及物體檢測（第三個功能）。該系統是基於不 同的深度學習模型構建的，但對於視力障礙的人使用上可能會成為挑戰。因此， 我們還提出了一種可以有效選擇功能的過程，以減輕視障人士的系統控制的複雜 性。最後，完成設計，製造和測試原型，並進行實驗驗證。利用原型機上獲得的 實驗結果，證明了所提系統的性能可靠度。基於辨識和分類準確性、計算時間及 實際適用性的結果證明，該系統是可行的，並且可以有效地用於幫助視障人士。 關鍵詞：年齡分類，情緒分類，臉部辨識，性別分類，對象檢測。 iii FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People Abstract Individuals with visual impairment confront many difficulties in their living, for example, unassisted navigation, access to information, and context-aware Although many smart devices were designed to assist visually impaired people, most of them aimed to provide navigation assistance and obstacle avoidance In this study, we focus on context-aware and surrounding object recognition Unlike most studies, which were implemented on servers or laptop computers, we propose a multifunctional embedded system based on deep learning for assisting the cognition of visually impaired people This proposed system also overcomes the limitation of area usage and enhances the capabilities of navigation tasks An embedded device (NVIDIA Jetson AGX Xavier) is employed as a central processor module in the system and connected to peripheral devices (webcam, speaker, monitor, mouse, and Bluetooth audio transmitter adapter) It performs almost all the system functions, including image collection, image processing, and result description First, the webcam of the system is used to capture the current scene of the user Then, this image is processed by following the selected function that is executed through a remote controller Lastly, the system converts the result description of the current scene from text to voice and delivers it to the user by the speaker Three main functions of this system include face recognition and emotion classification (the first function), age and gender classification (the second function), and object detection (the third function) This system is built based on different deep learning models, and it may become a challenge for visually impaired people Therefore, we also propose a process that can select functions efficiently to ease the complexity of the system control for visually impaired people Finally, a prototype is designed, fabricated, and tested for experimental validation The performance of the proposed system is demonstrated using results obtained from the experiments on the prototype iv FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People Results based on recognition and classification accuracy, computing time, and practical applicability prove that the proposed system is feasible and can be effectively used to assist visually impaired people Keywords: Age Classification, Emotion Classification, Face Recognition, Gender Classification, Object Detection v FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People Table of Contents Acknowledgement i 摘要 iii Abstract iv Table of Contents vi List of Figures ix List of Tables xi Chapter Introduction 1.1 Motivation 1.2 Overview of Research 1.3 Dissertation Organization Chapter Related Work 2.1 Face Recognition 2.2 Gender, Age and Emotion Classification 11 2.3 Object Detection 14 2.4 Smart Healthcare 16 Chapter System Overview 19 3.1 System Architecture 19 3.2 Function Selection 21 3.2.1 Remote Controller 21 3.2.2 Function Selection Process 23 3.3 NVIDIA Jetson AGX Xavier 25 3.3.1 NVIDIA Jetson Family Introduction 25 3.3.2 Technical Specification of NVIDIA Jetson AGX Xavier 26 Chapter Face Recognition Function 29 vi FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People 4.1 Overview of Face Recognition Function 29 4.2 Dataset Collection 30 4.3 Model Architectures 33 4.4 Enrolling a New Person 36 Chapter Gender, Age and Emotion Classification Function 38 5.1 Overview of Gender, Age and Emotion Classification Function 38 5.2 Gender Classification Schemes 39 5.3 Age Classification Schemes 41 5.4 Emotion Classification Schemes 42 Chapter Object Detection Function 47 6.1 Overview of Object Detection Function 47 6.2 Object Detection Schemes 48 6.2.1 Two-Stage Detectors 48 6.2.2 One-Stage Detectors 49 6.3 Arrangement of Result Description 52 Chapter System Prototype and Implementation 53 7.1 Devices in System Implementation 53 7.2 Initialization Program in Embedded System 55 7.3 Dataset Collection 58 Chapter Experimental Results 60 8.1 Evaluation of Face Recognition Results 60 8.1.1 Results Evaluation in Terms of Precision and Recall 60 8.1.2 Analysis Results of Face Recognition 63 8.1.3 Results Comparison in Multiple Standard Datasets 64 8.2 Examination Results of Gender, Age and Emotion Classification 65 8.2.1 Evaluation Results of Gender Classification 65 vii FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People (3) Improve the system The proposed system should be widely put into practice to more specifically and deeply evaluate the functions Thus, we plan to cooperate with the Visually Impaired Care Association of Taichung City to practically apply the proposed system to visually impaired people in their real-life Besides, the age and emotion functions will be improved by selecting or developing more efficient CNN models We also consider using the YOLOv4 model to improve the object detection function Furthermore, we intend to extend the system functions by combining several sensors, such as ultrasonic sensors, infrared, laser, and distance sensors 78 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People References [1] O Agbo-Ajala and S Viriri, “Deep Learning Approach for Facial Age Classification: A Survey of the State-of-the-Art,” Artificial Intelligence Review, pp 1–35, Jun 2020 [2] O Agbo-Ajala and S Viriri, “Face-Based Age and Gender Classification Using Deep Learning Model,” in Proceedings of Image and Video Technology, Sydney, New South Wales, Australia, pp 125–137, Nov 2019 [3] P Angin and B K Bhargava, “Real-time Mobile-Cloud Computing for ContextAware Blind Navigation,” International Journal of Next Generation Computing, vol 2, no 2, pp 1–13, Jul 2011 [4] O Arriaga, P G Ploger and M Valdenegro, “Real-Time Convolutional Neural Networks for Emotion and Gender Classification,” arXiv:1710.07557, pp 1–5, Oct 2017 [5] A Ashok and J John, “Facial Expression Recognition System for Visually Impaired,” in Proceedings of International Conference on Intelligent Data Communication Technologies and Internet of Things (ICICI), Coimbatore, Tamil Nadu, India, pp 244–250, Aug 2018 [6] V Aza, Indrabayu and I S Areni, “Face Recognition Using Local Binary Pattern Histogram for Visually Impaired People,” in Proceedings of International Seminar on Application for Technology of Information and Communication (iSemantic), Semarang, Indonesia, pp 241–245, Sep 2019 [7] J Bai, Z Liu, Y Lin, Y Li, S Lian and D Liu, “Wearable Travel Aid for Environment Perception and Navigation of Visually Impaired People,” Electronics, vol 8, no 6, pp 1–27, Jun 2019 79 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People [8] S A Bargal, E Barsoum, C C Ferrer and C Zhang, “Emotion Recognition in the Wild from Videos Using Images,” in Proceedings of 18th ACM International Conference on Multimodal Interaction, Tokyo, Japan, pp 433–436, Nov 2016 [9] J Cai, Z Meng, A S Khan, Z Li, J O’Reilly and Y Tong, “Island Loss for Learning Discriminative Features in Facial Expression Recognition,” in Proceedings of 13th IEEE International Conference on Automatic Face Gesture Recognition, Xi’an, China, pp 302–309, May 2018 [10] G Capi, “Development of a New Robotic System for Assisting and Guiding Visually Impaired People,” in Proceedings of IEEE International Conference on Robotics and Biomimetics (ROBIO), Guangzhou, China, pp 229–234, Dec 2012 [11] W J Chang, J P Su, L B Chen, M C Chen, C H Hsu, C H Yang, C Y Sie and C H Chuang, “An AI Edge Computing Based Wearable Assistive Device for Visually Impaired People Zebra-Crossing Walking,” in Proceedings of IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, Nevada, USA, pp 1–2, Jan 2020 [12] S Chaudhry and R Chandra, “Design of a Mobile Face Recognition System for Visually Impaired Persons,” arXiv:1502.00756, pp 1–11, Jun 2015 [13] S Chen, D Yao, H Cao and C Shen, “A Novel Approach to Wearable Image Recognition Systems to Aid Visually Impaired People,” Applied Sciences, vol 9, no 16, pp 1–20, Jan 2019 [14] J Cheng, Y Li, J Wang, L Yu and S Wang, “Exploiting Effective Facial Patches for Robust Gender Recognition,” Tsinghua Science and Technology, vol 24, no 3, pp 333–345, Jun 2019 [15] S A Cheraghi, V Namboodiri and L Walker, “Guidebeacon: Beacon-Based 80 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People Indoor Wayfinding for the Blind, Visually Impaired, and Disoriented,” in Proceedings of IEEE International Conference on Pervasive Computing and Communications (PerCom), Kona, Hawaii, USA, pp 121–130, Mar 2017 [16] S Das, “A Novel Emotion Recognition Model for the Visually Impaired,” in Proceedings of IEEE 5th International Conference for Convergence in Technology (I2CT), Pune, India, pp 1–6, Mar 2019 [17] J Deng, J Guo, N Xue and S Zafeiriou, “ArcFace: Additive Angular Margin Loss for Deep Face Recognition,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp 4690–4699, Jun 2019 [18] A Dhomne, R Kumar and V Bhan, “Gender Recognition Through Face Using Deep Learning,” Procedia Computer Science, vol 132, pp 2–10, Jan 2018 [19] C R Dow, H H Ngo, L H Lee, P Y Lai, K C Wang and V T Bui, “A Crosswalk Pedestrian Recognition System by Using Deep Learning and Zebra‐ Crossing Recognition Techniques,” Software: Practice and Experience, vol 50, no 5, pp 630–644, Aug 2019 [20] M Duan, K Li, C Yang and K Li, “A Hybrid Deep Learning CNN–ELM for Age and Gender Classification,” Neurocomputing, vol 275, pp 448–461, Jan 2018 [21] L Everding, L Walger, V S Ghaderi and J Conradt, “A Mobility Device for the Blind with Improved Vertical Resolution Using Dynamic Vision Sensors,” in Proceedings of IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom), Munich, Germany, pp 1–5, Sep 2016 [22] D Franklin, “Introducing Jetson Xavier NX,” NVIDIA Developer, Available 81 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People online: https://developer.nvidia.com/blog/jetson-xavier-nx-the-worlds-smallestai-supercomputer/ [23] R Girshick, “Fast R-CNN,” in Proceedings of IEEE International Conference on Computer Vision, Santiago, Chile, pp 1440–1448, Dec 2015 [24] R Girshick, J Donahue, T Darrell and J Malik, “Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, pp 580–587, Jun 2014 [25] V Gupta and S Mallick, “Face Recognition: An Introduction for Beginners,” Learn OpenCV, Available online: https://www.learnopencv.com/face- recognition-an-introduction-for-beginners [26] K He, G Gkioxari, P Dollár and R Girshick, “Mask R-CNN,” arXiv:1703.06870, pp 1–12, Jan 2018 [27] K He, X Zhang, S Ren and J Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp 770–778, Jun 2016 [28] P Hu, D Cai, S Wang, A Yao and Y Chen, “Learning Supervised Scoring Ensemble for Emotion Recognition in the Wild,” in Proceedings of 19th ACM International Conference on Multimodal Interaction, New York, NY, USA, pp 553–560, Nov 2017 [29] W Hu, K Wang, K Yang, R Cheng, Y Ye, L Sun and Z Xu, “A Comparative Study in Real-Time Scene Sonification for Visually Impaired People,” Sensors, vol 20, no 11, pp 1–17, Jan 2020 [30] Z Hu, Y Wen, J Wang, M Wang, R Hong and S Yan, “Facial Age Estimation 82 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People with Age Difference,” IEEE Transactions on Image Processing, vol 26, no 7, pp 3087–3097, Jul 2017 [31] M M Islam, M S Sadi, K Z Zamli and M M Ahmed, “Developing Walking Assistants for Visually Impaired People: A Review,” IEEE Sensors Journal, vol 19, no 8, pp 2814–2828, Apr 2019 [32] M T Islam, M Ahmad and A S Bappy, “Microprocessor-Based Smart Blind Glass System for Visually Impaired People,” in Proceedings of International Joint Conference on Computational Intelligence, Dhaka, Bangladesh, pp 151– 161, Dec 2018 [33] D K Jain, P Shamsolmoali and P Sehdev, “Extended Deep Neural Network for Facial Emotion Recognition,” Pattern Recognition Letters, vol 120, pp 69–74, Apr 2019 [34] A Jaiswal, A K Raju and S Deb, “Facial Emotion Detection Using Deep Learning,” in Proceedings of International Conference for Emerging Technology (INCET), Belgaum, India, pp 1–5, Jun 2020 [35] M A Jalal, L Mihaylova and R K Moore, “An End-to-End Deep Neural Network for Facial Emotion Classification,” in Proceedings of 22th International Conference on Information Fusion (FUSION), Ottawa, Ontario, Canada, pp 1–7, Jul 2019 [36] L Jiao, F Zhang, F Liu, S Yang, L Li, Z Feng and R Qu, “A Survey of Deep Learning-Based Object Detection,” IEEE Access, vol 7, pp 128837–128868, Sep 2019 [37] R C Joshi, S Yadav, M K Dutta and C M Travieso-Gonzalez, “Efficient MultiObject Detection and Smart Navigation Using Artificial Intelligence for Visually 83 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People Impaired People,” Entropy, vol 22, no 9, pp 1–17, Sep 2020 [38] M C Kang, S H Chae, J Y Sun, J W Yoo and S J Ko, “A Novel Obstacle Detection Method Based on Deformable Grid for the Visually Impaired,” IEEE Transactions on Consumer Electronics, vol 61, no 3, pp 376–383, Aug 2015 [39] R K Katzschmann, B Araki and D Rus, “Safe Local Navigation for Visually Impaired Users with a Time-of-Flight and Haptic Feedback Device,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol 26, no 3, pp 583–593, Mar 2018 [40] S Khade and Y H Dandawate, “Hardware Implementation of Obstacle Detection for Assisting Visually Impaired People in an Unfamiliar Environment by Using Raspberry Pi,” in Proceedings of Smart Trends in Information Technology and Computer Communications, Jaipur, Rajasthan, India, pp 889–895, Aug 2016 [41] K Khan, M Attique, I Syed and A Gul, “Automatic Gender Classification through Face Segmentation,” Symmetry, vol 11, no 6, pp 1–14, Jun 2019 [42] E Ko and E Y Kim, “A Vision-Based Wayfinding System for Visually Impaired People Using Situation Awareness and Activity-Based Instructions,” Sensors, vol 17, no 8, pp 1–34, Aug 2017 [43] A Krizhevsky, I Sutskever and G E Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” Communications of the ACM, vol 60, no 6, pp 84–90, May 2017 [44] G Levi and T Hassner, “Age and Gender Classification Using Convolutional Neural Networks,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, Massachusetts, USA, pp 34–42, Jun 2015 84 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People [45] B Li, J P Muñoz, X Rong, J Xiao, Y Tian and A Arditi, “ISANA: Wearable Context-Aware Indoor Assistive Navigation with Obstacle Avoidance for the Blind,” in Proceedings of Computer Vision – ECCV 2016 Workshops, Amsterdam, Netherlands, pp 448–462, Oct 2016 [46] S Li and W Deng, “Deep Facial Expression Recognition: A Survey,” IEEE Transactions on Affective Computing, pp 1–20, Mar 2020 [47] H Liao, Y Yan, W Dai and P Fan, “Age Estimation of Face Images Based on CNN and Divide-and-Rule Strategy,” Mathematical Problems in Engineering, vol 2018, pp 1–8, Jun 2018 [48] S S Liew, M K Hani, S A Radzi and R Bakhteri, “Gender Classification: A Convolutional Neural Network Approach,” Turkish Journal of Electrical Engineering & Computer Sciences, vol 24, no 3, pp 1248–1264, Mar 2016 [49] F C Lin, H H Ngo and C R Dow, “A Cloud-Based Face Video Retrieval System with Deep Learning,” The Journal of Supercomputing, vol 76, no 11, pp 8473–8493, Nov 2020 [50] F Lin, Y Wu, Y Zhuang, X Long and W Xu, “Human Gender Classification: A Review,” International Journal of Biometrics, vol 8, no 3–4, pp 275–300, Jan 2016 [51] T Y Lin, P Goyal, R Girshick, K He and P Dollar, “Focal Loss for Dense Object Detection,” in Proceedings of IEEE International Conference on Computer Vision, Venice, Italy, pp 2980–2988, Oct 2017 [52] T Y Lin, M Maire, S Belongie, L Bourdev, R Girshick, J Hays, P Perona, D Ramanan, C L Zitnick and P Dollár, “Microsoft COCO: Common Objects in Context,” arXiv:1405.0312, pp 1–15, Feb 2015 85 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People [53] Y Lin, K Wang, W Yi and S Lian, “Deep Learning Based Wearable Assistive System for Visually Impaired People,” in Proceedings of IEEE/CVF International Conference on Computer Vision Workshops, Seoul, South Korea, pp 1–9, Oct 2019 [54] M Liu, S Li, S Shan and X Chen, “AU-Aware Deep Networks for Facial Expression Recognition,” in Proceedings of 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, Shanghai, China, pp 1–6, Apr 2013 [55] W Liu, D Anguelov, D Erhan, C Szegedy, S Reed, C Y Fu and A C Berg, “SSD: Single Shot MultiBox Detector,” arXiv:1512.02325, pp 1–17, Dec 2016 [56] N Long, K Wang, R Cheng, W Hu and K Yang, “Unifying Obstacle Detection, Recognition, and Fusion Based on Millimeter Wave Radar and RGB-Depth Sensors for the Visually Impaired,” Review of Scientific Instruments, vol 90, no 4, pp 1–12, Apr 2019 [57] A Mancini, E Frontoni and P Zingaretti, “Mechatronic System to Help Visually Impaired Users During Walking and Running,” IEEE Transactions on Intelligent Transportation Systems, vol 19, no 2, pp 649–660, Feb 2018 [58] M L Mekhalfi, F Melgani, A Zeggada, F G B D Natale, M A M Salem and A Khamis, “Recovering the Sight to Blind People in Indoor Environments with Smart Technologies,” Expert Systems with Applications, vol 46, pp 129–138, Mar 2016 [59] V V Meshram, K Patil, V A Meshram and F C Shu, “An Astute Assistive Device for Mobility and Object Recognition for Visually Impaired People,” IEEE Transactions on Human-Machine Systems, vol 49, no 5, pp 449–460, Oct 2019 86 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People [60] B Mocanu, R Tapu and T Zaharia, “DEEP-SEE FACE: A Mobile Face Recognition System Dedicated to Visually Impaired People,” IEEE Access, vol 6, pp 51975–51985, Sep 2018 [61] A A Nada, M A Fakhr and A F Seddik, “Assistive Infrared Sensor Based Smart Stick for Blind People,” in Proceedings of Science and Information Conference (SAI), London, United Kingdom, pp 1149–1154, Jul 2015 [62] M Nakajima and S Haruyama, “New Indoor Navigation System for Visually Impaired People Using Visible Light Communication,” EURASIP Journal on Wireless Communications and Networking, vol 2013, no 1, pp 1–10, Feb 2013 [63] S H Nam, Y H Kim, N Q Truong, J Choi and K R Park, “Age Estimation by Super-Resolution Reconstruction Based on Adversarial Networks,” IEEE Access, vol 8, pp 17103–17120, Jan 2020 [64] L B Neto, F Grijalva, V R M L Maike, L C Martini, D Florencio, M C C Baranauskas, A Rocha and S Goldenstein, “A Kinect-Based Wearable Face Recognition System to Aid Visually Impaired Users,” IEEE Transactions on Human-Machine Systems, vol 47, no 1, pp 52–64, Feb 2017 [65] H H Ngo, F C Lin, Y T Sehn, M Tu and C R Dow, “A Room Monitoring System Using Deep Learning and Perspective Correction Techniques,” Applied Sciences, vol 10, no 13, pp 1–22, Jul 2020 [66] O M Parkhi, A Vedaldi and A Zisserman, “Deep Face Recognition,” in Proceedings of British Machine Vision Conference (BMVC), Swansea, United Kingdom, pp 1–12, Sep 2015 [67] J L Pech-Pacheco, G Cristobal, J Chamorro-Martinez and J FernandezValdivia, “Diatom Autofocusing in Brightfield Microscopy: A Comparative 87 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People Study,” in Proceedings of 15th International Conference on Pattern Recognition ICPR-2000, Barcelona, Spain, pp 314–317, Sep 2000 [68] S Pertuz, D Puig and M A Garcia, “Analysis of Focus Measure Operators for Shape-from-Focus,” Pattern Recognition, vol 46, no 5, pp 1415–1432, May 2013 [69] P Punyani, R Gupta and A Kumar, “Neural Networks for Facial Age Estimation: A Survey on Recent Advances,” Artificial Intelligence Review, vol 53, no 5, pp 3299–3347, Jun 2020 [70] J Redmon, S Divvala, R Girshick and A Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, USA, pp 779– 788, Jun 2016 [71] J Redmon and A Farhadi, “YOLO9000: Better, Faster, Stronger,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA, pp 7263–7271, Jul 2017 [72] J Redmon and A Farhadi, “YOLOv3: An Incremental Improvement,” arXiv:1804.02767, pp 1–6, Apr 2018 [73] S Ren, K He, R Girshick and J Sun, “Faster R-CNN: Towards Real-Time Object Detection With Region Proposal Networks,” in Proceedings of Advances in Neural Information Processing Systems 28, Montreal, Canada, pp 91–99, Dec 2015 [74] F Schroff, D Kalenichenko and J Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, pp 815– 88 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People 823, Jun 2015 [75] C Shang and H Ai, “Cluster Convolutional Neural Networks for Facial Age Estimation,” in Proceedings of IEEE International Conference on Image Processing (ICIP), Beijing, China, pp 1817–1821, Sep 2017 [76] K Simonyan and A Zisserman, “Very Deep Convolutional Networks for LargeScale Image Recognition,” arXiv:1409.1556, pp 1–14, Apr 2015 [77] C Szegedy, S Ioffe, V Vanhoucke and A Alemi, “Inception-v4, InceptionResNet and the Impact of Residual Connections on Learning,” arXiv:1602.07261, pp 1–12, Aug 2016 [78] C Szegedy, W Liu, Y Jia, P Sermanet, S Reed, D Anguelov, D Erhan, V Vanhoucke and A Rabinovich, “Going Deeper with Convolutions,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Boston, Massachusetts, USA, pp 1–9, Jun 2015 [79] M S R Tanveer, M M A Hashem and M K Hossain, “Android Assistant Eyemate for Blind and Blind Tracker,” in Proceedings of 18th International Conference on Computer and Information Technology (ICCIT), Dhaka, Bangladesh, pp 266–271, Dec 2015 [80] R Tapu, B Mocanu and T Zaharia, “Seeing without Sight - An Automatic Cognition System Dedicated to Blind and Visually Impaired People,” in Proceedings of IEEE International Conference on Computer Vision Workshops, Venice, Italy, pp 1452–1459, Oct 2017 [81] L Ţepelea, I Gavriluţ and A Gacsádi, “Smartphone Application to Assist Visually Impaired People,” in Proceedings of 14th International Conference on Engineering of Modern Electric Systems (EMES), Oradea, Romania, pp 228–231, 89 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People Jun 2017 [82] Y Tian, X Yang, C Yi and A Arditi, “Toward a Computer Vision-Based Wayfinding Aid for Blind Persons to Access Unfamiliar Indoor Environments,” Machine Vision and Applications, vol 24, no 3, pp 521–535, Apr 2013 [83] R Trabelsi, I Jabri, F Melgani, F Smach, N Conci and A Bouallegue, “Indoor Object Recognition in RGBD Images with Complex-Valued Neural Networks for Visually-Impaired People,” Neurocomputing, vol 330, pp 94–103, Feb 2019 [84] H C Wang, R K Katzschmann, S Teng, B Araki, L Giarré and D Rus, “Enabling Independent Navigation for Visually Impaired People through a Wearable Vision-Based Feedback System,” in Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Marina Bay, Singapore, pp 6533–6540, May 2017 [85] K Yang, K Wang, L M Bergasa, E Romera, W Hu, D Sun, J Sun, R Cheng, T Chen and E López, “Unifying Terrain Awareness for the Visually Impaired through Real-Time Semantic Segmentation,” Sensors, vol 18, no 5, pp 1–32, May 2018 [86] T Y Yang, Y H Huang, Y Y Lin, P C Hsiu and Y Y Chuang, “SSR-Net: A Compact Soft Stagewise Regression Network for Age Estimation,” in Proceedings of Twenty-Seventh International Joint Conference on Artificial Intelligence, Stockholm, Sweden, pp 1078–1084, Jul 2018 [87] E Zahir, M S Hossain, M W Iqbal, I Jalil and S M Kabir, “Implementing and Testing an Ultrasonic Sensor Based Mobility Aid for a Visually Impaired Person,” in Proceedings of IEEE Region 10 Humanitarian Technology Conference (R10HTC), Dhaka, Bangladesh, pp 453–456, Dec 2017 90 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People [88] K Zhang, C Gao, L Guo, M Sun, X Yuan, T X Han, Z Zhao and B Li, “Age Group and Gender Estimation in the Wild with Deep RoR Architecture,” IEEE Access, vol 5, pp 22492–22503, Oct 2017 [89] K Zhang, Y Huang, Y Du and L Wang, “Facial Expression Recognition Based on Deep Evolutional Spatial-Temporal Networks,” IEEE Transactions on Image Processing, vol 26, no 9, pp 4193–4203, Sep 2017 [90] Caltech Faces, Available online: http://www.vision.caltech.edu/html- files/archive.html [91] Extended Yale Face Database B, Available online: http://vision.ucsd.edu/content/ extended-yale-face-database-b-b [92] Face Alignment Using MTCNN, Available online: https://github.com/ davidsandberg/facenet/tree/master/src/align [93] FaceNet, Available online: https://github.com/davidsandberg/facenet/ [94] Insight Face Tensorflow, Available online: https://github.com/luckycallor/ InsightFace-tensorflow [95] MIT-CBCL Face Recognition Database, Available online: http://cbcl.mit.edu/ software-datasets/heisele/facerecognition-database.html [96] NVIDIA Embedded Systems for Next-Generation Autonomous Machines, NVIDIA Website, Available online: https://www.nvidia.com/en-us/autonomousmachines/embedded-systems/ [97] NVIDIA Jetson AGX Xavier, NVIDIA Developer Blog, Available online: https://developer.nvidia.com/blog/nvidia-jetson-agx-xavier-32-teraops-airobotics/ [98] SSR-Net, Available online: https://github.com/shamangary/SSR-Net 91 FCU e-Theses & Dissertations (2021) A Multifunctional Embedded System Based on Deep Learning for Assisting the Cognition of Visually Impaired People [99] Vision Impairment and Blindness, World Health Organization, Available online: https://www.who.int/news-room/fact-sheets/detail/blindness-and-visualimpairment 92 FCU e-Theses & Dissertations (2021)