Xây dựng sản phẩm kính thông minh cho người khiếm thị bằng ứng dụng trí tuệ nhân tạo và internet vạn vật

751*,+&63+0 ,+&ơ11* +2$7,1+& BO CO +ẽ$/81771*+, 7ầ17ơ, ;ặ 17 Bҧ QJĈiQKJLiPӝWVӕPҥQJQѫ -ron Object detection [5] 17 Bҧ ng 1.3 So sánh Arduino Raspberry Pi 25 Bҧ QJĈһFWҧWәQJTXiWFiFWiFQKk 33 Bҧ ng 2ĈһF WҧEҧQJ8VHU 37 Bҧ QJĈһFWҧEҧQJ*ODVVHV 38 Bҧ QJĈһFWҧEҧQJ6RIWZDUH9HUVLRQ 38 Bҧ QJĈһFWҧEҧQJ*URXS 39 Bҧ ng ĈһFWҧEҧQJ*URXS0HPEHU 39 Bҧ QJĈһFWҧEҧQJ*ODVVHV/LQN 39 Bҧ QJĈһFWҧFDVDVӱGөQJFKӭFQă 44 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ 45 Bҧ QJĈһFWҧFDVӱGөQJFKӭF QăQJ&ҧQKEiRWӵÿӝQJ 46 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ 48 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ 49 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ ҵQJJLӑQJQyL 50 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ 52 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ ng mobile 53 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ ӱLGӳOLӋXҧQKFKөS 55 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ WUDYjFұSQKұWSK PӅPPӟL 56 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ 58 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ JQKұS 58 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ 59 697+ĈһQJ7KDQK+jR - 16CNTT3 iii Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ ân 60 Bҧ QJĈһFWҧWiFQKkQFRQFӫDWiF 62 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ 63 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ 63 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ ên 65 Bҧ QJĈһFWҧFDVӱGөQJF KӭFQăQJ4XҧQOtNtQKFK 66 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ ӭQJGөQJ 68 Bҧ QJĈһFWҧFDVӱGөQJFKӭFQăQJ 69 Bҧ QJ'DQKViFKFiFYұWKXҩQOX\ӋQ 75 Bҧ QJĈiQKJLiPRGHOKXҩQOX \ӋQWUrQWұSGӳOLӋX7 79 Bҧ QJ'DQKViFKFiFYұWVDXNKLÿL 82 Bҧ QJ3KҫQFӭQJYjJLiWKjQKVҧQS 83 697+ĈһQJ7KDQK+jR - 16CNTT3 iv DANH MӨC HÌNH VӀ Hu QK$,YjFiFOƭQKYӵFWKXӝF$, Hu QK0ӝWQѫ -URQÿѫQJLҧQ Hu nK0LQKKRҥWtQKFKҩWPRQJPXӕQ -ron 10 Hu QKĈӗWKӏKjPEѭӟFQKҧ\ 11 Hu QKĈӗWKӏKjPVLJPRLG 12 Hu QK0ҥQJQѫ -URQFѫEҧQ 13 Hu nK%ӝOӑFFҥQKOjFiFPDWUұQN -1, 15 Hu QK9tGөYӅEӝOӑFFҥQKÿӭQJS ngang trên) YӟLÿҫXYjROjҧQKVӕ YLӃWWD\ 16 Hu nh 1.9 Bài toán Object detection 16 Hu QK.ӃWTXҧGӵÿRiQVDXOӟS 19 Hu QK.KiFELӋWJLӳDPi\KӑFWUX\ 21 Hu QKӬQJGөQ JKӑFFKX\ӇQ7UDQ sfer Learning) 22 Hu QK7KLӃWEӏ5DVSEHUU\3L% 26 Hu QK0LQKKӑDTXiWUuQKOLrQNӃW mily App 32 Hu QK6ѫÿӗFDVӱGөQJWәQJTXiW 34 Hu QK6ѫÿӗNLӃQWU~FKӋWKӕQJWәQ 35 Hu nh 2.4 Mơ hình ER 36 Hu QK0{KuQKGӳOLӋXTXDQKӋ 37 Hu QK&ҩXWU~FҧQKYjFiFWK{QJWL LJLDQWӑDÿӝ*36 AWS S3 41 Hu QK7RSLFYjP{KuQKKRҥWÿӝQJF subscriber 42 Hu nh 2.8 Mơ hình SKҫQFӭQJ.tQKWK{QJPLQK 43 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ1 WKӇQJX\ KLӇP 44 697+ĈһQJ7KDQK+jR - 16CNTT3 v Hu nh 2.6ѫÿӗWXҫQWӵFKӭFQăQJ1KұQG 45 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ 46 Hu QK6ѫÿӗWXҫQWӵFKӭ FQăQJ&ҧQKEiRWӵÿӝQJY 47 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ 47 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJ&ҧQ 48 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ 49 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJ3Ki 50 Hu QK6ѫÿӗFDVӱG өQJFKӭFQăQJ7uPÿӗYұWE 50 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJ7uP 51 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ 52 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ 53 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJOLr QăQJ*ӱLWtQKLӋX626 54 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ 54 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJ*ӱL 55 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ PӅPPӟL 55 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJNLӇ PӅP 56 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ 57 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJĈăQ QKұS 59 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJĈăQ 60 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ 60 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJ&KӍ 61 Hu QK6ѫÿӗFDVӱGөQJFKӭFQ ăQJ4XҧQOtQKyP 62 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJ4Xҧ 64 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJ4Xҧ 64 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ 65 697+ĈһQJ7KDQK+jR - 16CNTT3 vi Hu nh 2.6ѫÿӗWXҫQWӵFKӭFQăQJ4XҧQO 67 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJ4Xҧ 67 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ 68 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJ7KL 69 Hu QK6ѫÿӗFDVӱGөQJFKӭFQăQJ 69 Hu QK6ѫÿӗWXҫQWӵFKӭFQăQJ1Kұ ông báo SOS 70 Hu QK6ѫÿӗOӟSKӋWKӕQJ 71 Hu nh 3.1 Quy trình KXҩQOX\ӋQGӳOLӋX 72 Hu QK&iFSKѭѫQJSKiSOjPJLjXW 75 Hu QK*iQQKmQWKӫF{QJFKRWӯQJK 77 Hu nh 3.4 File XML sinh sau trình gán nhãn 77 Hu QK&ҩXWU~FWKѭPөFFKXҭQEӏ 78 Hu QK%LӇXÿӗWKD\ÿәLÿӝWәQWKҩW n 79 Hu QK.tQKWK{QJPLQKVDXOҳSÿһW 85 Hu nh 3.&iFWKjQKSKҫQ.tQKWK{QJPLQK 85 Hu QK&KӭFQăQJQKұQGLӋQÿӗYұW kính 86 Hu QK&KӭFQăQJQKұQGLӋQ ÿӗYұWWUrQNtQK 86 Hu QK&KӭFQăQJQKұQGLӋQÿӗYұW 87 Hu QK&KӭFQăQJQ KұQGLӋQÿӗYұWWUrQNtQK 87 Hu QK*LDRGLӋQÿăQJQKұSYjWUDQ 88 Hu QK*LDRGLӋQ4XҧQOtQKyP 89 Hu QK*LDRGLӋQ4XҧQOtNtQK 89 Hu QK*LDRGLӋQQKұQWtQKLӋX626 90 697+ĈһQJ7KDQK+jR - 16CNTT3 vii DANH MӨC CÁC KÝ HIӊU, CHӲ VIӂT TҲT 9LӃW tҳW *LҧLWKtFK AI $UWLILFLDO,QWHOOLJHQFH7Ut QKkQWҥR IoT ,QWHUQHWRI7KLQJ,QWHUQHW CPU &HQWUDO3URFHVVLQJ8QLW%ӝ GPU *UDSKLFDO3URFHVVLQJ8QLW% API Application Programming Interface *LDRGLӋQOұS ӭQJGөQ g) ANNS Artificial Neural Networks 0ҥQJQѫ -URQQKkQWҥR CNN Convolutional Neural Network 0ҥQJQѫ -URQWtFKFKұ SSD Single Shot Multibox Detector 7rQJӑL PҥQJQѫ -ron) RAM Random Access Memory %ӝQKӟWUX\FұSQ IDE Integrated Development Environment 0{LWUѭӡQJW KӧSSKiWWULӇQӭQJGөQJ 39 B̫ ng 2.5 Ĉ̿FW̫E̫QJ*URXS Group 7KXӝFWtQK éQJKƭD LӇXGӳ Kích Khóa OLӋX GroupID Mã nhóm Integer GroupName Tên nhóm Varchar WKѭӟF X 255 B̫ ng 2.6 Ĉ̿F W̫E̫QJ*URXS0HPEHU GroupMember 7KXӝFWtQK Ý QJKƭD LӇXGӳ Kích Khóa OLӋX GroupID Mã nhóm Integer UID 0mQJѭӡLGQJ Integer Role &KӭFGDQK Integer WKѭӟF X X TXҧQWUӏYLrQ thành viên; -ĈDQJ ÿѭӧFPӡL B̫ ng 2.7 Ĉ̿FW̫E̫QJ*ODVVHV/LQN GlassesLink 7KXӝFWtQK éQJKƭD LӇXGӳ Kích Khóa OLӋX LinkID 0mOLrQNӃW Integer GroupID Mã nhóm GID Mã kính thơng minh Integer DateLink 1Jj\OLrQNӃW Date 697+ĈһQJ7KDQK+jR - 16CNTT3 Integer WKѭӟF X 40 GlassesName Tên kính thơng Varchar 255 minh BlindName 7rQQJѭӡLNKLӃPWKӏ Varchar 255 BlindYearBorn 1ăPVLQKQJѭӡL Integer NKLӃPWKӏ BlindAddress ĈӏDFKӍQJѭӡLNKLӃP Varchar 255 WKӏ Status 7uQKWUҥQJOLrQNӃW Integer ÿDQJFKӡQKұS Pm[iFQKұQÿm OLrQNӃW VerifyCode 0m[iFQKұQOLrQ Integer NӃW 2.2.3 Thành phҫQOѭXG ӳliӋ u ҧnh $:66OjGӏFKYөÿѭӧFVӱGөQJÿӇ +ӋWKӕQJ FyGҥQJGӳOLӋXҧQKOjҧQKÿҥLGLӋ PӛL GҥQJGӳOLӋXVӁÿѭӧFOѭXӣ mӛL WKѭPөFNKiFQKDX ҦQKDYDWDUVӁFyÿѭӡQJGүQ avatar/[uid].jpg => TURQJÿy : uid OjPmQJѭӡLGQJ ҦQK FKөSWӯNtQKVӁFyÿѭӡQJGү n: glassesdata/[gid]/[timestamp]_[latitude]_[longtitude]_[randomhashstring].jpg => 7URQJÿy - gid: mã cӫa kính - timestamp: thӡi gian chө pҧ QKÿѭ ӧFÿ ә i sӕmili giây kӇtӯ00:00:00 87&QJj\WKiQJQăP (Ví dө : 4/5/2020 3:23:25,0000 GMT => 1588564405000 (timestamp)) Timestamp giúp tӕ L ѭX GXQJ ӧng sӱ Oѭ dө QJÿ ӇOѭXWU ӳQJj\WKiQJYjFKtQK[iFÿ Ӄ n mili giây 697+ĈһQJ7KDQK+jR - 16CNTT3 41 - latitude, longtitudeKRjQKÿ ӝYjWXQJÿ ӝGPS cӫa kính tҥ i thӡLÿL Ӈ m chөp ҧ nh - randomhashstring: chuӛ i ngү u nhiên sinh bӣL/DPEGD$3, Ӈ ÿҧ m bҧ o tính bҧ o mұ t (chӍcó thӇlҩ \ÿѭ ӧFÿѭ ӡng dү n cӫa ҧ nh bҵ ng cách gӱi request tӟi API) Hu nh 2.6 &̭XWU~F̫QKYjFiFWK{QJWLQY͉ FO˱XWU AWS S3 2.2.4 Thành phҫn gӱLWK{QJEiRÿ Ӄ n ӭng dөQJGLÿ ӝng AWS SNS Firebase Cloud Messenging OjGӏFKYөVӱ JӱLWK{QJEiRÿҭ\3XVKQRILWLFD WLRQ ÿӃQӭQJGөQJGLÿ 616OjGӏFKYөTXҧQOt WLQQKҳQ616ÿѭӧFWәF ÿӅ YjVXEVFULEHUErQOҳQJQJKH  697+ĈһQJ7KDQK+jR - 16CNTT3 42 Hu nh 2.7 7RSLFYjP{KuQKKR̩Wÿ͡QJFͯD6 P͟LFOLHQWOj P͡W subscriber 0ӛLWRSLFӭQJYӟLPӛLQKyPJURXS PӛLVXEVFULE YӟLPӛLWKjQKYLrQPHPEHU WKXӝFJU KLFyEҩWFӭWKD\ÿәLQjRYӅWK{ PHPEHU KD\FiFNtQKWK{QJPLQKOLr KҳQ VӁÿѭӧFJӱLWӟLWRSLFYjPӑLWKjQK HWRSLFVXEVFUL ÿѭӧFWLQQKҳQ 616OjGӏFKYөWLQQKҳQWәQJTXiW QJѭӡL GQJ FXӕL Pj SKҧL FҫQ  GӏFK 0HVVHQJLQJ Yө JӱL GӏFK WK{QJ EiR ÿҭ\ WӟL sh WK 1RILWLFDWLRQGӏFKYөJӱLWK{QJEiR OTP SMS khác 2.3 ThiӃ t kӃKính thơng minh (Smart Glasses) 2.3.1 ThiӃ t kӃphҫn cӭng ĈӇÿiSӭQJFiFFKӭFQăQJÿһWUDӣ.t LQKWDFҫQFiF Raspberry Pi: Là nhân xӱOtÿmWtFKK ӧp sҹ n kӃ t nӕ i internet/wifi Camera: Cung cҩ p hình ҧ nh cho chӭFQăQJQK ұ n dҥ ng GPS module: Cung cҩ p khҧQăQJO ҩ y dӳliӋ u tӑDÿ ӝtӯvӋtinh 697+ĈһQJ7KDQK+jR - 16CNTT3 43 Earphone: gӗm tai nghe mic cung cҩ p khҧQăQJSKiW output bҵ ng âm nhұ n input bҵ ng giӑ ng nói phím bҩ m (button): nút bҩ Pÿ ӇchuyӇ n chӭFQăQJ VHQVRUÿRNKR ҧ ng cách: cung cҩ p khoҧ QJFiFKÿ Ӄ n bӅmһ t vұ t lý cho chӭFQăQJF ҧ nh báo Pin: Là nguӗ n cung cҩ p cho thiӃ t bӏhoҥ Wÿ ӝng Khung kính, dây dү n Hu nh 2.8 0{KuQKSK̯QFͱQJ.tQKWK{ 2.3.2 ThiӃ t kӃchӭFQăQJ &iFFKӭFQăQJWUrQ.tQKWK{QJPLQK - Nhұ n dҥ ng tên phân phân loҥ i vұ t thӇnguy hiӇ m - Cҧ nh báo tӵÿӝ ng vӅvұ t nguy hiӇ m - Cҧ nh báo tӵÿӝ ng vӅFKѭ ӟng ngҥ i vұ t - Phát âm tên cӫ a toàn bӝvұ t thӇWUѭ ӟc mһ t - 7uPÿ ӗvұ t bҵ ng giӑng nói 697+ĈһQJ7KDQK+jR - 16CNTT3 44 - Gӱi tín hiӋ u cҩ p cӭX626ÿ Ӄ n ӭng dө ng mobile - Gӱi dӳliӋ uҧ nh chө Sÿѭ ӧc tӑDÿ ӝGPS lҩ y tӯvӋtinh gӱLÿ Ӄ n ӭng dө ng mobile - Liên kӃ t vӟi ӭng dө ng mobile - KiӇ m tra cұ p nhұ t phiên bҧ n phҫ n mӅ m mӟi (nӃ u có) 2.3.2.1 ChӭFQăQJQK ұn dҥ ng tên phân loҥi vұt thӇnguy hiӇ m Hu nh 2.9 6˯ÿ͛FDV͵GͭQJ FKͱFQăQJ1 K̵QG̩QJWrQÿ͛Y̵W PKkQOR̩LY WK͋QJX\KL͋P B̫ ng 2.8 Ĉ̿FW̫FDVDV͵GͭQJFKͱFQă Mã Usecase UC01 Tên Usecase Nhұn dҥQJÿ ӗvұt Mơ tҧ Trích xuҩt hình ҧnh tӯcamera thӵc hiӋ n nhұn dҥQJFiFÿ ӗvұ t có hình ҧnh 1Jѭ ӡi thӵc hiӋ n NJѭ ӡi khiӃ m thӏ TiӅ QÿL Ӆ u kiӋ n Không Hұ XÿL Ӆ u kiӋ n /ѭXGDQKViFKFiFY ұ WÿmQK ұ n diӋ n &iFEѭ ӟc thӵc hiӋ n Tӵÿ ӝng Ngoҥ i lӋ(Exception) Không nhұ Qÿѭ ӧc hình ҧ nh tӯcamera 697+ĈһQJ7KDQK+jR - 16CNTT3 45 B̫ ng 2.9 Ĉ̿FW̫FDV͵GͭQJFKͱFQăQ J3KkQOR̩Lÿ͛Y̵W Mã Usecase UC02 Tên Usecase Phân loҥ i vұt thӇnguy hiӇ m Mô tҧ Ĉӑ c dӳliӋ u danh sách vұ WÿmQK ұn dҥng tӯchӭFQăQJQKkQG ҥQJÿ ӗvұt rӗ i kiӇ m tra tӯng phҫn tӱtrong danh sách có thuӝc danh sách vұt nguy hiӇ m hay không 1Jѭ ӡi thӵc hiӋ n 1Jѭ ӡi khiӃ m thӏ TiӅ QÿL Ӆ u kiӋ n Nhұn dҥQJWrQÿ ӗvұt Hұ XÿL Ӆ u kiӋ n /ѭXGDQKViFKFiFY ұ t nguy hiӇ PÿmSKiWKL Ӌ n &iFEѭ ӟc thӵc hiӋ n Tӵÿ ӝng Ngoҥ i lӋ(Exception) Không Hu nh 2.10 6˯ÿ͛WX̯Q W͹FKͱFQăQJ NK̵QG̩QJYj PKkQOR̩Lÿ͛Y 697+ĈһQJ7KDQK+jR - 16CNTT3 ... ӭQJGөQJWUtWXӋQKkQWҥRYj,QWHUQH ÿm[iFÿӏQKPөF tiêu: 697+ĈһQJ7KDQK+jR - 16CNTT3 - Xây dӵng hӋthӕ ng IoT Kính thơng minh, gӗm kính thông minh hӛtrӧ QJѭ ӡi khiӃ m thӏtrong viӋ c nhұ n dҥ ng, tìm kiӃ Pÿ ӗvұ... QOtQKyPJLDÿuQK - Liên kӃ t hӫ y liên kӃ t kính - Quan sát v? ?trí GPS ҧ nh chө p t? ?kính thơng minh ÿmOLrQN Ӄ t - Nhұ n thơng báo SOS t? ?kính thơng minh 697+ĈһQJ7KDQK+jR - 16CNTT3 4 Ĉӕ LWѭ... &Ѫ6 ӢLÝ THUYӂT 1.1 Trí tu? ?nhân tҥo máy hӑc 1.1.1 Trí tu? ?nhân tҥo (AI) 1.1.2 Máy hӑc (Machine Learning) 1.1.3 MҥQJQѫ -ron nhân tҥo (Artificial Neural Networks