Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)

Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)Nghiên cứu và phát triển các kỹ thuật định vị và định danh kết hợp thông tin hình ảnh và WiFi (LA tiến sĩ)

MINISTRY OF EDUCATION AND TRAINING HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY THUY PHAM THI THANH NGHIÊN CỨU VÀ PHÁT TRIỂN CÁC KỸ THUẬT ĐỊNH VỊ VÀ ĐỊNH DANH KẾT HỢP THÔNG TIN HÌNH ẢNH VÀ WIFI PERSON LOCALIZATION AND IDENTIFICATION BY FUSION OF VISION AND WIFI DOCTORAL THESIS OF COMPUTER SCIENCE Hanoi − 2017 MINISTRY OF EDUCATION AND TRAINING HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY THUY PHAM THI THANH NGHIÊN CỨU VÀ PHÁT TRIỂN CÁC KỸ THUẬT ĐỊNH VỊ VÀ ĐỊNH DANH KẾT HỢP THÔNG TIN HÌNH ẢNH VÀ WIFI PERSON LOCALIZATION AND IDENTIFICATION BY FUSION OF VISION AND WIFI Specialization: Computer Science Code No: 62480101 DOCTORAL THESIS OF COMPUTER SCIENCE SUPERVISORS: Assoc.Prof Thi Lan Le Dr Trung Kien Dao Hanoi − 2017 DECLARATION OF AUTHORSHIP I, Thuy Pham Thi Thanh, declare that this thesis titled, "Person Localization and Identification by Fusion of Vision and WiFi", and the work presented in it are my own I confirm that: This work was done wholly or mainly while in candidature for a PhD research degree at Hanoi University of Science and Technology Where any part of this thesis has previously been submitted for a degree or any other qualification at Hanoi University of Science and Technology or any other institution, this has been clearly stated Where I have consulted the published work of others, this is always clearly attributed Where I have quoted from the work of others, the source is always given With the exception of such quotations, this thesis is entirely my own work I have acknowledged all main sources of help Where the thesis is based on work done by myself jointly with others, I have made exactly what was done by others and what I have contributed myself Hanoi, January 2017 PhD STUDENT Thuy Pham Thi Thanh SUPERVISORS Assoc.Prof Thi Lan Le Dr Trung Kien Dao i ACKNOWLEDGEMENT This thesis is done at International Research Institute MICA, Hanoi University of Science and Technology First, I would like to express my sincere gratitude to my advisors Assoc.Prof Thi Lan Le and Dr Trung Kien Dao for the continuous support of my Ph.D study and related research, for their patience, motivation, and immense knowledge Their guidance helped me in all the time of research and writing of this thesis I could not have imagined having a better advisor and mentor for my Ph.D study Besides my advisors, I would like to thank the scientists, the authors of the published works which are cited in this thesis I am provided with valuable information resources from their works for my thesis In the process of implementation and completion of my research, I has received many supports from the board of MICA directors My sincere thanks go to Prof Yen Ngoc Pham , Prof Eric Castelli and Dr Son Viet Nguyen, who provided me with an opportunity to join researching works in MICA, and who gave access to the laboratory and research facilities Without their precious support would it have been impossible to conduct this research I would also like to thank board of directors of the University of Technology and Logistics where I worked I received financial support and time from my office and leaders for completing my doctoral thesis I gratefully acknowledge the financial support for publishing papers and conference fees from research projects B2013.01.48, and NAFOSTED 102.04-2013.32 I would like to thank my colleagues at Computer Vision Department and Pervasive Spaces and Interaction Department for their accompaniment during my research A special thanks to my family Words can not express how grateful I am to my mother and father for all of the sacrifices that they have made on my behalf I would also like to thank to my beloved husband and my sisters Thank you for supporting me for everything Hanoi, January 2017 PhD Student Thuy Pham Thi Thanh ii CONTENTS DECLARATION OF AUTHORSHIP i ACKNOWLEDGEMENT ii CONTENTS v SYMBOLS vi LIST OF TABLES x LIST OF FIGURES xvii LITERATURE REVIEW 1.1 WiFi-based localization 1.2 Vision-based person localization 1.2.1 Human detection 1.2.1.1 Motion-based detection 1.2.1.2 Classifier-based detection 1.2.2 Human tracking 1.2.3 Human localization 1.3 Person localization based on fusion of WiFi and visual properties 1.4 Vision-based person re-identification 1.5 Conclusion WIFI-BASED PERSON LOCALIZATION 2.1 Framework 2.2 Probabilistic propagation model 2.2.1 Parameter estimation 2.2.2 Reduction of Algorithm Complexity 2.3 Fingerprinting database and KNN matching 2.4 Experimental results 2.4.1 Testing environment and data collection 2.4.2 Experiments for propagation model 2.4.3 Localization experiments 2.4.3.1 Evaluation metrics 2.4.3.2 Experimental results 2.5 Conclusion iii 13 14 18 19 19 21 22 23 24 26 29 30 30 32 33 34 35 38 38 40 43 43 43 49 VISION-BASED PERSON LOCALIZATION 50 3.1 Introduction 50 3.2 Experimental datasets 53 3.3 Shadow Removal 56 3.3.1 Chromaticity-based feature extraction and shadow-matching score calculation 57 3.3.2 Shadow-matching score utilizing physical properties 60 3.3.3 Density-based score fusion scheme 62 3.3.4 Experimental Evaluation 65 3.4 Human detection 67 3.4.1 Fusion of background subtraction and HOG-SVM 67 3.4.2 Experimental evaluation 69 3.4.2.1 Dataset and evaluation metrics 69 3.4.2.2 Experimental results 70 3.5 Person tracking and localization 72 3.5.1 Kalman filter 72 3.5.2 Person tracking and data association 73 3.5.3 Person localization and linking trajectories in camera network 80 3.5.3.1 Person localization 80 3.5.3.2 Linking person’s trajectories in camera network 82 3.5.4 Experimental evaluation 84 3.5.4.1 Initial values 84 3.5.4.2 Evaluation metrics for person tracking in one camera FOV 85 3.5.4.3 Experimental results 87 3.6 Conclusion 90 PERSON IDENTIFICATION AND RE-IDENTIFICATION CAMERA NETWORK 4.1 Face recognition system 4.1.1 Framework 4.1.2 Experimental evaluation 4.1.2.1 Testing scenarios 4.1.2.2 Measurements 4.1.2.3 Testing data and results 4.2 Appearance-based person re-identification 4.2.1 Framework 4.2.2 Improved kernel descriptor for human appearance 4.2.3 Experimental results iv IN A 92 93 93 96 96 96 96 97 97 98 102 4.3 4.2.3.1 Testing datasets 102 4.2.3.2 Results and discussion 104 Conclusion 115 FUSION OF WIFI AND CAMERA FOR PERSON TION AND IDENTIFICATION 5.1 Fusion framework and algorithm 5.1.1 Framework 5.1.2 Fusion method 5.1.2.1 Kalman filter 5.1.2.2 Optimal Assignment 5.2 Dataset and Evaluation 5.2.1 Testing dataset 5.2.2 Experimental results 5.2.2.1 Experimental results on script data 5.2.2.2 Experimental results on script data 5.3 Conclusion LOCALIZA 117 118 118 120 121 123 124 124 128 128 129 132 PUBLICATIONS 137 BIBLIOGRAPHY 139 A 154 v ABBREVIATIONS TT Abbreviation Meaning AHPE Asymmetry based Histogram Plus Epitome AmI Ambient Intelligent ANN Artificial Neural Network AoA Angle of Arrival AP Access Point BB Bounding Box BGS Background Subtraction CCD Charge-Coupled Device DSM Direct Stein Method 10 EM Expectation Maximization 11 FAR False Acceptance Rate 12 FN False Negative 13 FOV Field of View 14 FP False Positive 15 fps f rame per second 16 JPDAF Joint Probability Data Association Filtering 17 GA Genetic Algorithm 18 GLOH Gradient Location and Orientation Histogram 19 GLONASS Global Navigation Satellite System 20 GMM Gaussian Mixture Model 21 GMOTA Global Multiple Object Tracking Accuracy 22 GNSS Global Navigation Satellite Systems 23 GPS Global Positioning System 24 HOG Histogram of Oriented Gradient 25 HSV Hue Saturation Value 26 ID Identity 27 IP Internet Protocol 28 KLT Kanade Lucas Tomasi 29 KNN K-Nearest Neighbors 30 LAP Linear Assignment Problem vi 31 LBP Local Binary Pattern 32 LBPH Local Binary Pattern Histogram 33 LDA Linear Discriminant Analysis 34 LMNR Large Margin Nearest Neighbor 35 LoB Line of Bearing 36 LOS Line of Sight 37 LR Large Region 38 MAC Media Access Control 39 MHT Multiple Hypothesis Tracking 40 MOTA Multiple Object Tracking Accuracy 41 MOG Mixture of Gaussian 42 MOTP Multiple Object Tracking Precision 43 MSCR Maximally Stable Colour Regions 44 NLoS None-Line-of-Sight 45 PCA Principal Component Analysis 46 PDF Probability Distribution Function 47 PLS Partial Least Squares 48 PNG Portable Network Graphics 49 PPM Probabilistic Propagation Model 50 RBF Radial Basis Function 51 RDC Relative Distance Comparison 52 Re-ID Re-Identification 53 RFID Radio Frequency Identification 54 RGB Red Green Blue 55 ROI Region of Interest 56 RSS Received Signal Strength 57 RSSI Received Signal Strength Indication 58 SD Shadow 59 SDALF Symmetry Driven Accumulation of Local Features 60 SIFT Scale Invariant Feature Transform 61 SKMGM Spatial Kinetic Mixture of Gaussian Model 62 SLAM Simultaneous Localization and Mapping 63 SMP Stable Marriage Problem 64 SPOT Structure Preserving Object Tracker 65 SR Small Region vii 66 STGMM Spatio Temporal Gaussian Mixture Model 67 STL Standard Template Library 68 SURF Speeded Up Robust Features 69 SVM Support Vector Machine 70 SVR Support Vector Regression 71 TAPPMOG Time Adaptive Per Pixel Mixtures of Gaussians 72 TDoA Time Difference of Arrival 73 TLGMM Two-Layer Gaussian Mixture Model 74 TN True Negative 75 ToA Time of Arrival 76 TP True Positive 77 UWB Ultra-Wideband 78 VOR VHF Omnidirectional Range 79 WLAN Wireless Local Area Network 80 WPS WiFi Positioning System viii [60] Freund Y., Iyer R., Schapire R.E., and Singer Y (2003) An efficient boosting algorithm for combining preferences The Journal of machine learning research, 4:pp 933–969 [61] Gaggioli A (2005) Optimal experience in ambient intelligence Ambient intelligence, 3543(5) [62] Garcia C.R., Quesada-Arencibia A., Cristobal T., Padron G., Perez R., and Alay´on F (2014) Using ambient intelligence to improve public transport accessibility In Ubiquitous Computing and Ambient Intelligence Personalisation and User Adapted Services, pp 17–20 Springer [63] Gauglitz S., H¨ollerer T., and Turk M (2011) Evaluation of interest point detectors and feature descriptors for visual tracking International journal of computer vision, 94(3):pp 335–360 [64] Geronimo D., Lopez A., Ponsa D., and Sappa A.D (2007) Haar wavelets and edge orientation histograms for on–board pedestrian detection In Pattern Recognition and Image Analysis, pp 418–425 Springer [65] Goldsmith A (2005) Wireless communications Cambridge university press [66] Guo L., Li L., Zhao Y., and Zhang M (2010) Study on pedestrian detection and tracking with monocular vision In Computer Technology and Development (ICCTD), 2010 2nd International Conference on, pp 466–470 IEEE [67] Gutierrez V., Izaguirre M., Peer J., Munoz L., Lopez D., and Sanchez M (2010) Ambient intelligence in intermodal transport services: A practical implementation in road logistics In Sensor Technologies and Applications (SENSORCOMM), 2010 Fourth International Conference on, pp 203–209 IEEE [68] Hamano A., Fujisaki K., Uchida S., and Shiku O (2013) Stable marriage algorithm for tracking intracellular objects In Computing and Networking (CANDAR), 2013 First International Symposium on, pp 305–307 IEEE [69] Harley P (1989) Short distance attenuation measurements at 900 mhz and 1.8 ghz using low antenna heights for microcells Selected Areas in Communications, IEEE Journal on, 7(1):pp 5–11 [70] Hartley R and Zisserman A (2003) Multiple view geometry in computer vision Cambridge university press [71] Haupt R.L and Haupt S.E (2004) Practical genetic algorithms John Wiley & Sons 144 [72] Hirzer M., Roth P.M., K¨ostinger M., and Bischof H (2012) Relaxed pairwise learned metric for person re-identification In Computer Vision–ECCV 2012 , pp 780–793 Springer [73] Hong S., Song J.B., Baek J.H., and Ryu J.K (2012) Visual odometry for outdoor environment using a downward-tilting camera and self-shadow removal algorithm In Control, Automation and Systems (ICCAS), 2012 12th International Conference on, pp 960–963 IEEE [74] Hsieh J.W., Hu W.F., Chang C.J., and Chen Y.S (2003) Shadow elimination for effective moving object detection by gaussian shadow modeling Image and Vision Computing, 21(6):pp 505–516 [75] Huang D., Shan C., Ardabilian M., Wang Y., and Chen L (2011) Local binary patterns and its application to facial image analysis: a survey Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 41(6):pp 765–781 [76] Huang J.B and Chen C.S (2009) Moving cast shadow detection using physicsbased features In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’09), pp 2310–2317 IEEE [77] Hwang S.Y and Song J.B (2011) Monocular vision-based slam in indoor environment using corner, lamp, and door features from upward-looking camera Industrial Electronics, IEEE Transactions on, 58(10):pp 4804–4812 [78] Đại học B ách Khoa H nội (2010) Nghiên cứu triển k hai phương p háp định vị người dùng nhà sử dụng cường độ tín hiệu wlan 802.1 [79] Đại học B ách Khoa H nội (2014) N ghiên cứu, ứ ng d ụng cntt x ây d ựng hệ thống giám sát, hỗ trợ chăm sóc bệnh nhân phòng điều trị thông minh [80] Jekabsons G and Zuravlyov V (2010) Refining wi-fi based indoor positioning In Proceedings of 4th International Scientific Conference Applied Information and Communication Technologies (AICT), Jelgava, Latvia, pp 87–95 [81] Johnson E., Olson E., and Boonthum-Denecke C (2012) Robot localization using overhead camera and leds In FLAIRS Conference Citeseer [82] Jones M.J and Viola P (2001) Robust real-time object detection In Workshop on statistical and computational theories of vision, volume 266, p 56 [83] Jungmin So Joo-Yub Lee C.H.Y.H.P (2013) An Improved Location Estimation Method for Wifi Fingerprint-based Indoor Localization International Journal of Software Engineering and Its Applications 145 [84] King T., Kopf S., Haenselmann T., Lubberger C., and Effelsberg W (2006) Compass: A probabilistic indoor positioning system based on 802.11 and digital compasses In Proceedings of the 1st international workshop on Wireless network testbeds, experimental evaluation & characterization, pp 34–40 ACM [85] Koestinger M., Hirzer M., Wohlhart P., Roth P.M., and Bischof H (2012) Large scale metric learning from equivalence constraints In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp 2288–2295 IEEE [86] Kothari N., Kannan B., Glasgwow E.D., and Dias M.B (2012) Robust indoor localization on a commercial smart phone Procedia computer science, 10:pp 1114–1120 [87] Kuhn H.W (1955) The hungarian method for the assignment problem Naval research logistics quarterly, 2(1-2):pp 83–97 [88] Laaraiedh M., Uguen B., Stephan J., Corre Y., Lostanlen Y., Raspopoulos M., and Stavrou S (2012) Ray tracing-based radio propagation modeling for indoor localization purposes In Computer Aided Modeling and Design of Communication Links and Networks (CAMAD), 2012 IEEE 17th International Workshop on, pp 276–280 IEEE [89] Lee B and Hedley M (2002) Background estimation for video surveillance In Image and vision computing New Zealand , pp 315–320 [90] Li Z., Chang S., Liang F., Huang T., Cao L., and Smith J (2013) Learning locally-adaptive decision functions for person verification In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3610–3617 [91] Liu N., Xiong R., Li Q., and Wang Y (2013) Human tracking using improved sample-based joint probabilistic data association filter In Intelligent Autonomous Systems 12 , pp 293–302 Springer [92] Liu Z., Huang K., Tan T., and Wang L (2007) Cast shadow removal combining local and global features In IEEE Conference on Computer Vision and Pattern Recognition, 2007 , pp 1–8 IEEE [93] Luo J and Zhan X (2014) Characterization of smart phone received signal strength indication for wlan indoor positioning accuracy improvement [94] Maji S., Berg A.C., and Malik J (2013) Efficient classification for additive kernel svms IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1):pp 66–77 146 [95] Marquardt D.W (1963) An algorithm for least-squares estimation of nonlinear parameters Journal of the society for Industrial and Applied Mathematics, 11(2):pp 431–441 [96] Martel-Brisson N and Zaccarin A (2007) Learning and removing cast shadows through a multidistribution approach IEEE Transactions on Pattern Analysis and Machine Intelligence, 29:p 11331146 [97] Martel-Brisson N and Zaccarin A (2008) Kernel-based learning of cast shadows from a physical model of light sources and surfaces for low-level segmentation In IEEE Conference on Computer Vision and Pattern Recognition, 2008 IEEE [98] Martin E., Vinyals O., Friedland G., and Bajcsy R (2010) Precise indoor localization using smart phones In Proceedings of the international conference on Multimedia, pp 787–790 ACM [99] Martinel N., Micheloni C., and Piciarelli C (2012) Distributed signature fusion for person re-identification In 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC), pp 1–6 IEEE [100] Martinel N., Micheloni C., and Piciarelli C (2013) Learning pairwise feature dissimilarities for person re-identification In 2013 Seventh International Conference on Distributed Smart Cameras (ICDSC), pp 1–6 IEEE [101] Mautz R (2012) Indoor positioning technologies Ph.D thesis, Habilitationsschrift ETH Z¨ urich, 2012 [102] Maybeck P.S (1990) The kalman filter: An introduction to concepts In Autonomous robot vehicles, pp 194–204 Springer [103] Mignon A and Jurie F (2012) Pcca: A new approach for distance learning from sparse pairwise constraints In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp 2666–2672 IEEE [104] Mirowski P., Ho T.K., Yi S., and MacDonald M (2013) Signalslam: Simultaneous localization and mapping with mixed wifi, bluetooth, lte and magnetic signals In Indoor Positioning and Indoor Navigation (IPIN), 2013 International Conference on, pp 1–10 IEEE [105] Mirowski P., Milioris D., Whiting P., and Kam Ho T (2014) Probabilistic radiofrequency fingerprinting and localization on the run Bell Labs Technical Journal , 18(4):pp 111–133 147 [106] Miyaki T., Yamasaki T., and Aizawa K (2007) Visual tracking of pedestrians jointly using wi-fi location system on distributed camera network In Multimedia and Expo, 2007 IEEE International Conference on, pp 1762–1765 IEEE [107] Monteiro G., Peixoto P., and Nunes U (2006) Vision-based pedestrian detection using haar-like features Robotica, 24:pp 46–50 [108] Munoz D., Lara F.B., Vargas C., and Enriquez-Caldera R (2009) Position location techniques and applications Academic Press [109] Nandakumar K., Chen Y., Dass S.C., and Jain A (2008) Likelihood ratio-based biometric score fusion IEEE transactions on pattern analysis and machine intelligence, 30(2):pp 342–347 [110] Ngô V.T and Nguyễn n.P (2011) Thuật toán định vị lập kế hoạch đường toàn cục cho robot tự hành [111] Nguyen N.B., Nguyen V.H., Duc T.N., Le D.D., and Duong D.A (2015) Attrel: an approach to person re-identification by exploiting attribute relationships In International Conference on Multimedia Modeling, pp 50–60 Springer [112] Nguyen V.H., Ngo T.D., Nguyen K.M., Duong D.A., Nguyen K., and Le D.D (2013) Re-ranking for person re-identification [113] NGUYEN V.T., LE T.L., TRAN T.H., MULLOT R., and COURBOULAY V (2015) A new hand representation based on kernels for hand posture recognition In The Eleventh IEEE International Conference on Automatic Face and Gesture Recognition (FG 2015) [114] Niculescu D and Nath B (2004) Vor base stations for indoor 802.11 positioning In Proceedings of the 10th annual international conference on Mobile computing and networking, pp 58–69 ACM [115] Nourani-Vatani N., Borges P.V., and Roberts J.M (2012) A study of feature extraction algorithms for optical flow tracking In Proc Australian Conf Robotics and Automation [116] O’Donoghue J and Herbert J (2006) An intelligent data management reasoning model within a pervasive medical environment In Proceedings of the 1st Workshop on Artificial Intelligence Techniques for Ambient Intelligence (AITAmI’06), pp 27–31 [117] Pahlavan K and Levesque A.H (2005) Wireless information networks, volume 93 John Wiley & Sons 148 [118] Pajovic M., Orlik P., Koike-Akino T., Kim K., Aikawa H., and Hori T (2015) An unsupervised indoor localization method based on received signal strength rss measurements [119] Pedagadi S., Orwell J., Velastin S., and Boghossian B (2013) Local fisher discriminant analysis for pedestrian re-identification In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3318–3325 [120] Pers J (2012) Mvl lab5: Multi-modal indoor person localization dataset [121] Pfeifer T (2005) Redundant positioning architecture Computer Communications, 28(13):pp 1575–1585 [122] Phạm H.H and Chu B.T (2011) Kỹ thuật định vị dựa wifi ứng dụng [123] Prati A and Cucchiara R (2008) Video analysis for ambient intelligence in urban environments Intelligent Env.: Methods, Algorithms and Applications, p 143 [124] Prati A., Mikic I., Trivedi M.M., and Cucchiara R (2003) Detecting moving shadows: algorithms and evaluation IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(7):pp 918–923 [125] Priyantha N.B., Chakraborty A., and Balakrishnan H (2000) The cricket location-support system In Proceedings of the 6th annual international conference on Mobile computing and networking, pp 32–43 ACM [126] Prosser B., Zheng W.S., Gong S., Xiang T., and Mary Q (2010) Person reidentification by support vector ranking In The British Machine Vision Conference (BMVC), volume 2, p [127] Reid D.B (1979) An algorithm for tracking multiple targets Automatic Control, IEEE Transactions on, 24(6):pp 843–854 [128] Rekimoto J., Shionozaki A., Sueyoshi T., and Miyaki T (2006) Placeengine: a wifi location platform based on realworld folksonomy In Internet conference, volume 2006, pp 95–104 [129] Roberts B and Pahlavan K (2009) Site-specific rss signature modeling for wifi localization In Global Telecommunications Conference, 2009 GLOBECOM 2009 IEEE , pp 1–6 IEEE [130] Roos T., Myllym¨aki P., Tirri H., Misikangas P., and Siev¨anen J (2002) A probabilistic approach to wlan user location estimation International Journal of Wireless Information Networks, 9(3):pp 155–164 149 [131] Ruiz-Ruiz A.J., Canovas O., Munoz R.A.R., and Alcolea P.E.L.d.T (2011) Burkard1999 In Mobile and Ubiquitous Systems: Computing, Networking, and Services, pp 37–48 Springer [132] Sabzmeydani P and Mori G (2007) Detecting pedestrians by learning shapelet features In Computer Vision and Pattern Recognition, 2007 CVPR’07 IEEE Conference on, pp 1–8 IEEE [133] Sanin A., Sanderson C., and Lovell B.C (2010) Improved shadow removal for robust person tracking in surveillance scenarios In 20th International Conference on Pattern Recognition (ICPR), pp 141–144 IEEE [134] Sanin A., Sanderson C., and Lovell B.C (2013) Shadow detection: A survey and comparative evaluation of recent methods Pattern recognition, 45(4):pp 1684–1695 [135] Santana-Mancilla P.C., Echeverr´ıa M.A.M., Santos J.C.R., Castellanos J.A.N., and D´ıaz A.P.S (2013) Towards smart education: Ambient intelligence in the mexican classrooms Procedia-Social and Behavioral Sciences, 106:pp 3141–3148 [136] Schiele B., Andriluka M., Majer N., Roth S., and Wojek C (2009) Visual people detection: Different models, comparison and discussion In IEEE International Conference on Robotics and Automation [137] Schneider N and Gavrila D.M (2013) Pedestrian path prediction with recursive bayesian filters: A comparative study In Pattern Recognition, pp 174–183 Springer [138] Schwartz W.R and Davis L.S (2009) Learning discriminative appearance-based models using partial least squares In 2009 XXII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI), pp 322–329 IEEE [139] Shim J.H and Cho Y.I (2016) A mobile robot localization via indoor fixed remote surveillance cameras Sensors, 16(2):p 195 [140] Sigari M.H., Mozayani N., and Pourreza H (2008) Fuzzy running average and fuzzy background subtraction: concepts and application International Journal of Computer Science and Network Security, 8(2):pp 138–143 [141] Simonnet D., Lewandowski M., Velastin S.A., Orwell J., and Turkbeyler E (2012) Re-identification of pedestrians in crowds using dynamic time warping In The 10th European Conference on Computer Vision Computer Vision (ECCV 2012), pp 423–432 Springer 150 [142] Stauffer C and Grimson W.E.L (1999) Adaptive background mixture models for real-time tracking In Computer Vision and Pattern Recognition, 1999 IEEE Computer Society Conference on., volume IEEE [143] Stauffer C and Grimson W.E.L (2000) Learning patterns of activity using realtime tracking Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(8):pp 747–757 [144] S¨ underhauf N and Protzel P (2007) Stereo odometry—a review of approaches Chemnitz University of Technology Technical Report [145] Taiana M., Nascimento J.C., and Bernardino A (2013) An improved labelling for the inria person data set for pedestrian detection In Pattern Recognition and Image Analysis, pp 286–295 Springer [146] Tchango A.F., Thomas V., Buffet O., Dutech A., and Flacher F (2014) Tracking multiple interacting targets using a joint probabilistic data association filter In Information Fusion (FUSION), 2014 17th International Conference on, pp 1–8 IEEE [147] Teixeira L.F and Corte-Real L (2009) Video object matching across multiple independent views using local descriptors and adaptive learning Pattern Recognition Letters, 30(2):pp 157–167 [148] Teixeira T., Dublon G., and Savvides A (2010) A survey of human-sensing: Methods for detecting presence, count, location, track, and identity ACM Computing Surveys, 5:pp 1–77 [149] Tian Y., Feris R.S., Liu H., Hampapur A., and Sun M.T (2011) Robust detection of abandoned and removed objects in complex surveillance videos Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 41(5):pp 565–576 [150] Tran Q., Vo B., Dinh T., and Duong D (2011) Automatic player detection, tracking and mapping to field model for broadcast soccer videos [151] Trần T.V., Trần C.C., Huỳnh C.T., Nguyễn H.N., Đỗ N.T., and Trần H (2011) Một kỹ thuật phát hiện, bám sát đối tượng ứng dụng [152] Tsai C.Y., Chou S.Y., Lin S.W., and Wang W.H (2009) Location determination of mobile devices for an indoor wlan application using a neural network Knowledge and information systems, 20(1):pp 81–93 151 [153] Tsai R.Y (1986) An efficient and accurate camera calibration technique for 3d machine vision In Proc IEEE Conf on Computer Vision and Pattern Recognition, 1986 [154] Van den Berghe S., Weyn M., Spruyt V., and Ledda A (2011) Combining wireless and visual tracking for an indoor environment In International Conference on Indoor Positioning and Indoor Navigation (IPIN-2011) [155] Van den Berghe S., Weyn M., Spruyt V., and Ledda A (2011) Fusing camera and wi-fi sensors for opportunistic localization Proc UBICOMM , pp 169–174 [156] Vorst P., Koch A., and Zell A (2011) Efficient self-adjusting, similarity-based location fingerprinting with passive uhf rfid In RFID-Technologies and Applications (RFID-TA), 2011 IEEE International Conference on, pp 160–167 IEEE [157] Wang T., Gong S., Zhu X., and Wang S (2014) Person re-identification by video ranking In Computer Vision–ECCV 2014 , pp 688–703 Springer [158] Wang X., Han T.X., and Yan S (2009) An hog-lbp human detector with partial occlusion handling In Computer Vision, 2009 IEEE 12th International Conference on, pp 32–39 IEEE [159] Welch G and Bishop G (2004) An introduction to the kalman filter Technical report, TR 95-041 [160] Welch G., Bishop G., Vicci L., Brumback S., Keller K., and Colucci D (2001) High-performance wide-area optical tracking: The hiball tracking system presence: teleoperators and virtual environments, 10(1):pp 1–21 [161] Weyn M (2011) Opportunstic Seamless Localization Lulu com [162] Wojek C and Schiele B (2008) A performance evaluation of single and multifeature people detection In Pattern Recognition, pp 82–91 Springer [163] Wojek C., Walk S., and Schiele B (2009) Multi-cue onboard pedestrian detection In Computer Vision and Pattern Recognition, 2009 CVPR 2009 IEEE Conference on, pp 794–801 IEEE [164] Wood L.A., Buscher M., and Ramirez L (2012) Response to emergence in emergency response [165] Youssef M., Youssef A., Rieger C., Shankar U., and Agrawala A (2006) Pinpoint: An asynchronous time-based location determination system In Proceedings of the 4th international conference on Mobile systems, applications and services, pp 165–176 ACM 152 [166] Zhang L and Maaten L (2013) Structure preserving object tracking In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1838–1845 [167] Zhang S., Staudt E., Faltemier T., and Roy-Chowdhury A.K (2015) A camera network tracking (camnet) dataset and performance baseline In 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp 365–372 IEEE [168] Zhang Z (2000) A flexible new technique for camera calibration Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(11):pp 1330–1334 [169] Zheng J., Wang Y., Nihan N., and Hallenbeck M (2006) Extracting roadway background image: Mode-based approach Transportation Research Record: Journal of the Transportation Research Board , (1944):pp 82–88 [170] Zheng W.S., Gong S., and Xiang T (2011) Person re-identification by probabilistic relative distance comparison In 2011 IEEE conference on Computer vision and pattern recognition (CVPR), pp 649–656 IEEE [171] Zheng W.S., Gong S., and Xiang T (2013) Reidentification by relative distance comparison IEEE Transactions on attern Analysis and Machine Intelligence, 35(3):pp 653–668 [172] Zivkovic Z (2004) Improved adaptive gaussian mixture model for background subtraction In Proceedings of the 17th International Conference on Pattern Recognition (ICPR), volume 2, pp 28–31 IEEE [173] Zou D., Meng W., Han S., Gong Z., and Yu B (2013) User aided self-growing approach on radio map construction for wlan based localization In Proceedings of the 26th International Technical Meeting of the Satellite Division of The Institute of Navigation 153 APPENDIX A Table A.1 Technical information of WiFi-based localization system Parameters Values/Types Meaning Scanning period (s) Period for mobile device to scan signal strengths from nearby APs Velocity 1-1.3 m/s Velocity of pedestrian in the testing environment 2000 (locations) Number of fingerprint locations for the first testing scenario 1200 (locations) Number of fingerprint locations for the second testing scenario 672 (measures) Number of measurements carried out for the training data of tuning system parameters in the first testing scenario 467 (measures) Number of measurements carried out for the training data of tuning system parameters in the second testing scenario Fingerprint locations WiFi signal strength measurements The parameter configuration for training process using Genetic Algorithm Genetic algorithm configuration Population size 20 Elite count Crossover fraction 0.5 Time limit No 154 Maximal generations No Tolerance 10−6 Selection Uniform Crossover Scattered Mutation Uniform Creation population Uniform Optimal parameters are produced by using the configuration data of Genetic Algorithm for the first and the second testing scenario, respectively Optimal parameters P0 -41 dBm P0 is known signal power at a reference distance r0 in dBm -36.1757 dBm n n is the path-loss exponent indicating the rate at which the path loss increases with the distance 1.1 2.2029 kσ 1.0035m−1 kσ is a constant, which also needs to be determined from experiment data 5.3147m−1 r0 (m) 2.5117 (m) kd 49.23 dB m·m−1 5.1311 dB m·m−1 kd is an attenuation factor per wall/floor thickness unit σ is standard deviation which is also a function of P KNN matching The number of the nearest neighbors of a location z θ 1.1 θ and λ are constants used to define the curve of exponential functions in the weight function λ 0.000002 k 155 Table A.2 Technical information of vision-based localization system Parameters Frame rate Values/Types 20 fps Meaning Camera frame rate Image resolution 640x480 pixels Image resolution for each frame captured from cameras 25x80 pixels Minimum scale of human in human detection 120x300 pixels Maximum scale of human in human detection The number of Gaussian components selected for shadow-matching score calculation based on chromaticity and physical features This figure is also chosen for shadow set in density-based score fusion scheme The number of Gaussian components selected for nonshadow set in density-based score fusion scheme 0.015 A fixed learning rate of GMM background subtraction 50% Jaccard similarity coefficient between the areas of the ground-truth bounding box BBgt and the detected bounding box BBdt If it is greater than 50% then a potential match between them is found, otherwise unmatched BBdt counted as false positive and unmatched BBgt as false negative ±10 pixels It is used to decide whether a new track appear or existing track will be removed from the monitoring region of a camera The controlled points in this case are defined as all points near the contour line of the defined threshold region K Gaussian components Learning rate η Jaccard index J The deviation of the defined threshold region 156 Standard deviation σ in both x and y directions of the image coordination system Distance threshold T ±10 pixels The Footpoint position deviation from real position, which is indicated in initial covariance matrix of Kalman filter ±5 pixels The velocity deviation shown in initial covariance matrix of Kalman filter ±5 pixels The Footpoint position deviation from real position, which is indicated in state noise covariance matrix of Kalman filter ±2 pixels The velocity deviation shown in state noise covariance matrix of Kalman filter ±3 pixels The Footpoint position deviation of measurement shown in measurement noise covariance matrix of Kalman filter 20 cm, 50 cm, 100 cm Distance thresholds are given to decide whether or not assign a target in ground truth data to an output of tracker Table A.3 Technical information of fusion-based localization system Parameters Values/Types Meaning Standard deviation in both X and Y directions of the real world coordinate system ±5 pixels The Footpoint position deviation from real position, which is shown in initial covariance matrix and state noise covariance matrix of Kalman filter ±3 pixels The velocity deviation indicated in initial covariance matrix and state noise covariance matrix of Kalman filter 157 ±3 pixels Frequency of location measurements η Velocity coefficient ∆T Velocity of pedestrian The average time deviation Deviation of Wifi position labeling n=1 time/second ∆T = The deviation for FootPoint measurement shown in measurement noise covariance matrix of Kalman filter The position is measured n times per second Velocity coefficient shown in the state equations of Kalman filter n 1-1.3 m/second The moving speed of pedestrians in the environment This shows the normal velocity of a pedestrian seconds The average time deviation between two consecutive samples of wifi signals cm/m The deviation per one meter in dataset labeling of wifi positions, in compared with the truth positions 158 ... THUY PHAM THI THANH NGHIÊN CỨU VÀ PHÁT TRIỂN CÁC KỸ THUẬT ĐỊNH VỊ VÀ ĐỊNH DANH KẾT HỢP THÔNG TIN HÌNH ẢNH VÀ WIFI PERSON LOCALIZATION AND IDENTIFICATION BY FUSION OF VISION AND WIFI Specialization:... from the distinctive sensors like WiFi and visual sensors ❼ Signal synchronization between different sensors of WiFi and camera This is a necessary step before testing and evaluating any fusing... environments by using WiFi and camera systems For this, the concrete objectives are: ❼ Constructing an improved method for WiFi- based localization The method al- lows to to use popular WiFi- enable devices,