Person reidentification in a surveiliance camera network743

MINISTRY OF EDUCATION AND TRAINING HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY NGUYEN THUY BINH PERSON RE-IDENTIFICATION IN A SURVEILLANCE CAMERA NETWORK DOCTORAL DISSERTATION OF ELECTRONICS ENGINEERING Hanoi−2021 MINISTRY OF EDUCATION AND TRAINING HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY NGUYEN THUY BINH PERSON RE-IDENTIFICATION IN A SURVEILLANCE CAMERA NETWORK Major: Electronics Engineering Code: 9520203 DOCTORAL DISSERTATION OF ELECTRONICS ENGINEERING SUPERVISORS: 1.Assoc Prof Pham Ngoc Nam 2.Assoc Prof Le Thi Lan Hanoi−2021 DECLARATION OF AUTHORSHIP I, Nguyen Thuy Binh, declare that the thesis titled "Person re-identification in a surveillance camera network" has been entirely composed by myself I assure some points as follows: This work was done wholly or mainly while in candidature for a Ph.D research degree at Hanoi University of Science and Technology The work has not been submitted for any other degree or qualifications at Hanoi University of Science and Technology or any other institutions Appropriate acknowledgement has been given within this thesis where reference has been made to the published work of others The thesis submitted is my own, except where work in the collaboration has been included The collaborative contributions have been clearly indicated Hanoi, January 20, 2021 PhD Student Nguyen Thuy Binh SUPERVISORS Assoc.Prof Pham Ngoc Nam Assoc.Prof Le Thi Lan i ACKNOWLEDGEMENT This dissertation was written during my doctoral course at School of Electronics and Telecommunications (SET) and International Research Institute of Multimedia, Information, Communication and Applications (MICA), Hanoi University of Science and Technology (HUST) I am so grateful for all people who always support and encourage me for completing this study First, I would like to express my sincere gratitude to my advisors Assoc Prof Pham Ngoc Nam and Assoc Prof Le Thi Lan for their effective guidance, their patience, continuous support and encouragement, and their immense knowledge I would like to express my gratitude to Dr Vo Le Cuong and Dr Ha thi Thu Lan for their help I would like to thank all member of School of Electronics and Telecommunications, International Research Institute of Multimedia, Information, Communications and Applications (MICA), Hanoi University of Science and Technology (HUST) as well as all of my colleagues in Faculty of Electrical-Electronic Engineering, University of Transport and Communications (UTC) They have always helped me with the research process and given helpful advice for me to overcome my own difficulties Moreover, the attention at scientific conferences has always been a great experience for me to receive many the useful comments During my Ph.D course, I have received many supports from the Management Board of School of Electronics and Telecommunications, MICA Institute, and Faculty of Electrical-Electronic Engineering My sincere thank to Assoc Prof Nguyen Huu Thanh, Dr Nguyen Viet Son and Assoc Prof Nguyen Thanh Hai for giving me a lot of support and help Without their precious support, it has been impossible to conduct this research Thanks to my employer, University of Transport and Communications (UTC) for all necessary support and encouragement during my Ph.D journey I am also grateful to Vietnam’s Program 911, HUST and UTC projects for their generous financial support Special thanks to my family and relatives, particularly, my beloved husband and our children, for their never-ending support and sacrifice Hanoi, 2021 Ph.D Student ii CONTENTS DECLARATION OF AUTHORSHIP i ACKNOWLEDGEMENT ii CONTENTS vi SYMBOLS vi SYMBOLS ix LIST OF TABLES xii LIST OF FIGURES xvi INTRODUCTION CHAPTER LITERATURE REVIEW 1.1 Person ReID classifications 1.1.1 Single-shot versus Multi-shot 1.1.2 Closed-set versus Open-set person ReID 1.1.3 Supervised and unsupervised person ReID 10 1.2 Datasets and evaluation metrics 11 1.2.1 Datasets 11 1.2.2 Evaluation metrics 16 1.3 Feature extraction 16 1.3.1 Hand-designed features 17 1.3.2 Deep-learned features 20 1.4 Metric learning and person matching 25 1.4.1 Metric learning 25 1.4.2 Person matching 28 1.5 Fusion schemes for person ReID 29 1.6 Representative frame selection 31 1.7 Fully automated person ReID systems 33 1.8 Research on person ReID in Vietnam 34 CHAPTER MULTI-SHOT PERSON RE-ID THROUGH REPRESENTATIVE FRAMES SELECTION AND TEMPORAL FEATURE POOLING 36 2.1 Introduction iii 36 2.2 Proposed method 36 2.2.1 Overall framework 36 2.2.2 Representative image selection 37 2.2.3 Image-level feature extraction 44 2.2.4 Temporal feature pooling 49 2.2.5 Person matching 50 2.3 Experimental results 55 2.3.1 Evaluation of representative frame extraction and temporal feature pooling schemes 55 2.3.2 Quantitative evaluation of the trade-off between the accuracy and computational time 61 2.3.3 Comparison with state-of-the-art methods 63 2.4 Conclusions and Future work 65 CHAPTER PERSON RE-ID PERFORMANCE IMPROVEMENT BASED ON FUSION SCHEMES 67 3.1 Introduction 67 3.2 Fusion schemes for the first setting of person ReID 69 3.2.1 Image-to-images person ReID 3.2.2 Images-to-images person ReID 69 75 3.2.3 Obtained results on the first setting 76 3.3 Fusion schemes for the second setting of person ReID 82 3.3.1 The proposed method 82 3.3.2 Obtained results on the second setting 86 3.4 Conclusions 89 CHAPTER QUANTITATIVE EVALUATION OF AN END-TO-END PERSON REID PIPELINE 91 4.1 Introduction 4.2 An end-to-end person ReID pipeline 91 92 4.2.1 Pedestrian detection 92 4.2.2 Pedestrian tracking 97 4.2.3 Person ReID 98 4.3 GOG descriptor re-implementation 99 4.3.1 Comparison the performance of two implementations 99 4.3.2 Analyze the effect of GOG parameters 99 iv 4.4 Evaluation performance of an end-to-end person ReID pipeline 101 4.4.1 The effect of human detection and segmentation on person ReID in singleshot scenario 102 4.4.2 The effect of human detection and segmentation on person ReID in multishot scenario 104 4.5 Conclusions and Future work PUBLICATIONS 112 Bibliography 113 v 107 ABBREVIATIONS No Abbreviation Meaning ACF Aggregate Channel Features AIT Austrian Institute of Technology AMOC Accumulative Motion Context BOW Bag of Words CAR Learning Compact Appearance Representation CIE The International Commission on Illumination CFFM Comprehensive Feature Fusion Mechanism CMC Cummulative Matching Characteristic CNN Convolutional Neural Network 10 CPM Convolutional Pose Machines 11 CVPDL Cross-view Projective Dictionary Learning 12 CVPR Conference on Computer V ision and Pattern Recognition 13 DDLM Discriminative Dictionary Learning Method 14 DDN Deep Decompositional Network 15 DeepSORT Deep learning Simple Online and Realtime Tracking 16 DFGP Deep Feature Guided Pooling 17 DGM Dynamic Graph Matching 18 DPM Deformable Part-Based Model 19 ECCV European Conference on Computer V ision 20 FAST 3D Fast Adaptive Spatio-Temporal 3D 21 FEP Flow Energy Profile 22 FNN Feature Fusion Network 23 FPNN Filter Pairing Neural Network 24 GOG Gaussian of Gaussian 25 GRU Gated Recurrent Unit 26 HOG Histogram of Oriented Gradients 27 HUST Hanoi University of Science and Technology 28 IBP Indian Buffet Process 29 ICCV International Conference on Computer V ision 30 ICIP International Conference on Image Processing vi 31 IDE ID-Discriminative Embedding 32 iLIDS-VID Imagery Library for Intelligent Detection Systems 33 ILSVRC ImageNet Large Scale V isual Recognition Competition 34 ISR TIterative Spare Ranking 35 KCF Kernelized Correlation Filter 36 KDES Kenel DEScriptor 37 KISSME Keep It Simple and Straightforward MEtric 38 kNN k-Nearest Neighbour 39 KXQDA Kernel Cross-view Quadratic Discriminative Analysis 40 LADF Locally -Adaptive Decision Functions 41 LBP Local Binary Pattern 42 LDA LinearDiscriminantAnalysis 43 LDFV Local Descriptor and coded by Feature Vector 44 LMNN Large Margin Nearest Neighbor 45 LMNN-R Large Margin Nearest Neighbor with Rejection 46 LOMO LOcal Maximal Occurrence 47 LSTM Long-Short Term Memory 48 LSTMC Long Short-Term Memory network with a Coupled gate 49 mAP mean Average Precision 50 MAPR Multimedia Analysis and Pattern Recognition 51 Mask R-CNN Mask Region with CNN 52 MCT Multi -Camera Tracking 53 MCCNN Multi-Channel CNN 54 MCML Maximally Collapsing Metric Learning 55 MGCAM Mask-Guided Contrastive Attention Model 56 ML Machine Learning 57 MLAPG Metric Learning by Accelerated Proximal Gradient 58 MLR Metric Learning to Rank 59 MOT Multiple Object Tracking 60 MSCR Maximal Stable Color Region 61 MSVF Maximally Stable Video Frame 62 MTMCT Multi-Target Multi-Camera Tracking 63 Person ReID Person Re -Identification 64 Pedparsing Pedestrian Parsing 65 PPN Pose Prediction Network vii 66 PRW Person Re-identification in the W ild 67 QDA Quadratic Discriminative Analysis 68 RAiD Re-Identification Across indoor-outdoor Dataset 69 RAP Richly Annotated Pedestrian 70 ResNet Residual Neural Network 71 RHSP Recurrent High-Structured Patches 72 RKHS Reproducing Kernel Hilbert Space 73 RNN Recurrent Neural Network 74 ROIs Region of Interests 75 SDALF Symmetry Driven Accumulation of Local Feature 76 SCNCD Salient Color Names based Color Descriptor 77 SCNN Siamese Convolutional Neural Network 78 SIFT Scale-Invariant Feature Transform 79 SILTP Scale Invariant Local Ternary Pattern 80 SPD Symmetric Positive Definite 81 SMP Stepwise Metric Promotion 82 SORT Simple Online and Realtime Tracking 83 SPIC Signal Processing: Image Communication 84 SVM Support V ector Machine 85 TAPR Temporally Aligned Pooling Representation 86 TAUDL Tracklet Association Unsupervised Deep Learning 87 TCSVT Transactions on Circuits and Systems for V ideo Technology 88 TII Transactions on Industrial Informatics 89 TPAMI Transactions on Pattern Analysis and Machine Intelligence 90 TPDL Top-push Distance Learning 91 Two-stream MR Two-stream Multirate Recurrent Neural Network 92 UIT University of Information Technology 93 UTAL Tracklet Association Unsupervised Deep Learning 94 VIPeR V iew-point Invariant Pedestrian Recognition 95 VNU-HCM Vietnam National University - Ho Chi Minh City 96 WH Weighted color Histogram 97 WHOS Weighted Histograms of Overlapping Stripes 98 WSC W eight-based Sparse Coding 99 XQDA Cross-view Quadratic Discriminative Analysis 100 YOLO Y ou Only Look Once viii Bibliography [1] Gong S., Cristani M., Loy C.C., and Hospedales T.M (2014) The re- identification challenge In Person re-identification , pp 1–20 Springer [2] Wang X (2013) Intelligent multi-camera video surveillance: A review Pattern recognition letters, 34(1):pp 3–19 [3] Gheissari N., Sebastian T.B., and Hartley R (2006) Person reidentification using spatiotemporal appearance In Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, volume 2, pp 1528–1535 IEEE [4] Bedagkar-Gala A and Shah S.K (2014) A survey of approaches and trends in person re-identification Image and Vision Computing , 32(4):pp 270–286 [5] Vezzani R., Baltieri D., and Cucchiara R (2013) People reidentification in surveillance and forensics: A survey ACM Computing Surveys (CSUR), 46(2):p 29 [6] Satta R (2013) Appearance descriptors for person re-identification: a comprehensive review arXiv preprint arXiv:1307.5748 [7] Gou M., Wu Z., Rates-Borras A., Camps O., Radke R.J., et al (2018) A systematic evaluation and benchmark for person re-identification: Features, metrics, and datasets IEEE transactions on pattern analysis and machine intelligence , 41(3):pp 523–536 [8] Leng Q., Ye M., and Tian Q (2019) A survey of open-world person re- identification IEEE Transactions on Circuits and Systems for Video Technology [9] Zheng L., Yang Y., and Hauptmann A.G (2016) Person re-identification: Past, present and future arXiv preprint arXiv:1610.02984 [10] Perronnin F and Dance C (2007) Fisher kernels on visual vocabularies for image categorization In 2007 IEEE conference on computer vision and pattern recognition, pp 1–8 IEEE [11] Chang Y.C., Chiang C.K., and Lai S.H (2012) Single-shot person re- identification based on improved random-walk pedestrian segmentation In Intelligent Signal Processing and Communications Systems (ISPACS), 2012 International Symposium on, pp 1–6 IEEE 113 [12] Wei Y.L and Lin C.H (2013) Single-shot person re-identification by gaussian mixture model of weighted color histograms In Intelligent Signal Processing and Communications Systems (ISPACS), 2013 International Symposium on, pp 47– 50 IEEE [13] Li W., Wu Y., Mukunoki M., and Minoh M (2013) Coupled metric learning for single-shot versus single-shot person reidentification Optical Engineering , 52(2):p 027203 [14] Farenzena M., Bazzani L., Perina A., Murino V., and Cristani M (2010) Person re-identification by symmetry-driven accumulation of local features In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pp 2360– 2367 IEEE [15] Bazzani L., Cristani M., Perina A., Farenzena M., and Murino V (2010) Multiple-shot person re-identification by hpe signature In Pattern Recognition (ICPR), 2010 20th International Conference on, pp 1413–1416 IEEE [16] Zheng W.S., Gong S., and Xiang T (2012) Transfer re-identification: From person to set-based verification In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp 2650–2657 IEEE [17] Cancela B., Hospedales T.M., and Gong S (2014) identification by multi-label assignment inference Open-world person re- [18] Zheng W.S., Gong S., and Xiang T (2015) Towards open-world person reidentification by one-shot group-based verification IEEE transactions on pattern analysis and machine intelligence, 38(3):pp 591–606 [19] Liao S., Mo Z., Zhu J., Hu Y., and Li S.Z (2014) Open-set person re- identification arXiv preprint arXiv:1408.0872 [20] Wang H., Zhu X., Xiang T., and Gong S (2016) Towards unsupervised openset person re-identification In 2016 IEEE International Conference on Image Processing (ICIP), pp 769–773 IEEE [21] Chen Y., Zhu X., and Gong S (2018) Deep association learning for unsupervised video person re-identification arXiv preprint arXiv:1808.07301 [22] Ye M., Ma A.J., Zheng L., Li J., and Yuen P.C (2017) Dynamic label graph matching for unsupervised video re-identification In Proceedings of the IEEE International Conference on Computer Vision, pp 5142–5150 [23] Ma X., Zhu X., Gong S., Xie X., Hu J., Lam K.M., and Zhong Y (2017) Person re-identification by unsupervised video matching Pattern Recognition , 65:pp 197–210 114 [24] Liu Z., Wang D., and Lu H (2017) Stepwise metric promotion for unsupervised video person re-identification In Proceedings of the IEEE International Conference on Computer Vision, pp 2429–2438 [25] Peng P., Xiang T., Wang Y., Pontil M., Gong S., Huang T., and Tian Y (2016) Unsupervised cross-dataset transfer learning for person re-identification In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1306–1315 [26] Pham T.T.T., Le T.L., Vu H., Dao T.K., et al (2017) Fully-automated person reidentification in multi-camera surveillance system with a robust kernel descriptor and effective shadow removal method Image and Vision Computing , 59:pp 44– 62 [27] Li S., Shao M., and Fu Y (2015) Cross-view projective dictionary learning for person re-identification In Twenty-Fourth International Joint Conference on Artificial Intelligence [28] Karanam S., Gou M., Wu Z., Rates-Borras A., Camps O., and Radke R.J (2018) A systematic evaluation and benchmark for person re-identification: Features, metrics, and datasets IEEE Transactions on Pattern Analysis & Machine Intelligence, (1):pp 1–1 [29] Gray D and Tao H (2008) Viewpoint invariant pedestrian recognition with an ensemble of localized features In European conference on computer vision , pp 262–275 Springer [30] Cheng D.S., Cristani M., Stoppa M., Bazzani L., and Murino V (2011) Custom pictorial structures for re-identification In British Machine Vision Conference (BMVC 2011) [31] Das A., Chakraborty A., and Roy-Chowdhury A.K (2014) Consistent reidentification in a camera network In European Conference on Computer Vision (2014), pp 330–345 Springer [32] Hirzer M., Beleznai C., Roth P.M., and Bischof H (2011) Person re-identification by descriptive and discriminative classification In Scandinavian conference on Image analysis (2011), pp 91–102 Springer [33] of Computer Graphics I and Vision (2011) 2011 Person re-id (prid) dataset https://www.tugraz.at/institute/icg/research/team-bischof/ lrs/downloads/prid11/ [Online; accessed 13-May-2020] [34] Yan Y., Ni B., Song Z., Ma C., Yan Y., and Yang X (2016) Person re- identification via recurrent feature aggregation In European Conference on Computer Vision (2016), pp 701–716 Springer 115 [35] Wang T., Gong S., Zhu X., and Wang S (2014) Person re-identification by video ranking In European Conference on Computer Vision , pp 688–703 Springer [36] Bak S (2012) Human re-identification through a video camera network Ph.D thesis [37] Wang X., Doretto G., Sebastian T., Rittscher J., and Tu P (2007) Shape and appearance context modeling In 2007 ieee 11th international conference on computer vision, pp 1–8 Ieee [38] Lejbølle A.R., Nasrollahi K., and Moeslund T.B (2017) Late fusion in part-based person re-identification In Proceedings of the 9th International Conference on Machine Learning and Computing (2017), pp 385–393 ACM [39] Bąk S and Bremond F (2014) Re-identification by covariance descriptors In Person re-identification, pp 71–91 Springer [40] Zhao R., Ouyang W., and Wang X (2014) Learning mid-level filters for person re-identification In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 144–151 [41] Lazebnik S., Schmid C., and Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, pp 2169–2178 IEEE [42] Ma B., Su Y., and Jurie F (2012) Local descriptors encoded by fisher vectors for person re-identification In Computer Vision–ECCV 2012 Workshops and Demonstrations, pp 413–422 Springer [43] Zhao R., Ouyang W., and Wang X (2013) Person re-identification by salience matching In Proceedings of the IEEE International Conference on Computer Vision, pp 2528–2535 [44] Yang Y., Yang J., Yan J., Liao S., Yi D., and Li S.Z (2014) Salient color names for person re-identification In European conference on computer vision , pp 536–551 Springer [45] Liao S., Hu Y., Zhu X., and Li S.Z (2015) Person re-identification by local maximal occurrence representation and metric learning In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp 2197– 2206 [46] Liao S., Zhao G., Kellokumpu V., Pietikäinen M., and Li S.Z (2010) Modeling pixel process with scale invariant local patterns for background subtraction in 116 complex scenes In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 1301–1306 IEEE [47] Zhang L., Xiang T., and Gong S (2016) Learning a discriminative null space for person re-identification In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1239–1248 [48] Zhang Y., Li B., Lu H., Irie A., and Ruan X (2016) Sample-specific svm learning for person re-identification In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1278–1287 [49] Matsukawa T., Okabe T., Suzuki E., and Sato Y (2016) Hierarchical gaussian descriptor for person re-identification In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1363–1372 [50] Layne R., Hospedales T.M., Gong S., and Mary Q (2012) Person re- identification by attributes In Bmvc , volume 2, p [51] Su C., Yang F., Zhang S., Tian Q., Davis L.S., and Gao W (2015) Multitask learning with low rank attribute embedding for person re-identification In Proceedings of the IEEE international conference on computer vision, pp 3739– 3747 [52] Shi Z., Hospedales T.M., and Xiang T (2015) Transferring a semantic representation for person re-identification and search In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4184–4193 [53] Li D., Zhang Z., Chen X., Ling H., and Huang K (2016) A richly annotated dataset for pedestrian attribute recognition arXiv preprint arXiv:1603.07054 [54] Yi D., Lei Z., Liao S., and Li S.Z (2014) Deep metric learning for person reidentification In 2014 22nd International Conference on Pattern Recognition , pp 34–39 IEEE [55] Li W., Zhao R., Xiao T., and Wang X (2014) Deepreid: Deep filter pairing neural network for person re-identification In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 152–159 [56] Cheng D., Gong Y., Zhou S., Wang J., and Zheng N (2016) Person re- identification by multi-channel parts-based cnn with improved triplet loss function In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1335–1344 [57] McLaughlin N., Martinez del Rincon J., and Miller P (2016) Recurrent convolutional network for video-based person re-identification In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp 1325–1334 117 [58] Hong Q.N., Nguyen T.B., and Le T.L (2018) Enhancing person re-identification based on recurrent feature aggregation network In Multimedia Analysis and Pattern Recognition (MAPR), 2018 1st International Conference on, pp 1–6 IEEE [59] Varior R.R., Shuai B., Lu J., Xu D., and Wang G (2016) A siamese long shortterm memory architecture for human re-identification In European conference on computer vision, pp 135–153 Springer [60] Varior R.R., Haloi M., and Wang G (2016) Gated siamese convolutional neural network architecture for human re-identification In European conference on computer vision, pp 791–808 Springer [61] Liu H., Feng J., Qi M., Jiang J., and Yan S (2017) End-to-end comparative attention networks for person re-identification IEEE Transactions on Image Processing, 26(7):pp 3492–3506 [62] into Deep learning D (2014) Networks with Parallel Concatenations (GoogLeNet) https://d2l.ai/chapter/convolutional-modern/googlenet html [Online; accessed 10-March-2020] [63] He K., Zhang X., Ren S., and Sun J (2016) Deep residual learning for image recognition In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778 [64] ul Hassan M (2014) ResNet (34, 50, 101): Residual CNNs for Image Classification Tasks https://neurohive.io/en/popular-networks/resnet/ [Online; accessed 10-March-2020] [65] Weinberger K.Q and Saul L.K (2009) Distance metric learning for large margin nearest neighbor classification Journal of Machine Learning Research , 10(Feb):pp 207–244 [66] Dikmen M., Akbas E., Huang T.S., and Ahuja N (2010) Pedestrian recognition with a learned metric In Asian conference on Computer vision , pp 501–512 Springer [67] Prosser B.J., Zheng W.S., Gong S., Xiang T., and Mary Q (2010) Person re-identification by support vector ranking In BMVC , volume 2, p [68] Zheng W.S., Gong S., and Xiang T (2012) Reidentification by relative distance comparison IEEE transactions on pattern analysis and machine intelligence , 35(3):pp 653–668 [69] Roth P.M., Hirzer M., Koestinger M., Beleznai C., and Bischof H (2014) Mahalanobis distance learning for person re-identification In Person Re-Identification , pp 247–267 Springer 118 [70] Matsukawa T and Suzuki E (2019) Kernelized cross-view quadratic discriminant analysis for person re-identification [71] Shalev-Shwartz S., Singer Y., and Ng A.Y (2004) Online and batch learning of pseudo-metrics In Proceedings of the twenty-first international conference on Machine learning, p 94 ACM [72] Chopra S., Hadsell R., LeCun Y., et al (2005) Learning a similarity metric discriminatively, with application to face verification In CVPR (1) , pp 539– 546 [73] Goldberger J., Hinton G.E., Roweis S.T., and Salakhutdinov R.R (2005) Neighbourhood components analysis In Advances in neural information processing systems, pp 513–520 [74] Moghaddam B., Jebara T., and Pentland A (2000) Bayesian face recognition Pattern Recognition, 33(11):pp 1771–1782 [75] Koestinger M., Hirzer M., Wohlhart P., Roth P.M., and Bischof H (2012) Large scale metric learning from equivalence constraints In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp 2288–2295 IEEE [76] Shawe-Taylor J., Cristianini N., et al (2004) Kernel methods for pattern analysis Cambridge university press [77] Xiong F., Gou M., Camps O., and Sznaier M (2014) Person re-identification using kernel-based metric learning methods In European conference on computer vision, pp 1–16 Springer [78] Gao C., Wang J., Liu L., Yu J.G., and Sang N (2016) Temporally aligned pooling representation for video-based person re-identification In Image Processing (ICIP), 2016 IEEE International Conference on, pp 4284–4288 IEEE [79] Avraham T., Gurvich I., Lindenbaum M., and Markovitch S (2012) Learning implicit transfer for person re-identification In Computer Vision–ECCV 2012 Workshops and Demonstrations, pp 381–390 Springer [80] Graves A (2013) Generating sequences with recurrent neural networks arXiv preprint arXiv:1308.0850 [81] Bazzani L., Cristani M., and Murino V (2013) Symmetry-driven accumulation of local features for human characterization and re-identification Computer Vision and Image Understanding, 117(2):pp 130–144 [82] Wu Y., Minoh M., Mukunoki M., and Lao S (2012) Set based discriminative ranking for recognition In European Conference on Computer Vision , pp 497– 510 Springer 119 [83] Wu Y., Mukunoki M., and Minoh M (2014) Locality-constrained collaboratively regularized nearest points for multiple-shot person re-identification In Proc of The 20th Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV) Citeseer [84] Huang Z., Wang R., Shan S., and Chen X (2015) Projection metric learning on grassmann manifold with application to video based face recognition In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 140–149 [85] Wang R and Chen X (2009) Manifold discriminant analysis In Computer Vision and Pattern Recognition, 2009 CVPR 2009 IEEE Conference on, pp 429–436 IEEE [86] Wang R., Guo H., Davis L.S., and Dai Q (2012) Covariance discriminative learning: A natural and efficient approach to image set classification In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp 2496–2503 IEEE [87] Gao M., Ai H., and Bai B (2016) A feature fusion strategy for person reidentification In 2016 IEEE International Conference on Image Processing (ICIP), pp 4274–4278 IEEE [88] Liu Y., Song N., and Han Y (2019) Multi-cue fusion: Discriminative enhancing for person re-identification Journal of Visual Communication and Image Representation, 58:pp 46–52 [89] Johnson J., Yasugi S., Sugino Y., Pranata S., and Shen S (2018) Person reidentification with fusion of hand-crafted and deep pose-based body region features arXiv preprint arXiv:1803.10630 [90] Yuan L and Tian Z (2016) Person re-identification based on color and texture feature fusion In International Conference on Intelligent Computing , pp 341– 352 Springer [91] ur Rehman S., Chen Z., Shah J.H., and Raza M (2016) Multi-feature fusion based re-ranking for person re-identification In Audio, Language and Image Processing (ICALIP), 2016 International Conference on, pp 213–216 IEEE [92] Zeng M., Tian C., and Wu Z (2018) Person re-identification with hierarchical deep learning feature and efficient xqda metric In 2018 ACM Multimedia Conference on Multimedia Conference, pp 1838–1846 ACM [93] Eisenbach M., Kolarow A., Vorndran A., Niebling J., and Gross H.M (2015) Evaluation of multi feature fusion at score-level for appearance-based person 120 re-identification In 2015 International Joint Conference on Neural Networks (IJCNN), pp 1–8 IEEE [94] Lejbølle A.R., Nasrollahi K., and Moeslund T.B (2017) Enhancing person reidentification by late fusion of low-, mid-and high-level features Iet Biometrics [95] Zheng L., Wang S., Tian L., He F., Liu Z., and Tian Q (2015) Query-adaptive late fusion for image search and person re-identification In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (2015), pp 1741– 1750 [96] Zhao H., Tian M., Sun S., Shao J., Yan J., Yi S., Wang X., and Tang X (2017) Spindle net: Person re-identification with human body region guided feature decomposition and fusion In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1077–1085 [97] Wei S.E., Ramakrishna V., Kanade T., and Sheikh Y (2016) Convolutional pose machines In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp 4724–4732 [98] Xin W., Dongdong G., Peng L., and Zhe J (2016) Person re-identification by features fusion In Information Technology, Networking, Electronic and Automation Control Conference (2016), pp 285–289 IEEE [99] Wu S., Chen Y.C., Li X., Wu A.C., You J.J., and Zheng W.S (2016) An enhanced deep feature representation for person re-identification In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp 1–8 IEEE [100] Liu K., Ma B., Zhang W., and Huang R (2015) A spatio-temporal appearance representation for video-based pedestrian re-identification In Proceedings of the IEEE International Conference on Computer Vision (2015), pp 3810–3818 [101] Zhang W., Hu S., and Liu K (2017) Learning compact appearance representation for video-based person re-identification arXiv preprint arXiv:1702.06294 [102] Wang T., Gong S., Zhu X., and Wang S (2016) Person re-identification by discriminative selection in video ranking IEEE Trans Pattern Anal Mach Intell., 38(12):pp 2501–2514 [103] Frikha M., Chebbi O., Fendri E., and Hammami M (2016) Key frame selection for multi-shot person re-identification In International Workshop on Representations, Analysis and Recognition of Shape and Motion FroM Imaging Data (2016), pp 97–110 Springer 121 [104] Hassen Y.H., Ayedi W., Ouni T., and Jallouli M (2015) Multi-shot person reidentification approach based key frame selection In Eighth International Conference on Machine Vision (ICMV 2015), volume 9875, p 98751H International Society for Optics and Photonics [105] Hassen Y.H., Loukil K., Ouni T., and Jallouli M (2017) Images selection and best descriptor combination for multi-shot person re-identification In International Conference on Intelligent Interactive Multimedia Systems and Services (2017), pp 11–20 Springer [106] El-Alfy H., Muramatsu D., Teranishi Y., Nishinaga N., Makihara Y., and Yagi Y (2017) A visual surveillance system for person re-identification In Thirteenth International Conference on Quality Control by Artificial Vision 2017 , volume 10338, p 103380D International Society for Optics and Photonics [107] Zheng L., Zhang H., Sun S., Chandraker M., Yang Y., and Tian Q (2017) Person re-identification in the wild In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1367–1376 [108] Song C., Huang Y., Ouyang W., and Wang L (2018) Mask-guided contrastive attention model for person re-identification In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1179–1188 [109] Dollár P., Appel R., Belongie S., and Perona P (2014) Fast feature pyramids for object detection IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(8):pp 1532–1545 [110] Redmon J and Farhadi A (2018) Yolov3: An incremental improvement arXiv preprint arXiv:1804.02767 [111] He K., Gkioxari G., Dollár P., and Girshick R (2017) Mask r-cnn In Proceedings of the IEEE international conference on computer vision, pp 2961–2969 [112] Luo P., Wang X., and Tang X (2013) Pedestrian parsing via deep decompositional network In Proceedings of the IEEE international conference on computer vision, pp 2648–2655 [113] Nguyen T.B., Van Phu P., Le T.L., and Le C.V (2016) Background removal for improving saliency-based person re-identification In 2016 Eighth International Conference on Knowledge and Systems Engineering (KSE), pp 339–344 IEEE [114] McGuinness K and O’Connor N.E (2008) The k-space segmentation tool set [115] Le C.V., Tuan N.N., Hong Q.N., and Lee H.J (2017) Evaluation of recurrent neural network variants for person re-identification IEIE Transactions on Smart Processing & Computing, 6(3):pp 193–199 122 [116] Pham T.T.T., Le T.L., Dao T.K., and Le D.H (2015) A robust model for person re-identification in multimodal person localization UBICOMM 2015 , p 51 [117] Bo L., Ren X., and Fox D (2010) Kernel descriptors for visual recognition In Advances in neural information processing systems (2010), pp 244–252 [118] Nguyen N.B., Nguyen V.H., Duc T.N., Duong D.A., et al (2015) Using attribute relationships for person re-identification In Knowledge and Systems Engineering , pp 195–207 Springer [119] Nguyen N.B., Nguyen V.H., Duc T.N., Le D.D., and Duong D.A (2015) Attrel: an approach to person re-identification by exploiting attribute relationships In International Conference on Multimedia Modeling, pp 50–60 Springer [120] Layne R., Hospedales T.M., and Gong S (2014) Attributes-based re- identification In Person re-identification , pp 93–117 Springer [121] Nguyen N.B., Nguyen V.H., Ngo T.D., and Nguyen K.M (2017) Person reidentification with mutual re-ranking Vietnam Journal of Computer Science , 4(4):pp 233–244 [122] Nguyen V.H., Nguyen K., Le D.D., Duong D.A., and Satoh S (2013) Person re-identification using deformable part models In International Conference on Neural Information Processing, pp 616–623 Springer [123] Viet N.C., Cong D.T., and Ho-Phuoc T (2015) Manifold-based learning for person re-identification In 2015 International Conference on Advanced Technologies for Communications (ATC), pp 688–691 IEEE [124] Le T.L., Thonnat M., Boucher A., and Brémond F (2009) Appearance based retrieval for tracked objects in surveillance videos In Proceedings of the ACM International Conference on Image and Video Retrieval , CIVR ’09, pp 40:1– 40:8 ACM, New York, NY, USA ISBN 978-1-60558-480-5 doi:10.1145/1646396 1646444 [125] Lucas B.D., Kanade T., et al (1981) An iterative image registration technique with an application to stereo vision [126] Li P., Wang Q., and Zhang L (2013) A novel earth mover’s distance methodology for image matching with gaussian mixture models In Proceedings of the IEEE International Conference on Computer Vision, pp 1689–1696 [127] Singh B., Parwate D., and Shukla S (2009) Radiosterilization of fluoroquinolones and cephalosporins: Assessment of radiation damage on antibiotics by changes in optical property and colorimetric parameters AAPS PharmSciTech , 10(1):pp 34–43 123 [128] Wikipedia (2020) Illuminant D65 https://en.wikipedia.org/wiki/ Illuminant_D65/ [Online; accessed 10-March-2020] [129] Popov V., Ostarek M., and Tenison C (2018) Practices and pitfalls in inferring neural representations NeuroImage , 174:pp 340–351 [130] John Lu Z (2010) The elements of statistical learning: data mining, inference, and prediction Journal of the Royal Statistical Society: Series A (Statistics in Society), 173(3):pp 693–694 [131] Li Z., Chang S., Liang F., Huang T.S., Cao L., and Smith J.R (2013) Learning locally-adaptive decision functions for person verification In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3610–3617 [132] Geng S., Yu M., Liu Y., Yu Y., and Bai J (2018) Re-ranking pedestrian reidentification with multiple metrics Multimedia Tools and Applications , pp 1– 23 [133] Li M., Zhu X., and Gong S (2018) Unsupervised person re-identification by deep learning tracklet association In Proceedings of the European Conference on Computer Vision (ECCV), pp 737–753 [134] Li M., Zhu X., and Gong S (2019) Unsupervised tracklet person re-identification IEEE transactions on pattern analysis and machine intelligence [135] Zeng Z., Li Z., Cheng D., Zhang H., Zhan K., and Yang Y (2017) Twostream multirate recurrent neural network for video-based pedestrian reidentification IEEE Transactions on Industrial Informatics , 14(7):pp 3179–3186 [136] Liu H., Jie Z., Jayashree K., Qi M., Jiang J., Yan S., and Feng J (2017) Videobased person re-identification with accumulative motion context IEEE transactions on circuits and systems for video technology, 28(10):pp 2788–2802 [137] Liu Z., Chen J., and Wang Y (2016) A fast adaptive spatio-temporal 3d feature for video-based person re-identification In Image Processing (ICIP), 2016 IEEE International Conference on, pp 4294–4298 IEEE [138] Li Y., Zhuo L., Li J., Zhang J., Liang X., and Tian Q (2017) Video-based person re-identification by deep feature guided pooling In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2017), pp 39–46 [139] Zhang D., Wu W., Cheng H., Zhang R., Dong Z., and Cai Z (2017) Imageto-video person re-identification with temporally memorized similarity learning IEEE Transactions on Circuits and Systems for Video Technology 124 [140] Wang G., Lai J., and Xie X (2017) P2snet: Can an image match a video for person re-identification in an end-to-end way? IEEE Transactions on Circuits and Systems for Video Technology [141] Ojala T., Pietikainen M., and Harwood D (1994) Performance evaluation of texture measures with classification based on kullback discrimination of distributions In Pattern Recognition, 1994 Vol 1-Conference A: Computer Vision & Image Processing., Proceedings of the 12th IAPR International Conference on, volume 1, pp 582–585 IEEE [142] Zheng Y., Sheng H., Zhang B., Zhang J., and Xiong Z (2015) Weight-based sparse coding for multi-shot person re-identification Science China Information Sciences (2015), 58(10):pp 1–15 [143] Jia Y et al (2013) Caffe: an open source convolutional architecture for fast feature embedding (2013) http://caffe.berkeleyvision.org/ [144] Kittler J., Hatef M., Duin R.P., and Matas J (1998) On combining classifiers IEEE transactions on pattern analysis and machine intelligence, 20(3):pp 226– 239 [145] Kittler J., Hatef M., Duin R.P.W., and Matas J (Mar 1998) On combining classifiers IEEE Transactions on Pattern Analysis and Machine Intelligence , 20(3):pp 226–239 ISSN 0162-8828 doi:10.1109/34.667881 [146] Lisanti G., Masi I., Bagdanov A.D., and Del Bimbo A (2015) Person reidentification by iterative re-weighted sparse ranking IEEE transactions on pattern analysis and machine intelligence, 37(8):pp 1629–1642 [147] Sheng H., Zhou X., Zheng Y., Liu Y., and Yang D (2017) Person re-identification with discriminative dictionary learning DEStech Transactions on Computer Science and Engineering, (csae) [148] Chen L., Yang H., Zhu J., Zhou Q., Wu S., and Gao Z (2017) Deep spatialtemporal fusion network for video-based person re-identification In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 63–70 [149] Chen L., Yang H., and Gao Z (2020) Comprehensive feature fusion mechanism for video-based person re-identification via significance-aware attention Signal Processing: Image Communication, p 115835 [150] Ren S., He K., Girshick R., and Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks In Advances in neural information processing systems, pp 91–99 125 [151] Friedman J., Hastie T., Tibshirani R., et al (2000) Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors) The annals of statistics, 28(2):pp 337–407 [152] Redmon J., Divvala S., Girshick R., and Farhadi A (2016) You only look once: Unified, real-time object detection In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788 [153] Nguyen H.Q., Nguyen T.B., Le T.A., Le T.L., Vu T.H., and Noe A (2019) Comparative evaluation of human detection and tracking approaches for online tracking applications In 2019 International Conference on Advanced Technologies for Communications (ATC), pp 348–353 IEEE [154] Matsukawa T., Okabe T., Suzuki E., and Sato Y (2017) Hierarchical gaussian descriptors with application to person re-identification arXiv preprint arXiv:1706.04318 [155] Liu H., Qin L., Cheng Z., and Huang Q (2013) Set-based classification for person re-identification utilizing mutual-information In 2013 IEEE International Conference on Image Processing, pp 3078–3082 IEEE [156] Tian M., Yi S., Li H., Li S., Zhang X., Shi J., Yan J., and Wang X (2018) Eliminating background-bias for robust person re-identification In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5794– 5803 [157] Ghorbel M., Ammar S., Kessentini Y., and Jmaiel M (2019) Improving person re-identification by background subtraction using two-stream convolutional networks In International Conference on Image Analysis and Recognition , pp 345–356 Springer [158] Springer (2016) MARS: A Video Benchmark for Large-Scale Person Re- identification [159] Liu Z., Zhang Z., Wu Q., and Wang Y (2015) Enhancing person re-identification by integrating gait biometric Neurocomputing , 168:pp 1144 – 1156 ISSN 09252312 doi:https://doi.org/10.1016/j.neucom.2015.05.008 [160] Li W., Zhu X., and Gong S (2018) Harmonious attention network for person reidentification In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2285–2294 [161] Yu H.X., Wu A., and Zheng W.S (2018) Unsupervised person re-identification by deep asymmetric metric embedding IEEE Transactions on Pattern Analysis and Machine Intelligence, 42:pp 956–973 126 [162] Leng Q., Ye M., and Tian Q (2020) A survey of open-world person reidentification IEEE Transactions on Circuits and Systems for Video Technology , 30(4):pp 1092–1108 127 ... open-set person ReID In Figure 1. 3a) the person appears on both cameras, while she appears only on the camera -A in Figure 1.3b) Camera -A Camera- B Camera -A (a) Close-set person ReID Camera- B (b)... non-overlapping cameras This dataset is used for the single-shot case • CAVIAR4REID [30] This dataset contains 1,220 images of 72 pedestrians captured from two non-overlapping cameras in a shopping mall... total persons appear in all cameras All images in this dataset are normalized to the same size of 64 × 128 The large illumination variations caused by collecting images from different scenarios