Luận án tiến sĩ nghiên cứu phát triển các phương pháp của lý thuyết đồ thị và otomat trong giấu tin mật và mã hóa tìm kiếm

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY NGUYEN THI THANH NHAN INTERACTIVE AND MULTI-ORGAN BASED PLANT SPECIES IDENTIFICATION Major: Computer Science Code: 9480101 INTERACTIVE AND MULTI-ORGAN BASED PLANT SPECIES IDENTIFICATION SUPERVISORS: Assoc Prof Dr Le Thi Lan Prof Dr Hoang Van Sam Hanoi − 2020 HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY Nguyen Thi Thanh Nhan INTERACTIVE AND MULTI-ORGAN BASED PLANT SPECIES IDENTIFICATION Major: Computer Science Code: 9480101 DOCTORAL DISSERTATION OF COMPUTER SCIENCE SUPERVISORS: Assoc Prof Dr Le Thi Lan Prof Dr Hoang Van Sam Hanoi − 2020 DECLARATION OF AUTHORSHIP I, Nguyen Thi Thanh Nhan, declare that this dissertation entitled, ”Interactive and multi-organ based plant species identification”, and the work presented in it is my own I confirm that: This work was done wholly or mainly while in candidature for a Ph.D research degree at Hanoi University of Science and Technology Where any part of this dissertation has previously been submitted for a degree or any other qualification at Hanoi University of Science and Technology or any other institution, this has been clearly stated Where I have consulted the published work of others, this is always clearly at-tributed Where I have quoted from the work of others, the source is always given With the exception of such quotations, this dissertation is entirely my own work I have acknowledged all main sources of help Where the dissertation is based on work done by myself jointly with others, I have made exactly what was done by others and what I have contributed myself Hanoi, May, 2020 PhD Student Nguyen Thi Thanh Nhan SUPERVISORS i ACKNOWLEDGEMENT First of all, I would like to thank my supervisors Assoc Prof Dr Le Thi Lan at The International Research Institute MICA - Hanoi University of Science and Technology, Assoc Prof Dr Hoang Van Sam at Vietnam National University of Forestry for their inspiration, guidance, and advice Their guidance helped me all the time of research and writing this dissertation Besides my advisors, I would like to thank Assoc Prof Dr Vu Hai, Assoc Prof Dr Tran Thi Thanh Hai for their great discussion Special thanks to my friends/colleagues in MICA, Hanoi University of Science and Technology: Hoang Van Nam, Nguyen Hong Quan, Nguyen Van Toi, Duong Nam Duong, Le Van Tuan, Nguyen Huy Hoang, Do Thanh Binh for their technical supports They have assisted me a lot in my research process as well as they are co-authored in the published papers Moreover, I would like to thank reviewers of scientific conferences, journals and protection council, reviewers, they help me with many useful comments I would like to express a since gratitude to the Management Board of MICA In-stitute I would like to thank the Thai Nguyen University of Information and Commu-nication Technology, Thai Nguyen over the years both at my career work and outside of the work As a Ph.D student of the 911 program, I would like to thank this program for financial support I also gratefully acknowledge the financial support for attending the conferences from the Collaborative Research Program for Common Regional Is-sue (CRC) funded by ASEAN University Network (Aun-Seed/Net), under the grant reference HUST/CRC/1501 and NAFOSTED (grant number 106.06-2018.23) Special thanks to my family, to my parents-in-law who took care of my family and created favorable conditions for me to study I also would like to thank my beloved husband and children for everything they supported and encouraged me for a long time to study Hanoi, May, 2020 Ph.D Student Nguyen Thi Thanh Nhan ii CONTENTS DECLARATION OF AUTHORSHIP ACKNOWLEDGEMENT i ii CONTENTS v SYMBOLS vi SYMBOLS viii LIST OF TABLES xi LIST OF FIGURES xvi INTRODUCTION 1 LITERATURE REVIEW 10 1.1 Plant identification 1.1.1 Manual plant identification 1.1.2 Plant identification based on semi-automatic graphic tool 1.1.3 Automated plant identification 1.2 Automatic plant identification from images of single organ 1.2.1 Introducing the plant organs 1.2.2 General model of image-based plant identification 1.2.3 Preprocessing techniques for images of plant 1.2.4 Feature extraction 1.2.4.1 Hand-designed features 1.2.4.2 Deeply-learned features 1.2.5 Training methods 1.3 Plant identification from images of multiple organs 1.3.1 Early fusion techniques for plant identification from images of multiple organs 1.3.2 Late fusion techniques for plant identification from images of multiple organs 1.4 Plant identification studies in Vietnam 1.5 Plant data collection and identification systems 1.6 Conclusions iii 10 10 11 12 13 13 16 17 19 20 22 25 28 30 31 33 35 43 LEAF-BASED PLANT IDENTIFICATION METHOD BASED ON KERNEL DESCRIPTOR 2.1 The framework of leaf-based plant identification method 2.2 Interactive segmentation 2.3 Feature extraction 2.3.1 Pixel-level features extraction 2.3.2 Patch-level features extraction 2.3.2.1 Generate a set of patches from an image with adaptive size 2.3.2.2 Compute patch-level feature 2.3.3 Image-level features extraction 2.3.4 Time complexity analysis 2.4 Classification 2.5 Experimental results 2.5.1 Datasets 2.5.1.1 ImageCLEF 2013 dataset 2.5.1.2 Flavia dataset 2.5.1.3 LifeCLEF 2015 dataset 2.5.2 Experimental results 2.5.2.1 Results on ImageCLEF 2013 dataset 2.5.2.2 Results on Flavia dataset 2.5.2.3 Results on LifeCLEF 2015 dataset 2.6 Conclusions FUSION SCHEMES FOR MULTI-ORGAN BASED PLANT IDENTIFICATION 3.1 Introduction 3.2 The proposed fusion scheme RHF 3.3 The choice of classification model for single organ plant identification 3.4 Experimental results 3.4.1 Dataset 3.4.2 Single organ plant identification results 3.4.3 Evaluation of the proposed fusion scheme in multi-organ plant identification 3.5 Conclusion TOWARDS BUILDING AN AUTOMATIC PLANT RETRIEVAL BASED ON PLANT IDENTIFICATION 4.1 Introduction 4.2 Challenges of building automatic plant identification systems iv 45 45 46 50 50 51 51 52 55 56 57 57 57 57 57 58 58 58 61 61 68 69 69 71 77 79 80 81 81 89 90 90 90 4.3 The framework for building automatic plant identification system 94 4.4 Plant organ detection 96 4.5 Case study: Development of image-based plant retrieval in VnMed application 101 4.6 Conclusions 106 CONCLUSIONS AND FUTURE WORKS 107 4.6.1 Short term 4.6.2 Long term Bibliography 108 108 110 PUBLICATIONS 124 APPENDIX 125 v ABBREVIATIONS No Abbreviation Meaning AB ANN Ada Boost Artificial Neural Network Br Branch CBF Classification Base Fusion CNN Convolution Neural Network CNNs Convolution Neural Networks CPU Central Processing Unit CMC Cumulative Match Characteristic Curve DT Decision Tree 10 En Entire 11 FC Fully Connected 12 Fl Flower 13 FN False Negative 14 FP False Positive 15 GPU Graphics Processing Unit 16 GUI Graphic-User Interface 17 HOG Histogram of Oriented Gradients 18 ILSVRC ImageNet Large Scale Visual Recognition Competition 19 KDES Kernel DEScriptors 20 KNN K Nearest Neighbors 21 Le Leaf 22 L-SVMLinear Support Vector Machine 23 MCDCNNMulti Column Deep Convolutional Neural Networks 24 NB 25 NNB Naive Bayes Nearest NeighBor 26 OPENCV OPEN source Computer Vision Library 27 PC Persional Computer 28 PCA Principal Component Analysis 29 PNN Probabilistic Neural Network 30 QDA Quadratic Discriminant Analysis vi 31 RAM 32 ReLU Random Acess Memory Rectified Linear Unit 33 RHF Robust Hybrid Fusion 34 RF Random Forest 35 ROI Region Of Interest 36 SIFT Scale-Invariant Feature Transform 37 SM SoftMax 38 SURF Speeded Up Robust Features 39 SVM Support Vector Machine 40 SVM-RBF Support Vector Machine-Radial Basic Function kernel 41 TP True Positive 42 TN True Negative vii MATH SYMBOLS No Symbol Meaning % × 1% = 1/100 Multiplication + Addition Summation - sum of all values in range of series P Multiplication R / = Division Equality 10 ≥ Greater than or equal to 11 ≤ Less than or equal to 12 13 π kwk π = 3.141592654 L2 normalize of vector w 14 15 xi sign(x) The i-th element of vector x The sign function that determines the sign Equals if x ≥ 0, −1 16 ∈ if x < Is member of 17 18 max ∀ The function takes the largest number from a list For all 19 20 m I(x, y) Spatial moment of an image The intensity value at (x, y) of an image 21 − Subtraction 22 23 O arctan(x) Complexity of an algorithm It returns the angle whose tangent is a given number 24 cos(θ) Function of calculating cosine value of angle θ 25 sin(θ) Function of calculating sine value of angle θ 26 m(z) The magnitude of the gradient vector at pixel z 27 θ(z) The orientation of gradient vector at pixel z 28 ˜ θ(z) exp(x) The normalized gradient vector 29 R d Set of real number Set of real number has d dimensions e x viii