HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY NGUYEN THI THANH NHAN INTERACTIVE AND MULTI ORGAN BASED PLANT SPECIES IDENTIFICATION Major Computer Science Code 9480101 INTERACTIVE AND MULTI ORGAN BASED PL[.]
HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY NGUYEN THI THANH NHAN INTERACTIVE AND MULTI-ORGAN BASED PLANT SPECIES IDENTIFICATION Major: Computer Science Code: 9480101 INTERACTIVE AND MULTI-ORGAN BASED PLANT SPECIES IDENTIFICATION SUPERVISORS: Assoc Prof Dr Le Thi Lan Assoc Prof Dr Hoang Van Sam Hanoi − 2020 HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY Nguyen Thi Thanh Nhan INTERACTIVE AND MULTI-ORGAN BASED PLANT SPECIES IDENTIFICATION Major: Computer Science Code: 9480101 DOCTORAL DISSERTATION OF COMPUTER SCIENCE SUPERVISORS: Assoc Prof Dr Le Thi Lan Assoc Prof Dr Hoang Van Sam Hanoi − 2020 DECLARATION OF AUTHORSHIP I, Nguyen Thi Thanh Nhan, declare that this dissertation entitled, ”Interactive and multi-organ based plant species identification”, and the work presented in it is my own I confirm that: This work was done wholly or mainly while in candidature for a Ph.D research degree at Hanoi University of Science and Technology Where any part of this dissertation has previously been submitted for a degree or any other qualification at Hanoi University of Science and Technology or any other institution, this has been clearly stated Where I have consulted the published work of others, this is always clearly attributed Where I have quoted from the work of others, the source is always given With the exception of such quotations, this dissertation is entirely my own work I have acknowledged all main sources of help Where the dissertation is based on work done by myself jointly with others, I have made exactly what was done by others and what I have contributed myself Hanoi, January, 2020 PhD Student Nguyen Thi Thanh Nhan SUPERVISORS i ACKNOWLEDGEMENT First of all, I would like to thank my supervisors Assoc Prof Dr Le Thi Lan at The International Research Institute MICA - Hanoi University of Science and Technology, Assoc Prof Dr Hoang Van Sam at Vietnam National University of Forestry for their inspiration, guidance, and advice Their guidance helped me all the time of research and writing this dissertation Besides my advisors, I would like to thank Dr Vu Hai, Assoc Prof Dr Tran Thi Thanh Hai for their great discussion Special thanks to my friends/colleagues in MICA, Hanoi University of Science and Technology: Hoang Van Nam, Nguyen Hong Quan, Nguyen Van Toi, Duong Nam Duong, Le Van Tuan, Nguyen Huy Hoang, Do Thanh Binh for their technical supports They have assisted me a lot in my research process as well as they are co-authored in the published papers Moreover, I would like to thank reviewers of scientific conferences, journals and protection council, reviewers, they help me with many useful comments I would like to express a since gratitude to the Management Board of MICA Institute I would like to thank the Thai Nguyen University of Information and Communication Technology, Thai Nguyen over the years both at my career work and outside of the work As a Ph.D student of the 911 program, I would like to thank this program for financial support I also gratefully acknowledge the financial support for attending the conferences from the Collaborative Research Program for Common Regional Issue (CRC) funded by ASEAN University Network (Aun-Seed/Net), under the grant reference HUST/CRC/1501 and NAFOSTED (grant number 106.06-2018.23) Special thanks to my family, to my parents-in-law who took care of my family and created favorable conditions for me to study I also would like to thank my beloved husband and children for everything they supported and encouraged me for a long time to study Hanoi, January, 2020 Ph.D Student Nguyen Thi Thanh Nhan ii CONTENTS DECLARATION OF AUTHORSHIP i ACKNOWLEDGEMENT ii CONTENTS v SYMBOLS vi SYMBOLS viii LIST OF TABLES x LIST OF FIGURES xiv INTRODUCTION 1 LITERATURE REVIEW 1.1 Plant identification 1.1.1 Manual plant identification 1.1.2 Plant identification based on semi-automatic graphic tool 1.1.3 Automated plant identification 1.2 Automatic plant identification from images of single organ 1.2.1 Introducing the plant organs 1.2.2 General model of image-based plant identification 1.2.3 Preprocessing techniques for images of plant 1.2.4 Feature extraction 1.2.4.1 Hand-designed features 1.2.4.2 Deeply-learned features 1.2.5 Classification methods 1.3 Plant identification from images of multiple organs 1.3.1 Early fusion techniques for plant identification from images multiple organs 1.3.2 Late fusion techniques for plant identification from images multiple organs 1.4 Plant identification studies in Vietnam 1.5 Plant data collection and identification systems 1.6 Conclusions iii of of 10 10 10 12 12 13 13 16 18 20 20 22 26 28 30 31 33 35 44 LEAF-BASED PLANT IDENTIFICATION METHOD BASED ON KERNEL DESCRIPTOR 2.1 The framework of leaf-based plant identification method 2.2 Interactive segmentation 2.3 Feature extraction 2.3.1 Pixel-level features extraction 2.3.2 Patch-level features extraction 2.3.2.1 Generate a set of patches from an image with adaptive size 2.3.2.2 Compute patch-level feature 2.3.3 Image-level features extraction 2.4 Experimental results 2.4.1 Datasets 2.4.1.1 ImageCLEF 2013 dataset 2.4.1.2 Flavia dataset 2.4.1.3 LifeCLEF 2015 dataset 2.4.2 Experimental results 2.4.2.1 Results on ImageCLEF 2013 dataset 2.4.2.2 Results on Flavia dataset 2.4.2.3 Results on LifeCLEF 2015 dataset 2.5 Conclusions FUSION SCHEMES FOR MULTI-ORGAN BASED PLANT IDENTIFICATION 3.1 Introduction 3.2 The proposed fusion scheme RHF 3.3 The choice of classification model for single organ plant identification 3.4 Experimental results 3.4.1 Dataset 3.4.2 Single organ plant identification results 3.4.3 Evaluation of the proposed fusion scheme in multi-organ plant identification 3.5 Conclusion 45 45 46 50 50 51 51 52 55 56 56 56 57 57 58 58 60 61 66 67 67 69 75 77 78 79 79 87 A FRAMEWORK FOR AUTOMATIC PLANT IDENTIFICATION WITHOUT DEDICATED DATASET AND A CASE STUDY FOR BUILDING IMAGE-BASED PLANT RETRIEVAL 88 4.1 Introduction 88 4.2 Challenges of building automatic plant identification systems 88 iv 4.3 4.4 4.5 4.6 The framework for building automatic plant identification system without dedicated dataset 92 Plant organ detection 93 Case study: Development of image-based plant retrieval in VnMed application 99 Conclusions 104 CONCLUSIONS AND FUTURE WORKS 105 4.6.1 Short term 106 4.6.2 Long term 106 Bibliography 108 PUBLICATIONS 121 APPENDIX 122 v ABBREVIATIONS No Abbreviation Meaning AB Ada Boost ANN Artificial Neural Network Br Branch CBF Classification Base Fusion CNN Convolution Neural Network CNNs Convolution Neural Networks CPU Central Processing Unit CMC Cumulative Match Characteristic Curve DT Decision Tree 10 En Entire 11 FC Fully Connected 12 Fl Flower 13 FN False Negative 14 FP False Positive 15 GPU Graphics Processing Unit 16 GUI Graphic-User Interface 17 HOG Histogram of Oriented Gradients 18 ILSVRC ImageNet Large Scale Visual Recognition Competition 19 KDES Kernel DEScriptors 20 KNN K Nearest Neighbors 21 Le Leaf 22 L-SVM Linear Support Vector Machine 23 MCDCNN Multi Column Deep Convolutional Neural Networks 24 NB Naive Bayes 25 NNB Nearest NeighBor 26 OPENCV OPEN source Computer Vision Library 27 PC Persional Computer 28 PCA Principal Component Analysis 29 PNN Probabilistic Neural Network 30 QDA Quadratic Discriminant Analysis vi 31 RAM Random Acess Memory 32 ReLU Rectified Linear Unit 33 RHF Robust Hybrid Fusion 34 RF Random Forest 35 ROI Region Of Interest 36 SIFT Scale-Invariant Feature Transform 37 SM SoftMax 38 SURF Speeded Up Robust Features 39 SVM Support Vector Machine 40 SVM-RBF Support Vector Machine-Radial Basic Function kernel 41 TP True Positive 42 TN True Negative vii MATH SYMBOLS No Symbol P R d Meaning Summation - sum of all values in range of series Set of real number R π π = 3.141592654 kwk L2 normalize of vector w xi The i-th element of vector x sign(x) The sign function that determines the sign Equals if x ≥ 0, −1 if x < ∈ Is member of max The function takes the largest number from a list 10 arctan(x) It returns the angle whose tangent is a given number 11 cos(θ) Function of calculating cosine value of angle θ 12 sin(θ) Function of calculating sine value of angle θ 13 m(z) The magnitude of the gradient vector at pixel z 14 The orientation of gradient vector at pixel z 15 θ(z) ˜ θ(z) The normalized gradient vector 16 exp(x) ex 17 argmax(x) It indicates the element that reaches its maximum value 18 ⊗ The Kronecker product 19 xT Q Transposition of vector x 20 Set of real number has d dimensions Product of all values in range of series 21 q The query-image set 22 si (Ik ) The confidence score of the plant species i−th when using image Ik as a query from a single organ plant 23 c The predicted class of the species for the query q 24 C The number of species in dataset 25 km˜ The gradient magnitude kernel 26 ko The orientation kernel 27 kp The position kernel 28 m(z) ˜ The normalized gradient magnitude viii [13] Bonnet P., Arbonnier M., and Grard P (2005) A graphic tool for the identification of west african savannas trees in Smithsonian Botanical Symposium, 2005 [14] Goăeau H., Joly A., Bonnet P., Selmi S., Molino J.F., Barth´el´emy D., and Boujemaa N (2014) Lifeclef plant identification task 2014 In CLEF2014 Working Notes Working Notes for CLEF 2014 Conference, Sheffield, UK, September 1518, 2014 , pp 598–615 [15] Hsu T.H., Lee C.H., and Chen L.H (2011) An interactive flower image recognition system Multimedia Tools and Applications, 53(1):pp 53–73 [16] Zhang Y and Wu L (2012) Classification of fruits using computer vision and a multiclass support vector machine Sensors, 12(9):pp 12489–12505 [17] Goăeau H., Bonnet P., Joly A., Boujemaa N., Barthelemy D., Molino J.F., Birnbaum P., Mouysset E., and Picard M (2011) The imageclef 2011 plant images classification task In ImageCLEF 2011 , pp 119 [18] Goăeau H., Bonnet P., Joly A., Yahiaoui I., Barth´el´emy D., Boujemaa N., and Molino J.F The imageclef 2012 plant identification task In ImageCLEF 2012 [19] Yanikoglu B., Aptoula E., and Tirkaz C (2014) Automatic plant identification from photographs Machine vision and applications, 25(6):pp 1369–1383 [20] Wang X.F., Huang D.S., Du J.X., Xu H., and Heutte L (2008) Classification of plant leaf images with complicated background Applied mathematics and computation, 205(2):pp 916–926 [21] Hong A.x., Chen G., Li J.l., Chi Z.r., and Zhang D (2004) A flower image retrieval method based on roi feature Journal of Zhejiang University Science, 5(7):pp 764–772 [22] Angelova A., Zhu S., and Lin Y (2013) Image segmentation for large-scale subcategory flower recognition In Applications of Computer Vision (WACV), 2013 IEEE Workshop on, pp 39–45 IEEE [23] Arora A., Gupta A., Bagmar N., Mishra S., and Bhattacharya A (2012) A plant identification system using shape and morphological features on segmented leaflets: Team iitk, clef 2012 In CLEF (Online Working Notes/Labs/Workshop) [24] Bakic V., Mouine S., Ouertani-Litayem S., Verroust-Blondet A., Yahiaoui I., Goăeau H., and Joly A (2013) Inrias participation at imageclef 2013 plant identification task In CLEF (Online Working Notes/Labs/Workshop) 2013 109 [25] Cerutti G., Antoine V., Tougne L., Mille J., Valet L., Coquin D., and Vacavant A Reves participation-tree species classification using random forests and botanical features In Conference and Labs of the Evaluation Forum (CLEF) [26] Cerutti G., Tougne L., Mille J., Vacavant A., and Coquin D (2013) Understanding leaves in natural images–a model-based approach for tree species identification Computer Vision and Image Understanding, 117(10):pp 1482–1501 [27] Cerutti G., Tougne L., Mille J., Vacavant A., and Coquin D (2013) A modelbased approach for compound leaves understanding and identification In 2013 IEEE International Conference on Image Processing, pp 14711475 [28] Rzanny M., Seeland M., Wăaldchen J., and Măader P (2017) Acquiring and preprocessing leaf images for automated plant identification: understanding the tradeoff between effort and information gain Plant Methods, 13(1):pp 1–11 [29] Nilsback M.E and Zisserman A (2009) An automatic visual flora-segmentation and classification of flower images Ph.D thesis, Oxford University [30] Wang X.F., Du J.X., and Zhang G.J (2005) Recognition of leaf images based on shape features using a hypersphere classifier In International Conference on Intelligent Computing, pp 87–96 [31] Chen Q., Abedini M., Garnavi R., and Liang X (2014) Ibm research australia at lifeclef2014: Plant identification task In CLEF (Working Notes), pp 693–704 [32] Nakayama H (2013) Nlab-utokyo at imageclef 2013 plant identification task In CLEF (Working Notes) [33] Wang Z., Sun X., Zhang Y., Ying Z., and Ma Y (2016) Leaf recognition based on pcnn Neural Computing and Applications, 27(4):pp 899–908 [34] Jamil N., Hussin N.A.C., Nordin S., and Awang K (2015) Automatic plant identification: Is shape the key feature? Procedia Computer Science, 76:pp 436–442 [35] Mouine S., Yahiaoui I., and Verroust-Blondet A (2013) Combining leaf salient points and leaf contour descriptions for plant species recognition In International Conference Image Analysis and Recognition, pp 205–214 [36] Rodrigo R., Samarawickrame K., and Mindya S (2013) An intelligent flower analyzing system for medicinal plants In WSCG 2013 Conference on Computer Graphics, Visualization and Computer Vision, pp 41–44 V´aclav Skala-UNION Agency 110 [37] Angelova A., Zhu S., Lin Y., Wong J., and Shpecht C (2012) Development and deployment of a large-scale flower recognition mobile app NEC Labs America Technical Report [38] Aakif A and Khan M.F (2015) Automatic classification of plants based on their leaves Biosystems Engineering, 139:pp 66–75 [39] Du J.X., Wang X.F., and Zhang G.J (2007) Leaf shape based plant species recognition Applied mathematics and computation, 185(2):pp 883–893 [40] Du J.X., Shao M.W., Zhai C.M., Wang J., Tang Y., and Chen C.L.P (2016) Recognition of leaf image set based on manifold–manifold distance Neurocomputing, 188:pp 131–138 [41] Hu R., Jia W., Ling H., and Huang D (2012) Multiscale distance matrix for fast plant leaf recognition IEEE transactions on image processing, 21(11):pp 4667–4672 [42] Lee K.B and Hong K.S (2013) An implementation of leaf recognition system using leaf vein and shape International Journal of Bio-Science and Bio-Technology, 5(2):pp 57–66 [43] Bo L., Ren X., and Fox D (2010) Kernel descriptors for visual recognition In Advances in neural information processing systems, pp 244–252 [44] Le T.L., Tran D.T., and Pham N.H (2014) Kernel descriptor based plant leaf identification In Image Processing Theory, Tools and Applications (IPTA), 2014 4th International Conference on, pp 1–5 [45] Yoo H.J (2015) Deep convolution neural networks in computer vision IEIE Transactions on Smart Processing & Computing, 4(1):pp 35–43 [46] Krizhevsky A., Sutskever I., and Hinton G.E (2012) Imagenet classification with deep convolutional neural networks In Advances in neural information processing systems, pp 1097–1105 [47] Mythili C and Kavitha V (2014) Recognition of plant leaf in medicine Journal of Convergence Information Technology, 9(2):pp 61–69 [48] Ghasab M.A.J., Khamis S., Mohammad F., and Fariman H.J (2015) Feature decision-making ant colony optimization system for an automated recognition of plant species Expert Systems with Applications, 42(5):pp 2361–2370 [49] Hall D., McCool C., Dayoub F., Sunderhauf N., and Upcroft B (2015) Evaluation of features for leaf classification in challenging conditions In Applications of Computer Vision (WACV), 2015 IEEE Winter Conference on, pp 797–804 111 [50] Wang B., Brown D., Gao Y., and La Salle J (2015) March: Multiscale-archheight description for mobile retrieval of leaf images Information Sciences, 302:pp 132–148 [51] Zhao C., Chan S.S., Cham W.K., and Chu L (2015) Plant identification using leaf shapes—a pattern counting approach Pattern Recognition, 48(10):pp 3203– 3215 [52] Prasad S., Peddoju S.K., and Ghosh D (2016) An adaptive plant leaf mobile informatics using rssc Multimedia Tools and Applications, pp 1–25 ISSN 15737721 doi:10.1007/s11042-016-4040-8 [53] VijayaLakshmi B and Mohan V (2016) Kernel-based pso and frvm: An automatic plant leaf type detection using texture, shape, and color features Computers and Electronics in Agriculture, 125:pp 99–112 [54] Nilsback M.E and Zisserman A (2006) A visual vocabulary for flower classification In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, pp 1447–1454 [55] Nilsback M.E and Zisserman A (2008) Automated flower classification over a large number of classes In Computer Vision, Graphics & Image Processing, 2008 ICVGIP’08 Sixth Indian Conference on, pp 722–729 [56] Arivazhagan S., Shebiah R.N., Nidhyanandhan S.S., and Ganesan L (2010) Fruit recognition using color and texture features Journal of Emerging Trends in Computing and Information Sciences, 1(2):pp 90–94 [57] Huang Z.K (2006) Bark classification using rbpnn based on both color and texture feature International Journal of Computer Science and Network Security, 6(10):pp 100–103 [58] Zeiler M.D and Fergus R (2014) Visualizing and understanding convolutional networks In European conference on computer vision, pp 818–833 [59] Simonyan K and Zisserman A (2014) Very deep convolutional networks for large-scale image recognition CoRR, abs/1409.1556 [60] Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhoucke V., and Rabinovich A (2015) Going deeper with convolutions In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1–9 [61] He K., Zhang X., Ren S., and Sun J (2015) Deep residual learning for image recognition CoRR, abs/1512.03385 112 [62] http://cs231n.github.io/convolutional-networks/(retrievel 17/january/2018) [63] Yosinski J., Clune J., Bengio Y., and Lipson H (2014) How transferable are features in deep neural networks? In Advances in neural information processing systems, pp 3320–3328 [64] Russakovsky O., Deng J., Su H., Krause J., Satheesh S., Ma S., Huang Z., Karpathy A., Khosla A., Bernstein M., et al (2015) Imagenet large scale visual recognition challenge International Journal of Computer Vision, 115(3):pp 211 252 [65] Wăaldchen J., Rzanny M., Seeland M., and Măader P (2018) Automated plant species identification—trends and future directions PLoS computational biology, 14(4):p e1005993 [66] Thyagharajan K and Raji I.K (2018) A review of visual descriptors and classification techniques used in leaf species identification Archives of Computational Methods in Engineering, pp 1–28 [67] Cortes C and Vapnik V (1995) Support-vector networks Machine learning, 20(3):pp 273–297 [68] Weston J and Watkins C (1998) Multi-class support vector machines Technical report, Citeseer [69] Chang C.C and Lin C.J (2011) Libsvm: A library for support vector machines ACM transactions on intelligent systems and technology (TIST), 2(3):pp 27:1– 27:27 [70] Burges C.J (1998) A tutorial on support vector machines for pattern recognition Data mining and knowledge discovery, 2(2):pp 121–167 [71] Cerutti G., Tougne L., Sacca C., Joliveau T., Mazagol P.O., Coquin D., and Vacavant A (2013) Late information fusion for multi-modality plant species identification In Working notes for Conference and Labs of the Evaluation Forum [72] Choi S (2015) Plant identification with deep convolutional neural network: Snumedinfo at lifeclef plant identification task 2015 In CLEF (Working Notes) [73] Ghazi M.M., Yanikoglu B., and Aptoula E (2017) Plant identification using deep neural networks via optimization of transfer learning parameters Neurocomputing, 235:pp 228–235 [74] Ge Z., McCool C., Sanderson C., and Corke P (2015) Content specific feature learning for fine-grained plant classification In CLEF (Working Notes) 113 [75] Lee S.H., Chan C.S., Mayo S.J., and Remagnino P (2017) How deep learning extracts and learns leaf features for plant classification Pattern Recognition, 71:pp 1–13 [76] He A and Tian X (2016) Multi-organ plant identification with multi-column deep convolutional neural networks In 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC)2016 , pp 002020002025 ă and Ozdemir ă [77] Mehdipour Ghazi M., Yanıko˘glu B., Aptoula E., Muslu O., M.C (2015) Sabanci-okan system in lifeclef 2015 plant identification competition CLEF (Conference and Labs of the Evaluation Forum) [78] Ahmed A., Atito S., Yanıko˘glu B., and Aptoula E (2017) Plant identification with large number of classes: Sabanciu-gebzetu system in plantclef 2017 CLEF [79] Jovi´c M., Hatakeyama Y., Dong F., and Hirota K (2006) Image retrieval based on similarity score fusion from feature similarity ranking lists In International conference on Fuzzy Systems and Knowledge Discovery, pp 461–470 [80] Kittler J., Hatef M., Duin R.P., and Matas J (1998) On combining classifiers IEEE transactions on pattern analysis and machine intelligence, 20(3):pp 226– 239 [81] Makihara Y., Muramatsu D., Iwama H., and Yagi Y (2013) On combining gait features In 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp 1–8 IEEE [82] He M., Horng S.J., Fan P., Run R.S., Chen R.J., Lai J.L., Khan M.K., and Sentosa K.O (2010) Performance evaluation of score level fusion in multimodal biometric systems Pattern Recognition, 43(5):pp 1789–1800 [83] Mohamed I., Diane L., and Fr´ed´eric P (2014) Plant species recognition using bag-of-word with svm classifier in the context of the lifeclef challenge Working Notes of CLEF [84] Champ J., Lorieul T., Servajean M., and Joly A (2015) A comparative study of fine-grained classification methods in the context of the lifeclef plant identification challenge 2015 [85] Să underhauf N., McCool C., Upcroft B., and Perez T (2014) Fine-grained plant classification using convolutional neural networks for feature extraction In CLEF (Working Notes), pp 756–762 [86] Reyes A.K., Caicedo J.C., and Camargo J.E (2015) Fine-tuning deep convolutional networks for plant recognition In CLEF (Working Notes) 114 [87] Dimitrovski I., Madjarov G., Kocev D., and Lameski P (2014) Maestra at lifeclef 2014 plant task: Plant identification using visual data In CLEF (Working Notes), pp 705–714 ˇ [88] Sulc M and Matas J (2017) Learning with noisy and trusted labels for finegrained plant recognition Working notes of CLEF [89] T´oth B.P., T´oth M.J., Papp D., and Sză ucs G (2016) Deep learning and svm classification for plant recognition in content-based large scale image retrieval In CLEF (Working Notes), pp 569–578 [90] Sz˝ ucs G., Papp D., and Lovas D (2014) Viewpoints combined classification, method in image-based plant identification task In Working Notes for CLEF 2014 Conference, volume 1180, pp 763770 [91] Affouard A., Goăeau H., Bonnet P., Lombardo J.C., and Joly A (2017) Pl@ntnet app in the era of deep learning In ICLR 2017 Workshop Track-5th International Conference on Learning Representations, pp 1–6 [92] Zhu H., Huang X., Zhang S., and Yuen P.C (2017) Plant identification via multipath sparse coding Multimedia Tools and Applications, 76(3):pp 4599– 4615 [93] Lee S.H., Chang Y.L., and Chan C.S (2017) Lifeclef 2017 plant identification challenge: Classifying plants using generic-organ correlation features Working Notes of CLEF , 2017 [94] Le T.L and Pham N.H (2013) Mica at imageclef 2013 plant identification task In CLEF (Working Notes) [95] Pham N.H., Le T.L., Grard P., and Nguyen V.N (2013) Computer aided plant identification system In Computing, Management and Telecommunications (ComManTel), 2013 International Conference on, pp 134–139 [96] Nguyen Q.K., Le T.L., and Pham N.H (2013) Leaf based plant identification system for android using surf features in combination with bag of words model and supervised learning In 2013 International Conference on Advanced Technologies for Communications (ATC 2013), pp 404–407 [97] Phan Thi Thu Hong Doan Thi Thu Ha N.T.T Ung dung phan lop anh chup la cay bang phuong phap may vector ho tro (leaf image classification using support vector machine) Journal of Science and Development, 11:pp 1045–1052 [98] Nguyen Van Huan N.V.T (2015) Ky thuat tra cuu anh cay duoc lieu dua vao noi dung phuc vu cho phat hien, quan ly va khai thac (technique of searching 115 medicinal plants based on content for detection, management and exploitation) In Proceedings of The 2015 National Conference on Electronics, Communications and Information Technology (ECIT 2015), pp 353–357 [99] Anh H V., Hoa.T D., Bao.T N., and Van-Huy P (2019) Vietnamese herbal plant recognition using deep convolutional features International Journal of Machine Learning and Computing, 9:pp 363–367 [100] Duong-Trung N., Quach L.D., and Nguyen C.N (2019) Learning deep transferability for several agricultural classification problems International Journal of Advanced Computer Science and Applications, 10(1):pp 58–67 [101] Thanh T.K.N., Truong Q.B., Truong Q.D., and Xuan H.H (2018) Depth learning with convolutional neural network for leaves classifier based on shape of leaf vein In Asian Conference on Intelligent Information and Database Systems, pp 565– 575 [102] Hoang Le Uyen Thuc Nguyen Van Duc H.T.T.L.V.T.L.T.M.H (2017) Tu dong nhan dang mot so loai hoa canh bien o viet nam dua vao ky thuat thi giac may tinh (automatically identify some popular ornamental flowers in vietnam based on computer vision technique) In Proceedings of The XX National Conference on Selected issues of information and communication technology), pp 366–371 [103] Nguyen Dac Thanh Le Thanh Ha T.Q.L (2017) Nhan dang va phan loai hoa qua anh mau (identification and classification fruits in color images): Thesis of software engineering master: 60480103 [104] Joly A., Goăeau H., Bonnet P., Spampinato C., Glotin H., Rauber A., Vellinga W.P., Fisher R., and Mă uller H (2014) Are species identification tools biodiversity-friendly? In Proceedings of the 3rd ACM International Workshop on Multimedia Analysis for Ecological Data, pp 3136 [105] Goăeau H., Bonnet P., Joly A., Baki´c V., Barbe J., Yahiaoui I., Selmi S., Carr´e J., Barth´el´emy D., Boujemaa N., et al (2013) Pl@ntnet mobile app In Proceedings of the 21st Association for Computing Machinery(ACM) international conference on Multimedia, pp 423–424 [106] Goăeau H., Bonnet P., Joly A., Affouard A., Bakic V., Barbe J., Dufour S., Selmi S., Yahiaoui I., Vignau C., et al (2014) Pl@ntnet mobile 2014: Android port and new features In Proceedings of international conference on multimedia retrieval , p 527 [107] http://www.inaturalist.org/(retrieved 15/january/2017) 116 [108] Scanlon E., Woods W., and Clow D (2014) Informal participation in science in the uk: Identification, location and mobility with ispot Educational Technology & Society, 17(2):pp 58–71 [109] Silvertown J., Harvey M., Greenwood R., Dodd M., Rosewell J., Rebelo T., Ansine J., and McConway K (2015) Crowdsourcing the identification of organisms: A case-study of ispot ZooKeys, 480:p 125 [110] https://itunes.apple.com/nz/app/flora-finder/id688613607?mt=8(retrieved 15/may/2017) [111] http://www.plantifier.com(retrieved 15/may/2017) [112] Nilsback M.E and Zisserman A (Dec 2008) Automated flower classification over a large number of classes In 2008 Sixth Indian Conference on Computer Vision, Graphics Image Processing, pp 722729 doi:10.1109/ICVGIP.2008.47 [113] Joly A., Goăeau H., Glotin H., Spampinato C., Bonnet P., Vellinga W.P., Planqu´e R., Rauber A., Palazzo S., Fisher B., et al (2015) Lifeclef 2015: multimedia life species identification challenges In International Conference of the CrossLanguage Evaluation Forum for European Languages, pp 462–483 [114] Prasad S., Peddoju S.K., and Ghosh D (2017) Efficient plant leaf representations: A comparative study In Region 10 Conference, TENCON 2017-2017 IEEE , pp 1175–1180 [115] Wu S.G., Bao F.S., Xu E.Y., Wang Y.X., Chang Y.F., and Xiang Q.L (2007) A leaf recognition algorithm for plant classification using probabilistic neural network In 2007 IEEE international symposium on signal processing and information technology, pp 11–16 [116] Gu X., Du J.X., and Wang X.F (2005) Leaf recognition based on the combination of wavelet transform and gaussian interpolation In International Conference on Intelligent Computing, pp 253–262 [117] Cerutti G., Tougne L., Mille J., Vacavant A., and Coquin D Guiding active contours for tree leaf segmentation and identification In CLEF 2011, Conference on Multilingual and Multimodal Information Access Evaluation [118] Bo L and Sminchisescu C (2009) Efficient match kernel between sets of features for visual recognition In Advances in neural information processing systems, pp 135–143 [119] NGUYEN V.T (2015) Visual interpretation of hand postures for humanmachine interaction Ph.D thesis, Universit´e de La Rochelle 117 [120] Achanta R., Hemami S., Estrada F., and Susstrunk S (2009) Frequency-tuned salient region detection In Computer vision and pattern recognition, 2009 cvpr 2009 ieee conference on, pp 1597–1604 [121] Meyer F and Beucher S (1990) Morphological segmentation Journal of visual communication and image representation, 1(1):pp 21–46 [122] Beucher S and Meyer F (1993) The morphological approach to segmentation: the watershed transformation Mathematical morphology in image processing, 34:pp 433–481 [123] Kornilov A and Safonov I (2018) An overview of watershed algorithm implementations in open source libraries Journal of Imaging, 4(10):pp 1–15 [124] Le T.L., Tran D.T., and Hoang V.N (2014) Fully automatic leaf-based plant identification, application for vietnamese medicinal plant search In Proceedings of the fifth symposium on information and communication technology, pp 146– 154 [125] Dey S., Bhattacharya B.B., Kundu M.K., and Acharya T.A (2002) A simple architecture for computing moments and orientation of an image Fundamenta Informaticae, 52(4):pp 285–295 [126] Lowe D.G (1999) Object recognition from local scale-invariant features In Computer vision, 1999 The proceedings of the seventh IEEE international conference on, volume 2, pp 1150–1157 [127] Maji S., Berg A.C., and Malik J (2013) Efficient classification for additive kernel svms IEEE transactions on pattern analysis and machine intelligence, 35(1):pp 66–77 [128] Vedaldi A and Zisserman A (2012) Efficient additive kernels via explicit feature maps IEEE transactions on pattern analysis and machine intelligence, 34(3):pp 480–492 [129] Chaki J., Parekh R., and Bhattacharya S (2015) Recognition of whole and deformed plant leaves using statistical shape features and neuro-fuzzy classifier In 2015 IEEE 2nd international conference on recent trends in information systems (ReTIS), pp 189–194 [130] Chaki J., Parekh R., and Bhattacharya S (2015) Plant leaf recognition using texture and shape features with neural classifiers Pattern Recognition Letters, 58:pp 61–68 118 [131] Wang Z., Sun X., Ma Y., Zhang H., Ma Y., Xie W., and Zhang Y (2014) Plant recognition based on intersecting cortical model In 2014 International joint conference on neural networks (IJCNN), pp 975–980 [132] Kheirkhah F.M and Asghari H (2018) Plant leaf classification using gist texture features IET Computer Vision, 13(4):pp 369–375 [133] Tsolakidis D.G., Kosmopoulos D.I., and Papadourakis G (2014) Plant leaf recognition using zernike moments and histogram of oriented gradients In Hellenic Conference on Artificial Intelligence, pp 406–417 [134] Du J.x., Zhai C.M., and Wang Q.P (2013) Recognition of plant leaf image based on fractal dimension features Neurocomputing, 116:pp 150–156 [135] Priya C.A., Balasaravanan T., and Thanamani A.S (2012) An efficient leaf recognition algorithm for plant classification using support vector machine In International conference on pattern recognition, informatics and medical engineering (PRIME-2012), pp 428–432 [136] Jain A., Nandakumar K., and Ross A (2005) Score normalization in multimodal biometric systems Pattern recognition, 38(12):pp 2270–2285 [137] Wang F and Han J (2009) Multimodal biometric authentication based on score level fusion using support vector machine Opto-electronics review , 17(1):pp 59–64 [138] Abdiansah A and Wardoyo R (2015) Time complexity analysis of support vector machines (svm) in libsvm International journal computer and application, 128(3):pp 28–34 [139] Singh A and Kisku D.R (2019) Detection of rare genetic diseases using facial 2d images with transfer learning In 2018 8th International Symposium on Embedded Computing and System Design (ISED), pp 26–30 [140] Van Horn G., Mac Aodha O., Song Y., Cui Y., Sun C., Shepard A., Adam H., Perona P., and Belongie S (2018) The inaturalist species classification and detection dataset In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8769–8778 [141] Gaston K.J and O’Neill M.A (2004) Automated species identification: why not? Philosophical transactions of the Royal society B: Biological sciences, 359(1444):pp 655667 [142] Săoderkvist O (2001) Computer vision classification of leaves from swedish trees Master’s thesis, Linkăoping University 119 [143] Mouine S., Yahiaoui I., and Verroust-Blondet A (2012) Advanced shape context for plant species identification using leaf image retrieval In Proceedings of the 2nd (Association for Computing Machinery)ACM international conference on multimedia retrieval , p 49 [144] Novotn` y P and Suk T (2013) Leaf recognition of woody species in central europe Biosystems engineering, 115(4):pp 444–452 [145] Mzoughi O., Yahiaoui I., and Boujemaa N (2012) Petiole shape detection for advanced leaf identification In 2012 19th IEEE International Conference on Image Processing, pp 1033–1036 [146] Lee S.H., Chan C.S., Wilkin P., and Remagnino P (2015) Deep-plant: Plant identification with convolutional neural networks In Image Processing (ICIP), 2015 IEEE International Conference on, pp 452–456 [147] Seeland M., Rzanny M., Alaqraa N., Wăaldchen J., and Măader P (2017) Plant species classification using flower images—a comparative study of local feature representations PloS one, 12(2):p e0170629 [148] Deng J., Dong W., Socher R., Li L.J., Li K., and Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database In Computer Vision and Pattern Recognition, 2009 CVPR 2009 IEEE Conference on, pp 248255 [149] Goăeau H., Joly A., Yahiaoui I., Bakic V., Verroust-Blondet A., Bonnet P., Barth´el´emy D., Boujemaa N., and Molino J.F (2014) Plantnet participation at lifeclef2014 plant identification task In CLEF2014 Working Notes, pp 724– 737 [150] Jia Y., Shelhamer E., Donahue J., Karayev S., Long J., Girshick R., Guadarrama S., and Darrell T (2014) Caffe: Convolutional architecture for fast feature embedding In Proceedings of the 22nd ACM international conference on Multimedia, pp 675–678 [151] Zintgraf L.M., Cohen T.S., Adel T., and Welling M (2017) Visualizing deep neural network decisions: Prediction difference analysis CoRR, abs/1702.04595 [152] Le Thi Lan Hoang Van Nam V.H.T.T.T.H (2014) Vietnamese medicinal plant retrieval system for android In The 17th nationalconference: selected problems about IT and Telecommunication, Taynguyen, Vietnam ISBN 978-604-67-04263 [153] Loi D.T (2014) Nhung cay thuoc va vi thuoc Viet Nam (The medicinal plants and herbal formulation in Vietnam) Y hoc 120 PUBLICATIONS [1] Thi-Lan Le, Duong-Nam Duong, Van-Toi Nguyen, Hai Vu, Van-Nam Hoang and Thi Thanh-Nhan Nguyen, (2015) Complex Background Leaf-based Plant Identification Method Based on Interactive Segmentation and Kernel Descriptor, Proceedings of the 2nd International Workshop on Environmental Multimedia Retrieval, ISBN: 978-1-4503-3274-3, pp.3-8 [2] Thi-Lan Le, Duong-Nam Duong, Hai Vu and Thanh-Nhan Nguyen (2015) Mica at lifeclef 2015: Multi-organ plant identification, In CEUR-WS.org/Vol-1391CLEF2015 Working note proceedings, ISSN: 1613-0073, vol 1391 [3] Thi Thanh Nhan Nguyen, Van Tuan Le, Thi Lan Le, Hai Vu, Natapon Pantuwong and Yasushi Yagi (2016), Flower species identification using deep convolutional neural networks, AUN/SEED-Net Regional Conference and Information Engineering 2016, Yangon, Myanmar, ISBN: 978-99971-0-231-7, pp.51-56 [4] Thi Thanh-Nhan Nguyen, Thi-Lan Le, Hai Vu, Huy-hoang Nguyen and VanSam Hoang (2017), A combination of Deep Learning and Hand-Designed Feature for Plant Identification Based on Leaf and Flower, In Asian Conference on Intelligent Information and Database Systems, Studies in Computational Intelligence, volume 710, Springer, ISBN: 978-3-319-56659-7, pp 223-233 [5] Nguyen Thi Thanh Nhan, Do Thanh Binh, Nguyen Huy Hoang, Vu Hai, Tran Thi Thanh Hai, Thi-Lan Le (2018), Score-based Fusion Schemes for Plant Identification from Multi-organ Images, VNU Journal of Science: Computer Science and Communication Engineering, Vol 34, No.2, ISSN 2588-1086, pp.1-15 [6] Thi Thanh Nhan Nguyen, Thi-Lan Le, Hai Vu, Van-Sam Hoang, Thanh-Hai Tran (2018), Crowdsourcing for botanical data collection towards to automatic plant identification: A review, Journal of Computers and Electronics in Agriculture (SCIE), vol 155, ISSN:0168-1699, pp.412-425 [7] Nguyen Thi Thanh Nhan, Le Thi Lan, Vu Hai, Hoang Van Sam (2018), Automatic Plant Organ Detection from Images using Convolutional Neural Networks, Journal of Research and Development on Information and Communication Technology (in Vietnamese), vol V-1, No 39, ISSN: 1859-3526, pp 17-25 [8] Thi Thanh-Nhan Nguyen, Thi-Lan Le, Hai Vu, Van-Sam Hoang (2019), Towards an automatic plant identification system without dedicated dataset International Journal of Machie Learning and Computing (Scopus), vol 9, No.1, ISSN: 2010-3700, pp.26-34 APPENDIX A Table A.1 Details of 50 species used in chapter ID Species name #Leaf #Flower #Branch #Entire 54 Cotinus coggygria Scop 91 38 43 18 55 Pistacia lentiscus L 119 33 51 12 151 Daucus carota L 22 157 23 64 326 Ilex aquifolium L 103 55 49 23 329 Hedera helix L 244 46 58 44 333 Aristolochia clematitis L 40 31 46 26 505 Bidens pilosa L 39 43 35 24 702 Erigeron sumatrensis Retz 56 36 37 42 1212 Silybum marianum (L.) Gaertn 46 54 48 44 10 1321 Alnus glutinosa (L.) Gaertn 60 74 91 14 11 1325 Betula pendula Roth 88 34 64 38 12 1328 Carpinus betulus L 67 47 66 14 13 1329 Corylus avellana L 158 65 61 21 14 1837 Buddleja davidii Franch 39 77 47 24 15 1845 Opuntia ficus-indica (L.) Mill 64 55 23 29 16 1966 Lonicera xylosteum L 48 50 75 14 17 1968 Sambucus nigra L 74 78 37 20 18 1972 Viburnum opulus L 40 39 46 13 19 1973 Viburnum tinus L 179 109 64 16 20 2631 Diospyros kaki L.f 73 49 30 19 21 2648 Arbutus unedo L 58 84 72 24 22 2776 Amorpha fruticosa L 56 57 54 20 23 2841 Ceratonia siliqua L 49 29 50 31 24 2918 Gleditsia triacanthos L 40 30 87 16 25 3124 Robinia pseudoacacia L 89 90 56 20 26 3279 Castanea sativa Mill 52 55 66 17 27 3283 Quercus coccifera L 59 40 71 22 28 3288 Quercus ilex L 162 106 80 49 29 3435 Geranium robertianum L 33 80 44 55 122 30 3465 Aesculus hippocastanum L 81 105 51 26 31 3506 Juglans regia L 53 36 62 27 32 3750 Laurus nobilis L 156 84 78 17 33 3797 Liriodendron tulipifera L 82 40 48 42 34 3798 Magnolia grandiflora L 71 85 60 14 35 3801 Alcea rosea L 43 77 48 23 36 3831 Malva sylvestris L 63 200 80 32 37 3846 Broussonetia papyrifera (L.) Vent 76 39 104 17 38 3849 Morus alba L 104 19 52 113 39 3942 Fraxinus angustifolia Vahl 47 45 100 20 40 3956 Olea europaea L 92 32 47 41 41 4026 Chelidonium majus L 58 83 42 63 42 4074 Pittosporum tobira (Thunb.) W.T.Aiton 127 75 52 41 43 4094 Plantago lanceolata L 39 64 10 82 44 4109 Platanus x hispanica Mill ex Mă unchh 99 42 47 33 45 4111 Platanus orientalis L 57 27 35 21 46 4721 Crataegus monogyna Jacq 135 151 79 21 47 5522 Veronica persica Poir 14 114 56 60 48 14869 Ulex europaeus L 66 93 44 50 49 14887 Verbascum thapsus L 94 110 50 80 50 30131 Euphorbia amygdaloides L 65 47 55 65 Total 3870 3309 2774 1661 123