Building a diagram recognition application with computer vision approach

VIETNAM NATIONAL UNIVERSITY - HO CHI MINH CITY HO CHI MINH UNIVERSITY OF TECHNOLOGY COMPUTER SCIENCE AND ENGINEERING FACULTY ——————– * ——————— GRADUATION THESIS Building A Diagram Recognition Application with Computer Vision Approach Committee: Computer Science Advisor: Dr Nguyen Duc Dung Reviewer: Dr Tran Tuan Anh —–o0o—– Students: Huynh Tan Thanh 1752048 Nguyen Quang Sang 1752465 HO CHI MINH CITY, 12/2021 - - KHOA: KH & KT Máy tính KHMT QUANG SANG NGÀNH: KHMT MSSV: 1752465 MSSV: 1752048 CLC-K2017 (Building A Diagram Recognition Application with Computer Vision Approach) - Investigate approaches in computer vision for diagram recognition problem - Design the framework and the processing pipeline for the diagram recognition system - Collect data and perform labeling tasks on the data - Implement the recognition model, which uses both the DL approach and traditional Computer vision algorithms in the pipeline - Implement the mobile application - Evaluating the application and performance of the proposed system 1) TS , Khoa KH&KT Máy tính KHOA KH & KT MÁY TÍNH MSSV: 1752465, 1752048 Ngành (chuyên ngành): KHMT (Building A Diagram Recognition Application with Computer Vision Approach) H - - - The students proposed a solution for the diagram recognition problem, which utilizes the advantages of computer vision techniques and machine learning approaches In addition, the students have successfully built a mobile application that allows users to interact with the system easier The application was built with useful features and easy to use interface The students also did a lot of evaluation as well as proposed some improvement in the recognition algorithm Some algorithms used in the project are not so advanced and may not be able to handle some difficult cases in the problem The evaluation results are promising but still need to improve further, especially when investigating various cases in recognition   ฀ a b c 9/10 KHOA KH & KT MÁY TÍNH -Ngày 27 tháng 12 2021 Nguy n Quang Sang, Hu nh T n Th nh MSSV: 1752465, 1752048 Ngành (chuyên ngành): Khoa H áy Tính Building A Diagram Recognition Application with Computer Vision Approach Tr - The thesis presents a system that can convert handwritten flowcharts into digital documents - The system is built quite full of features and has a good application demo - This thesis has quite a large amount of work including recognizing shapes, handwriting, arrows and building demo app - The thesis has experiments and is quite fully cited This thesis also presents quite detailed algorithms and models - This application requires many techniques combined, leading to a lot of work in many technique areas This is also one of the weaknesses of the thesis when the research works on the topic have not been strongly developed For example, the handwriting entry The team can focus on developing a few key techniques instead of all of them, the rest can use existing results - The evaluation parameters are not detailed and user-oriented, for example, is the assessment of the arrow considered fair for all arrow types? - The data used to train the model is not clearly presented The application should explore more about usability, adapting to the user, instead of just focusing on general accuracy - Models should be analyzed in more detail, rather than just using it    a The evaluation methods proposed in this thesis is effective? For example, is the assessment of the arrow considered fair for all arrow types? is there any general evaluation for the application? b What are the main strengths of this thesis? Also, what is the main point that users should use your app? c What is your next research priority? Gi 8.7 /10 Tr Declaration We hereby undertake that this is our own research project under the guidance of Dr Nguyen Research content and results are truthful and have never been published before The data used for the analysis and comments are collected by us from many different sources and will be clearly stated in the references Additionally, a number of reviews and figures of other authors and organizations we use will have citations and origins clearly stated in the report If we detect any fraud, we take full responsibility for the content of our graduation thesis Ho Chi Minh City University of Technology is not related to the copyright and copyright infringement caused by us in the implementation process Nguyen Quang Sang Huynh Tan Thanh Acknowledgments We would like to express my deepest thanks to Dr.Nguyen Duc Dung for the continuous support in studying and implementing this thesis This project would not have been possible without your thoughtful and passionate guides Besides our advisor, we would also like to thank all of our faculty lecturers, who gave us the valuable knowledge to this wonderful project With the invaluable experience from this golden opportunity, we became more confident in our research ability and technical skills We strongly believe that there is no perfection, especially in the science field With that in mind, we will always have room for more enhancement and would love to hear your opinion about any improvement Best regards, Nguyen Quang Sang Huynh Tan Thanh Abstract Graphical language has been and is always one of the most effective tools for demonstrating ideas to others Besides text and images, a flow chart plays a vital role in providing people a clearer view of a plan, or a process with simple symbols, notations Nowadays, many meetings still enjoy the traditional way by using board, paper to draw diagrams expressing their thoughts on the topics discussed A problem occurs when saving these drawings as a reference for future purposes since we cannot edit the diagram taken from the picture These drawn pictures need to be re-drawn by some tools to be suitable in professional documents In addition, the re-drawn tool can be a computer or a particular device like electronics drawing boards and digital pens, which cost a lot and is not the most convenient tools to use Therefore, a new approach is necessary to convert hand-drawing charts pictures into digital ones The approach can help us avoid re-drawn tasks, simplify the sharing process between users, and be able to export them into another form like picture files (png, jpg), document files (pdf), or standard diagram editing files (drawio) The application must be able to run on popular platforms and accessible to everyone Contents Introduction 1.1 Overview 1.2 Project Goals 1 2 Related works 2.1 Related Applications 2.1.1 Object recognition 2.1.2 Diagram tools 2.1.3 Diagram recognition applications on mobile devices 2.2 Diagram recognition 2.3 Handwriting Text recognition 2.3.1 Preprocessing Phase 2.3.2 Recognition Phase 4 4 5 7 Background 3.1 Faster R-CNN 3.1.1 Backbone CNN 3.1.2 Regional Proposal Network 3.1.3 Non-Maximum Suppression 3.1.4 Region of Interest Pooling (RoI Pooling) 3.2 Mask R-CNN 3.2.1 Object Mask (Binary Mask) 3.2.2 Feature Pyramid Network 3.2.3 Region of Interest Align (RoI Align) 3.3 Handwriting Text Recognition 3.3.1 Long Short Term Memory (LSTM) 3.3.2 Gated Recurrent Unit (GRU) 3.3.3 Bidirectional RNN (BRNN) 3.3.4 Connectionist Temporal Classification (CTC) 12 12 12 13 14 16 16 17 18 18 19 19 20 21 22 Proposed model 4.1 Diagram Recognition Approach 4.1.1 Preparing diagram dataset 4.1.2 Recognition model 4.1.2.1 Feature map generator 4.1.2.2 Proposal generator 4.1.2.3 Instance generator 4.1.3 Diagram building 4.1.4 Symbol-Arrow relationship 25 25 25 28 28 29 31 32 33 i 4.2 4.3 4.1.5 The relationship of text Handwriting Text Recognition Approach Digital diagram output format System design 5.1 Requirements 5.1.1 Functional requirement 5.1.2 Nonfunctional requirement 5.1.3 Hardware requirement 5.2 System Architecture 5.3 Framework 5.3.1 Flutter 5.3.2 Nodejs 5.4 Database Design 5.4.1 Diagram File Design 5.5 Feature design 5.5.1 Usecase Design 5.5.2 Login/Register Screen 5.5.3 Diagram List 5.5.4 Create diagram 5.5.4.1 Diagram Scanning 5.5.4.2 Create from blank 5.5.5 Diagram Editing 5.5.6 Exporting 5.5.6.1 Converting to drawio files 5.5.7 Member Management 5.5.8 Version and History 36 36 37 40 40 40 41 41 42 42 42 43 44 44 45 45 46 47 47 47 48 49 53 54 54 54 Experiments 6.1 Initial experiments 6.1.1 Preprocessing 6.1.2 Recognition 6.2 Experiments on the recognition pipeline 6.2.1 Perform training and evaluation on HTR model 6.2.2 Perform training and evaluation on diagram recognition model 6.2.3 Perform experiments on the combination of diagram recognition model and HTR model 6.3 Display diagram on device 6.3.1 Interactive Viewer and Matrix4 6.3.2 Rendering diagram recognition on device 55 55 55 56 58 58 59 Conclusion and Future Work 7.1 Conclusion 7.2 Challenges 7.3 Future work 68 68 68 69 A Usecase detail 70 59 63 63 63 USER INTERFACE DESIGN B.3.2 Zoom View (a) Full zoom (b) Zoom 100 (a) Zoom 75 (b) Zoom 50 Figure B.11: Zoom options 92 USER INTERFACE DESIGN B.3.3 Edit Option Edit bar - Edit bar - Change vertex content Adjust content size 93 USER INTERFACE DESIGN Vertex type options Zoom out to add line between vertices Change vertex content Adjust content size 94 USER INTERFACE DESIGN Vertex background - main color Vertex background - shade color Return without save warning Save sheet 95 USER INTERFACE DESIGN B.4 Diagram history (a) History and Comment (b) Version View 96 Appendix C Testing C.1 Login and register ID Description TCLR001 Login with correct system account TCLR002 TCLR003 TCLR004 Login with incorrect system account Register new system account Register with an existed account username Step • Open the app and choose login tab • Input correct username and password • Click login Expected Result Pass User is logged in and the app go to home screen Passed • Open the app and choose login tab • Input incorrect username and password • Click login System rejects inputted account and show incorrect username or password error message Passed • Open the app and choose register tab • Input new username and password • Click register An new account is created User is logged in and the app go to home screen Passed • Open the app and choose register tab • Input username that has already been in database and password • Click register System rejects inputted account and shows duplicated username error message Passed 97 TESTING C.2 Home screen options The below test cases assume that user has already logged in and is in home screen ID TCH001 TCH002 TCH003 TCH004 TCH005 TCH006 TCH007 Description Find a diagram in home screen Create new diagram Preview a diagram Add a comment Preview a previous version Delete a diagram Export to image Step • Input a first few character of the diagram name • Click on create diagram button on home screen • Input name and confirm • Create the new diagram and choose save at the end • Input version message and confirm Expected Result A filter list of diagram is shown on home screen Pass Passed A new diagram is created and saved in the database User is pushed back to the home screen Passed • Click on the first diagram • Click on the diagram in the option box User is moved to the diagram preview screen Passed • Click on the first diagram • Click on the “History“ button in the option box • Input new comment and send A new comment is posted top the latest version Passed The first version of the diagram is shown Passed The diagram deleted Passed • Click on the first diagram • Click on the “History“ button in the option box • Click on the arrow next to the name of the first version at the bottom of the list • Choose a diagram on home screen • Click on “Edit diagram“ in the option box and confirm • Click on the first diagram • Click on the “Export“ button in the option box • Choose “PNG“ type • Click “Export“ is A new image of the diagram is created and share panel pops up Passed 98 TESTING TCH008 TCH009 TCH010 TCH011 TCH012 Export to PDF and save to storage Add a member Remove member a Scan a diagram from the camera Scan a diagram from an iamge • Click on the first diagram • Click on the “Export“ button in the option box • Choose “PDF“ type • Enable “Save to storage“ • Click “Export“ • Click on the first diagram • Click on the “Management“ button in the option box • Input member email • Choose “Editor“ role • Confirm • Click on the first diagram • Click on the “Management“ button in the option box • Click delete button of the first member • Choose the “Scan with camera“ button in the home screen • Take a picture • Adjust the diagram with marks on the screen and confirm • Confirm the final diagram picture • Choose the “From Gallery“ button in the home screen • Choose a picture • Adjust the diagram with marks on the screen and confirm • Confirm the final diagram picture A new pdf file of the diagram is created and save in the storage Then, the share panel pops up Passed The member is added to the project as an editor Passed The first member is removed from the project Passed The picture scanned and result is shown is the Passed The picture scanned and result is shown is the Passed 99 TESTING C.3 Editing The below test cases assume that user has already logged in and is in editing screen ID TCE001 TCE002 TCE003 TCE004 TCE005 TCE006 TCE007 Description Rename diagram Create vertex the new Delete a vertex Undo Redo Move vertex Move vertex out of the diagram Step • Choose a diagram on home screen • Click on Edit diagram in the option box • Click on diagram name on the app bar • Inputs name and confirm Then chooses save • Inputs version message and confirm Expected Result Pass A new version of the diagram is created and saved in the database with new name User is pushed back to the home screen Passed A new vertex is created on the diagram Passed • Clicks on a vertex on the diagram • Chooses delete button The selected vertex is deleted Passed • Creates a new vertex on the diagram • Chooses undo on the app bar The new vertex is deleted Passed the deleted vertex is recreated Passed • Drag a vertex to the center the selected vertex moves accordingly Passed • Drag a vertex to the top left and out of the diagram the selected vertex moves to the top left and stop Passed • Clicks on the vertex button on editing option list • Drags a new vertex to the diagram • Creates a new vertex on the diagram • Chooses undo on the app bar • Chooses redo on the app bar 100 TESTING ID Description TCE008 Change vertex size TCE009 Change vertex color TCE010 Change vertex border color TCE011 Change vertex content TCE012 TCE013 Change vertex content text size Delete arrow Step • Clicks on a vertex on the diagram • Chooses size button on the vertex option • Adjust height and width to 50% • Clicks on a vertex on the diagram • Chooses color button on the vertex option • Chooses red color • Choose first color shade • Clicks on a vertex on the diagram • Chooses border color button on the vertex option • Chooses red color • Choose first color shade • Double tap on a vertex on the diagram • Input “test“ and save • Clicks on a vertex on the diagram • Chooses text size button on the vertex option • Adjust height and width to 50% • Clicks on a vertex on the diagram • Chooses content color button on the vertex option • Chooses red color • Choose first color shade Expected Result Pass The vertex size is change accordingly Passed The vertex color change to red Passed The vertex color change to red Passed The vertex content is change to “test“ Passed The vertex text size is change accordingly Passed The vertex color change to red Passed 101 TESTING ID Description TCE014 Create new arrow TCE015 TCE016 TCE017 TCE018 Change arrow connect point Change arrow direction Change arrow color Change vertex type Step • Create two vertices • Clicks on the first vertex on the diagram • Choose add arrow button on the vertex option • Choose the second vertex on the diagram • Create two vertices and an arrow • Click on the arrow • Choose a new connect point on the first vertex • Create two vertices and an arrow • Click on the arrow • Choose switch direction button on the vertex option • Create two vertices and an arrow • Click on the arrow • Choose delete arrow button on the vertex option • Clicks on a vertex on the diagram • Chooses type button on the vertex option • Select the second one Expected Result Pass A new arrow is created connecting the selected vertices Passed The arrow connect point changes accordingly Passed The arrow direction reverses Passed The arrow is deleted Passed The vertex type change accordingly Passed 102 References [1] M Bresler, T V Phan, D Prusa, M Nakagawa, and V Hlavac Recognition system for on-line sketched diagrams Proceedings of the 14th International Conference on Frontiers in Handwriting Recognition (ICFHR2014), page 563–568, 2014 [2] M Bresler, D Prusa, and V Hlavac Detection of arrows in on-line sketched diagrams using relative stroke positioning Winter Conference on Applications of Computer Vision, page 610–617, 2015 [3] J.-P Valois, M Cote, and M Cheriet Online recognition of sketched electrical diagrams Proceedings of Sixth International Conference on Document Analysis and Recognition, 2001 [4] G Feng, C Viard-Gaudin, and Z Sun On-line hand-drawn electric circuit diagram recognition using 2d dynamic programming Pattern Recognition, page 3215–3223, 2009 [5] A Lemaitre, H Mouchere, J Camillerapp, and B Couasnon Interest of syntactic knowledge for on-line flowchart recognition Graphics Recognition New Trends and Challenges, page 89–98, 2013 [6] C Wang, H Mouchere, C Viard-Gaudin, and L Jin Combined segmentation and recognition of online handwritten diagrams with high order markov random field International Conference on Frontiers in Handwriting Recognition(ICFHR), page 252–257, 2016 [7] C Wang, H Mouchere, and C Viard-Gaudin Online flowchart understanding by combining max-margin markov random field with grammatical analysis International Journal on Document Analysis and Recognition(IJDAR), page 123–136, 2017 [8] M Bresler, D Prusa, and V Hlavac Modeling flowchart structure recognition as a maxsum problem International Conference on Document Analysis and Recognition(ICDAR), page 1215–1219, 2013 [9] M Bresler, D Prusa, and V Hlavac Online recognition of sketched arrow-connected diagrams International Journal on Document Analysis and Recognition, page 253–267, 2016 [10] M Bresler, D Prusa, and V Hlavac Recognizing off-line flowcharts by reconstructing strokes and using on-line recognition techniques International Conference on Frontiers in Handwriting Recognition, page 48–53, 2016 [11] F D Julca-Aguilar and N S T Hirata Symbol detection in online handwritten graphics using faster r-cnn International Workshop on Document Analysis Systems, page 151156, 2018 [12] Bernhard Schăafer, Margret Keuper, and Heiner Stuckenschmidt Arrow r-cnn for handwritten diagram recognition International Journal on Document Analysis and Recognition, pages 1–5, 2021 [13] Ram Sagar These machine learning techniques make google lens a success https:// analyticsindiamag.com/these-machine-learning-techniques-make-googlelens-a-success/, 2019 103 REFERENCES [14] Microsoft Microsoft math solver https://www.microsoft.com/vi-VN/ai/ai-labmicrosoft-math [15] Google Play Store Aipoly vision https://play.google.com/store/apps/details? id=com.aipoly.vision&hl=en&gl=US, April 17, 2017 [16] Google Play Store Vivino: Buy the right wine https://play.google.com/store/ apps/details?id=vivino.web.app&hl=en&gl=US, May 31, 2021 [17] Lucid Software Inc Lucidchart https://www.lucidchart.com/pages/product [18] JGraph Ltd Draw.io https://github.com/jgraph/drawio Last accessed 15 November 2021 [19] LEKHAPP LLC Lekh diagram https://apps.apple.com/us/app/lekh-diagram/ id576124115, 2012 Last accessed 10 November 2021 [20] DrawExpress Inc Drawexpress diagram lite https://play.google.com/store/ apps/details?id=com.drawexpress.lite, 11 2020 Last accessed 30 November 2021 [21] Tom Y Ouyang and Randall Davis Chemink: A natural real-time recognition system for chemical drawings 2011 [22] Qi Y., Szummer M., and Minka T.P Diagram structure recognition by bayesian conditional random fields Conference on Computer Vision and Pattern Recognition(CVPR), page 191–196, 2005 [23] Ahmad Montaser Awal, Guihuan Feng, Harold Mouchère, and Christian Viard-Gaudin First experiments on a new online handwritten flowchart database Proceedings of SPIE The International Society for Optical Engineering, 7874:1–10, 01 2011 [24] Ciprian Tomoiaga, Paul Feng, Mathieu Salzmann, and Patrick Jayet Field typing for improved recognition on heterogeneous handwritten forms International Conference On Document Analysis And Recognition, 2019 [25] Tran Hoang Thinh Graduation thesis: Building a diagram recognition problem with machine vision approach pages 17–37, 08 2021 [26] Philippe Gervais, Thomas Deselaers, Emre Aksan, and Otmar Hilliges The didi dataset: Digital ink diagram data ArXiv, abs/2002.09303, 2020 [27] Kentaro Wada labelme: Image polygonal annotation with python https://github com/wkentaro/labelme Last accessed September 2021 [28] Manoj Sonkusare and Narendra Sahu A survey on handwritten character recognition (hcr) techniques for english alphabets Advances in Vision Computing: An International Journal, pages 1–11, 2016 [29] Opencv https://docs.opencv.org/4.x/index.html, Nov 18, 2021 [30] Didier Guillevic and Ching Suen Cursive script recognition: A sentence level recognition scheme 09 1997 [31] Alessandro Vinciarelli and Juergen Luettin A new normalization technique for cursive handwritten words Pattern Recognition Letters, 22(9):1043–1050, 2001 [32] Joan Puigcerver Are multidimensional recurrent layers really necessary for handwritten text recognition? 01:67–72, 2017 [33] Théodore Bluche and Ronaldo Messina Gated convolutional recurrent neural networks for multilingual handwriting recognition 01:646–651, 2017 [34] R O Messina B Moysset Are 2d-lstm really dead for offline text recognition? International Journal on Document Analysis and Recognition (IJDAR), pages 1–16, 2019 104 REFERENCES [35] Arthur F S Neto, Byron L D Bezerra, Alejandro H Toselli, and Estanislau B Lima HTR-Flor: a deep learning system for offline handwritten text recognition pages 54–61, 11 2020 [36] R.M Bozinovic and S.N Srihari Off-line cursive script word recognition IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(1):68–83, 1989 [37] Shudong Yang, Xueying Yu, and Ying Zhou Lstm and gru neural network performance comparison study: Taking yelp review dataset as an example pages 98–101, 2020 [38] Basilis Gatos, Georgios Louloudis, Tim Causer, Kris Grint, Verónica Romero, Joan Andreu Sánchez, Alejandro H Toselli, and Enrique Vidal Ground-truth production in the transcriptorium project pages 237–241, 2014 [39] Urs-Viktor Marti and Horst Bunke The iam-database: an english sentence database for offline handwriting recognition International Journal on Document Analysis and Recognition, 5:39–46, 2002 [40] Emmanuèle Grosicki, Matthieu Carré, Jean-Marie Brodin, and Edouard Geoffrois Results of the rimes evaluation campaign for handwritten mail processing pages 941–945, 2009 [41] Andreas Fischer, Emanuel Indermăuhle, Horst Bunke, Gabriel Viehhauser, and Michael Stolz Ground truth creation for handwriting recognition in historical documents ACM International Conference Proceeding Series, pages 3–10, 01 2010 [42] Andreas Fischer, Volkmar Frinken, Alicia Fornés, and Horst Bunke Transcription alignment of latin manuscripts using hidden markov models 09 2011 [43] Kuo-Nan Chen, Chin-Hao Chen, and Chin-Chen Chang Efficient illumination compensation techniques for text images Digital Signal Processing, 22(5):726–733, 2012 [44] Kaiming He, X Zhang, Shaoqing Ren, and Jian Sun Deep residual learning for image recognition 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016 [45] Raimi Karim Illustrated: 10 cnn architectures https://towardsdatascience.com/ illustrated-10-cnn-architectures-95d78ace614d#e4b1, 07 2019 [46] Pavan Gurram Fast and faster region-based convolutional network https : / / medium.com/@gurrampavan6/fast-and-faster-region-based-convolutionalnetwork-6a391a5a247a, 11 2019 [47] Sambasivarao K Non-maximum suppression (nms) https://towardsdatascience com/non-maximum-suppression-nms-93ce178e177c, 10 2019 [48] Kemal Erdem Understanding region of interest — (roi pooling) https : / / towardsdatascience.com/understanding- region- of- interest- part- 1- roipooling-e4f5dd65bb44, 02 2020 [49] K He, G Gkioxari, P Dollar, and R B Girshick Mask r-cnn International Conference on Computer Vision, 2017 [50] R Girshick Fast r-cnn International Conference on Computer Vision, 2015 [51] Juan Tapia Farias Architecture of the original mask r-cnn framework https:// www.researchgate.net/figure/Architecture-of-the-original-Mask-R-CNNframework-The-CNN-represents-the-backbone-used_fig5_334998795, 08 2019 [52] T.-Y Lin, P Dollar, R B Girshick, K He, B Hariharan, and S J Belongie Feature pyramid networks for object detection arXiv:1612.03144, 2017 [53] Tsung-Yi Lin, Piotr Dollár, Ross B Girshick, Kaiming He, Bharath Hariharan, and Serge J Belongie Feature pyramid networks for object detection 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 936–944, 2017 105 REFERENCES [54] Kemal Erdem Understanding region of interest — (roi align and roi warp) https:// towardsdatascience.com/understanding- region- of- interest- part- 2- roialign-and-roi-warp-f795196fc193, 02 2020 [55] Sepp Hochreiter and Jurgen Schmidhuber Understanding lstm networks https: //colah.github.io/posts/2015-08-Understanding-LSTMs/, 2015 [56] Eugine Kang Long short-term memory (lstm): Concept https://medium.com/ @kangeugine/long-short-term-memory-lstm-concept-cb3283934359, 09 2017 [57] Rick Anderson Rnn, talking about gated recurrent unit https://technopremium.com/ blog/rnn-talking-about-gated-recurrent-unit/, 06 2019 [58] Ceshine Lee Understanding bidirectional rnn in pytorch https : / / towardsdatascience com / understanding - bidirectional - rnn - in - pytorch 5bd25a5dd66, 11 2017 [59] M Schuster and K.K Paliwal Bidirectional recurrent neural networks IEEE Transactions on Signal Processing, 45(11):2673–2681, 1997 [60] Harald Scheidl An intuitive explanation of connectionist temporal classification https: / / towardsdatascience com / intuitively - understanding - connectionist temporal-classification-3797e43a86c, 06 2018 [61] Harald Scheidl Beam search decoding in ctc-trained neural networks https:// towardsdatascience.com/beam- search- decoding- in- ctc- trained- neuralnetworks-5a889a3d85a7, 06 2018 [62] Czech Technical Univer-sity Fc database https://cmp.felk.cvut.cz/~breslmar/ flowcharts/ Last accessed 01 October 2021 [63] Jonathan Hui Understanding feature pyramid networks for object detection (fpn) https: //jonathan-hui.medium.com/understanding-feature-pyramid-networks-forobject-detection-fpn-45b227b9106c, 2018 Accessed: 10 November 2021 [64] Flutter https://flutter.dev/ [65] Nodejs https://nodejs.dev/ [66] JGraph Ltd mxgraph https://jgraph.github.io/mxgraph/ Last accessed 15 November 2021 106 ... A. 1 A. 2 A. 3 A. 4 A. 5 A. 6 A. 7 A. 8 A. 9 A. 10 A. 11 A. 12 A. 13 A. 14 Usecase List Usecase: Login Usecase: Sign up Usecase: Create new diagram Usecase: Scan diagram. .. there are two types of diagram recognition which are Online Diagram Recognition and Offline Diagram Recognition There was more attention to the Online Diagram Recognition to handle the diagram handwritten... conceptual design, relational database design, or diagrams demonstrating software architecture like usecase diagrams, sequence diagrams, and flowcharts They offer many kinds of symbols, arrows, and