Khóa luận tốt nghiệp: Development of an application to process house ownership certificate

Flow processing image t0 text.... In the thesis, I develope a mobile application that allows take the picture of a house ownership certificate, then sent for a service to extract the nec

VIETNAM NATIONAL UNIVERSITY HOCHIMINH CITY

UNIVERSITY OF INFORMATION TECHNOLOGY

ADVANCED PROGRAM IN INFORMATION SYSTEMS

NGO HUU ANH

BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS

HO CHI MINH CITY, 2020

NATIONAL UNIVERSITY HOCHIMINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY

ADVANCED PROGRAM IN INFORMATION SYSTEMS

NGO HUU ANH - 15520020

BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS

THESIS ADVISOR

DR QUAN THANH THO

HO CHI MINH CITY, 2020

ASSESSMENT COMMITTEE

The Assessment Committee is established under the Decision

¬— by Rector of the University of Information Technology

1 Assoc Prof., Dr Do Phuc - Chairman

2 Dr Nguyen Thanh Binh - Secretary

3 Dr Do Trong Hop - Member

First of all, I would like to express my gratitude to lecturers in the Information

System Department specifically, and all lecturers who taught me at UIT and other

universities in general You have been supporting me since the first day I joined

UIT Your support has prepared me with the necessary knowledge and experiences

to be confident in my graduation thesis

Also, I want to thank and really appreciate the help from my outstanding thesis

advisor, Dr Quan Thanh Tho, who has supported and facilitated me to have the

ability to complete my graduation thesis As a lecturer, he was always trying to

express his knowledge to me in the simplest way to understand, as an instructor, he

kept me motivated on my thesis, urging me to complete the thesis on time My

completion of the thesis today has a huge contribution from his support

Finally, during the process of making my graduation thesis, I might make some

mistakes Therefore, hopefully, I can receive the understanding from you, the

advisor, and the lecturers in UIT

Again, I would like to express my gratitude to say THANK YOU

Ho Chi Minh City, January 1, 2021

Ngô Hữu Anh

Chapter 2 Theories and technologieS UuSe€C - - «5x nen em nem 2

2.1 Theory of certificate DFOC€SSITE S2 Sc 5.2 x2 +t St SE 111121111 rkre 2

2.1.1 OCR PTOC€SSINE Ánh TT HT HT HT HT TH HH re 2 2.1.2 Checking cCertIfiCaf© «ch HT HT HH HT nh HH 4

2.2 Technologies us€C - 6k xxx 111v v1 TT 1101 ng nh He gry 6

3.5 Database design

Chapter 4 System implementation - - - SH vn 15

VÀ con

4.1.1 Connect application to Firebase = "

4.1.2 Build Authentication UI oe eee

4.1.3 Set up Database and Back-end

4.2 Upload and processing 1ITAÿ6 - + +3 1v TT ngư

Chapter 6 Conclusion - - Ăn Họ Họ vn 24

2 N99 0): 242n "c9 24

REEFERENCES 4E uvs/2o (TT nna vo ăễnnErreocDTÊÖ-oncKvssseereeennnnersrsrennnnnnnnersse 25

APPENDICES S-n cm eee 26

11

LIST OF FIGURES

cs Le

Figure 2.1 Flow processing image t0 text cccsssssescesessessesceeeeeeseeseeseeseeeseeseeseeseeeeseeaeeaeeaes 3 Figure 2.2 RGB color 1mage ccccceeecsesessesseseeeecseeesceeesesescesceesscscecesceeeacseaeaeeesseeeeacseeeaeeeees 4 Figure 2.3 Image after processing with identifiable bounding boxes - - 5

Figure 3.1 System architecture OV€TVICW St HT T TT HT TH TT HT TH TT Hư 8

Figure 3.2 Use-case diagram describes the functions of the mobile application 10 Figure 3.3 Sequence diagram describes the processing image function - ‹ 11

Figure 3.4 The section is designed to authenticate aCCOUII eeeeeseeeeeeeeeeaeeees 12

Figure 3.5 Choose image to upload and process IÍ ¿5< 6 5+ 2t S£+E+Esk+trekeereereeree 13

Figure 3.6 Image management function for USETS 00.0 cece eeteeeeseeeeeeeeeeseeeeaeeeeeeseeetaeeees 13

Figure 3.7 User management function for ađim11 - -ó- 5 + + + + *+*£*xE+eEeeeeeekeeeeereerexse 14 Figure 3.8 Database S{TUCfUTC Q2 2.120 911 11121111111 111 011 11H ng TH Hàn TT Hàng Ty 14 Figure 4.1 Database management with pøgAdmI1 c5 5< 6S + *E++vEskerekrereereeree 16 Figure 4.2 Connecting firebase storage in front-end 5 <6 5+ + S£+x+EsxeEseeereeseeerse 17

Figure 4.3 The images be stored on Storage Service cece + S222 St kEEkrkrrrrkre 17 Figure 4.4 The image is being processed and result after proCessing -.- ‹ «s«s 18

Figure 4.5 The image data of all users on the database - 5 5s + +++x++x+s+xeereersxss 19 Figure 4.6 The user interface is built to display the information - 55 +55 <++ 19

Figure 4.7 User data on the đdataaSe -ó- óc vn HH ng ng ngàn ưy 20

Figure 4.8 UL is built to display info of all IS€TS 5< 5652 S£++Eserseeereereerse 20

Figure 5.1 Specifications Of SVSf€TT G5 TH TH HH HT HH Hư 21

1V

LIST OF TABLES

Table 3-1 Detailed system components

Table 5-1 Table Test Cases

calle

In the thesis, I develope a mobile application that allows take the picture of a house

ownership certificate, then sent for a service to extract the necessary information,the

service AT will attempt to extract the coordinates and information collected will be

stored on database

vi

Chapter |

Introduction

1.1 Overview

In the 4.0 era, it is essential that organizations, companies, or the government are

trying to transfer digital copies of papers, especially house ownership certificates

Therefore, we must have a tool to support the transformation that is very

necessary Besides, mobile devices have become popular, along with the 5.0

network infrastructure developing in Vietnam and around the world So, in the

thesis I will build a mobile application which allows taking the picture of the

house ownership certificate, processing, extracting the necessary information and

storing them

1.2 Objective and Scope

- Apply Flutter technology to develop a mobile application that can allow take a

picture of a house ownership certificate then contact a service to analyze and

show specific information

- Finally, I build the information management

1.3 Expected results

- This application will work well on Android OS

- This application performs the main functions:

1 Get the information which you need from taking the picture of a house

ownership certificate

2 Store and retrieve information when I need it

Chapter 2

Theories and technologies used

2.1 Theory of certificate processing

2.1.1 OCR processing

Purposes: fastly converting an image to text and correcting the mistake results

extracted from the image-to-text stage

This is method of OCR processing:

- Tesseract Model:training Google Tesseract pre-train model with 50

Vietnamese fonts

- Pre-processing:using "imgtxtenh" [1] library to preprocess an image.The

function of 1mgtxtenh' is to clean/enhance noisy scanned text images

- Image segmentation:splitting an image into many rows of text.Inheriting the

result of CRAFT model and G-DBSCAN(clustering algorithm)

OCR processing flow:

- Preprocess on the entire original image with imgtxtenh to filter out noise and

background

- Initialize parameters for craft model

- Using the craft pretrain model to predict the position of words in the image

after filtering the noise, rotate the image if necessary

- Group words in rows with G-DBSCAN

- Separate each data row, pre-process and use the tesseract model to convert

each line of images into text

i Get the word

mode bounding boxes

Gathering points DBScan each Convert each central Com Puang contal

to cluster point point to an object point panos of word g

For each cluster :

Save images have output text

found,create a bounding box

bounding box 4

Figure 2.1 Flow processing image to text

2.1.2 Checking certificate

1 Executing X_Y_localization(ListTest) function has input as ListText that obtained

from the ocr processing stage,function has function of handling text and output are 2

lists that are containing elements which are suspected to be x, y

2 Executing norm_X_Y (list_x, list_y) function has input as two lists that are

processed in step 1, and output are values of two lists in float

3 Executing get_X_Y (float_list_x, float_list_y) function has input as two

normalized lists in step2: float_listX, float_listY.Output is a pair of reasonable values

x,y.

4 Finally, executing the conversion function to Lat,Long coordinates, with input is a

pair of reasonable values x, y in step3

Specifically, for example with a photo of certificate as follow:

2

3 | 121ni2s.76 | so1279.22

4 | ais | s9128AL

s | 121117694 | sos40.36 6

1

121116844 | §91M%/63

TV Những thay đổi sau khi cắp giẤy chứng nhận

ï dung thay đổi và cơ sở pháp lý ae ewe vi TẢ quan |

Figure 2.2 RGB color image

After preprocessing we get the image:

1.

{

- ListText content is: ['saa hhac sha edn voy aay\n\x0c', '\xOc', BANG LAET

KE ‘POA DO GOC RANH\n\x0c’, 1x0c', Xx0c', XxÓc', '\xOc', ¥(m)\n\x0c',

\xOc', 'b201144.76\n\x0c', '$91 309.05\n\x0c', \x0c', '1249031.57\n\x0c',

'S91288.63\n\x0c', \xOc', '1210125.76\n\x0c'", '§91279.22\n\x0c'", \x0c',

'1290138.72\n\x0c'", 'S9TTTRARn\x0c', Xx0c!, '1211176.908\n\x0c'",

'$91340.3%6\n\x0c', '\xOc', '(21916844\n\x0c', '$91348.45\n\x0c', '\xOc', '1211 1444.76\n\x0c', '$91309.05\n\x0c', 'thay ddi sau khi ciip gify

chirng\n\xOc’, '\xOc', 'TV Nban\n\n£Z\n\x0c', 'Xac nhan cua co quan\n\x0c',

Nói dung thay doi va co so phap ly\n\xÓc', 'cé tham\n\xOc’, 'quyén\n\x0c']

- The two lists containing the positioning of the suspected X and Y values are:

2.2 Technologies used

2.2.1 Flutter

Flutter [2] is an open-source UI software development kit created by Google It is

used to develop applications for Android, iOS, Linux, Mac, Windows, and the web

from a single codebase

With Flutter everything you see on the screen is called a widget A widget can

define a structural element, like a button or menu, a stylistic element, like a font or

color scheme, and even an aspect of layout, like padding Flutter can be efficient

about determining which parts of your UI have changed This lets you make changes

to your apps data and leaf Flutter to efficiently rebuild the application Flutter code

was written by Dart programming language

2.2.2 NodeJS

Node.js [3] is an open-source, cross-platform, back-end, JavaScript runtime

environment that executes JavaScript code outside a web browser Node.js lets

developers use JavaScript to write command line tools and for server-side -scripting

running scripts server-side to produce dynamic web page content before the page is

sent to the user's web browser Consequently, Node.js represents a "JavaScript

everywhere" paradigm, unifying web-application development around a single

programming language, rather than different languages for server-side and client-side

scripts

2.2.3 Firebase

Firebase [4] is a Backend-as-a-Service (Baas) It provides developers with a

variety of tools and services such as Authentication, Firebase storage, Database real

time to help them develop quality app and grow their users It is built on Google’s

infrastructure

2.2.4 PostgreSQL

PostgreSQL [5] is a free and open-source relational database management system

(RDBMS) emphasizing extensibility and SQL compliance It was developed at

the University of California, Berkeley

It is designed to handle a range of workloads, from single machines to data

warehouses or Web services with many concurrent users It is the default database

for macOS Server and is also available for Linux, FreeBSD, OpenBSD, and

Windows

Figure 3.1 System architecture overview

3.1.1 System components

Here are details of the components within the system architecture Includes the

technology adopted by me and the component's intended use

Client

Authentication:

- The technology: Firebase

- Authenticate via email / password or anonymous

Application:

- Technology used: Flutter, deployed on Android platform

Server

Backend system:

- The technology is Nodejs

- Connect with the database to manage user information

Backend AI:

- The technology is Python

- Processing image and return coordinates

Database

Firebase storage:

- The technology is Firebase

- Use for saving images

Postgres Database:

- Asa database to manage user information

- Management information: name, id, link image from Firebase

and coordinates

APIs

- Send the image link to service for processing and get the result

- Save the image link and extract coordinates to the database

- Send the user information to the database (id, name)

- View list of users (admin)

- View the user archives (image, coordinates)

- Delete user (admin)

- Delete information image by the user

- Update information of image by the user

Table 3-1 Detailed system components

3.2 Use Case diagram

Mobile application to process house ownership certificate

These are the flows of the application, the purpose of which present the idea and

facilitate the development process They help the viewer visualize exactly what the

product looks like

11

Lets choose image

Figure 3.4 The section is designed to authenticate account.

12

Lets choose image

Lets choose image

Processing AI please wait !!

Figure 3.5 Choose image to upload and process it.

09:52 AM

Figure 3.6 Image management function for users.

13

0352 AM

0452 AM @

Cao Van Lam

Sign Out Tran Tuan Nam

Dinh Nam Tien

Figure 3.7 User management function for admin

Chapter 4

System implementation

4.1 Authentication

4.1.1 Connect application to Firebase

To use services such as authentication or storage which are required for the

project, the application needs to connect to Firebase In this section, the

application needs to use the firebase authentication service to perform user

authentication.Set up database and back-end

4.1.2 Build Authentication UI

This is a part of user interface that I built by Flutter Right side is sign in screen

and left side is sign up screen

Sign In Create New Account

Forgot Password?

Create New Account

Haye an Account? Sign In

4.1.3 Set up Database and Back-end

As mentioned in the system architecture, I need a database to store user

information, although Firebase Authentication service can be used to manage the

account of users like a database, however, to facilitate the management of data

related to the user such as link of the image, coordinates, So I chose PostgreSQL

to build a database for storing all of that data.

To get data from the database and bring it to the user interface, I use NodeJS to

build a server

[Admin Filey Object» Tools» Helpy i

P

Browser © EfW Q Dashboard Properties SQL Statistics Dependencies Dependents § sysdb/postgres 8publicusers/sysd < > 2@

> @ catalogs si =nx z av wy malay RRR wim © ap le ey ss ci

> ÍF; Event Triggers avid |e N

> extensions

uery Editor Query History Scratch Pad x

> Foreign Tables 3 7 AGoAfo2kzdVPgoyrzBKizmexUt33 Lucas Moura 1

> (Functions 4 11 CHHqGjz3laDKqQzhm1lSbltD3 HaryWinks 1

> [BiMaterialized Views 5 4 cPmuo9xIglgrRkBEhZ0IOhMFSC/2—_ HugoLioris 1

> ()Procedures 6 10 czcvPSTYvubVoI991CKS10qpF53 Jose Mourinho 1

> 1.3Sequences 7 6 GSLDRSAWZPTIeIPJZIMHLEC7rw1 Son Heung Min 1

v fETables (8) 8 14 khDI6Q0LaySƒHBL625Cqbr9jg2 _Davinson Sanchez 1

> role

> Bui

10 8 NŒ80P4S02Đ6gZEzB7I3LHdLg2 Steven Bergwin 1

> Eluses | MzE8JTaLdg mĩ

> Mager Futian n 2 Ppyb|3fPYTyqEhf9nwVJcAtYEH2 Eric Dier 1

> iTypes 2 12 VHOKSsprCVOKDPAPEJuohalVeNx1 Serge Aurier 1

> [views 1B 3 XSHBDINNSSRLEMtsiszZzzscipGm1 Harry Kane 1

> 4 Login/Group Roles 1 9 _Yfj7insf9hCJ6vzh4THDICckSƯ2 Matt Doherty 1

> a Tablespaces Pac

Figure 4.1 Database management with pgAdmin

This is pgAdmin that is PostgreSQL database management tool that provides

many features such as SQL Query or backup.

16