Khóa luận tốt nghiệp: AI-based hairstyle application for barbershop

Result of some current hairstyle transfer applications.. CHAPTER 1 : INTRODUCTION1.1 The need of the subject As a result of the growing development in life and demand for beauty, the har

SVIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY

NGUYEN DUC THONG THÁI — 19522188

LE THI KIM CHI - 19521283

GRADUATION THESIS

AI-BASED HAIRSTYLE APPLICATION FOR BARBERSHOP

BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS

TP HO CHI MINH, 2023

VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY

ADVANCE PROGRAM IN INFORMATION SYSTEM

NGUYEN DUC THONG THAI — 19522188

LE THI KIM CHI - 19521283

GRADUATION THESIS

AI-BASED HAIRSTYLE APPLICATION FOR BARBERSHOP

BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS

THESIS ADVISOR PH.D NGUYEN THANH BINH

TP HO CHI MINH, 2023

ASSESSMENT COMMITTEE

The Assessment Committee is established under the Decision , date

M¬ỪỮ by the Rector of the University of Information Technology

¬ cece eee ne eee e eee na een ens — Chairman

tee ene tee ene ene ene ened — Secretary

cent ee tenet nena e nena naa — Member

CHAPTER 2 : REVIEW OF LITERATURE

2.1 Current Hairstyle changing applications

2.2 Hairstyle transfer approaches

CHAPTER 3 : BACKGROUND KNOWLEDGE

3.1 Language and Framework

3.1.1 Python

3.1.1.1 About Python3.1.1.2 Python with AI Hairstyle3.1.1.3 Comparing Pytorch with TensorFlow3.1.2 ReactJS

3.1.2.1 About ReactJS3.1.2.2 Core concept of ReactJS3.1.2.3 ReactJS with AI Hairstyle3.1.3 NodeJS

3.1.3.1 About NodeJS3.1.3.2 Core concept of NodeJS3.1.3.3 Nodejs with AI Hairstyle3.1.3.4 Comparing NodeJS with other frameworks3.2 DBMS

3.2.1 About MongoDB3.2.2 Core concept of MongoDB3.2.3 MongoDB with AI Hairstyle

ON AN Dn C3) VY WY

YN NY NY NY NY NN NN N YF KF FP FP KF =c= FF FF eRao NY NN WD FP WN NY KF DN NDA HBR WO WO WwW

3.2.4 Comparing NoSQL with SQL3.3 Tools and Utilities

3.3.1 About Android Studio3.3.2 Android Studio with AI Hairstyle3.4 Generative adversarial network (GAN)

3.4.1 Introduction to GAN3.4.2 How do GANs work?

3.4.3 GAN with AI HairstyleCHAPTER 4 : OUR SYSTEM ANALYSIS AND DESIGN

4.1 Al approach for AI-HairStyle-Engine

4.1.1 Overview4.1.2 GAN inversion4.1.3 Processing steps in detail

4.1.3.1 Embedding4.1.3.2 Target Hair Alignment4.1.3.3 Source Inpaiting

4.1.3.4 Blending4.2 System Analysis and Design

4.2.1 Design system architecture4.2.2 Main using flow

4.2.3 Database structure

4.2.3.1 Hairstyle4.2.3.2 Admin

29303031323232363737373840 404247485151

54

55555657585859

60

62

4.2.8 Design Interface 67CHAPTER 5 : IMPLEMENTATION AND DEPLOYMENT 76

5.1 Client — Mobile Application UI 76

5.1.1 Homepage 765.1.2 List of Hairstyles 775.1.3 Search Hairstyles 785.1.4 Taking image 795.1.5 Homepage after choosing hairstyle and taking an image 825.1.6 Loading screen 835.1.7 Result screen 845.2 Dashboard UI 85

5.2.1 Login screen 855.2.2 Hairstyle management screen 865.2.3 Update hairstyle screen 875.2.4 Deleting a hairstyle screen 875.2.5 Add hairstyle screen 885.3 Deploy Environment 90

5.3.1 Frontend 905.3.2 Database 905.3.3 Cloudinary 915.3.4 Backend 92CHAPTER 6 : CONCLUSION 95

6.1 Comparison with other approach 956.2 Challenge and methodology 98

6.2.1 The problem of GAN inversion 986.2.2 Our methodology 996.3 Future work 100

REFERENCES 101

LIST OF FIGURES

Figure 1-1 Output of application Hair Makeove .:cecceesceesceeseceneceneeesceceaeeeeeeteeeeas 2Figure 1-2 An example of AI-Hairstyle application a) Input a picture of a user b)

Hairstyle was chosen by the user c) Output of AT-Hairstyle application that is user in

figure 1-2a after applying hairstyle 1-2Ư - - «5 s13 1E HH ng ng ng kg 3Figure 2-1 Feedback of some current Hairstyle transfer applications - - ‹- 8Figure 2-2 Result of some current hairstyle transfer applications a) original image, b)

Hairstyle Try On, c) Hair try-on — hair-styling, d) Hair Makeover application 9Figure 3-1 Survey popular of active SOÍYWAT - Gà HH TH ng ngư, 13Figure 3-2 Example of components in ReactsS - 5 c5 kg re 19Figure 3-3 Example of JSX clement ccc eee escesceseneeneceesesseeseeseeeeeesesseeeeaeeaeeaes 19Figure 3-4 Example of Props in ReaCtJS - csccseeseeseescesceesssssesessesaesaeeseeseees 20Figure 3-5 Example of Event Handling in ReactJS ooo ee csesessceseeeeeeeneeeeeneeneeseeaes 20Figure 3-6 General architecture Of GATN - LH HH TH ngư 32Figure 3-7 Min-Max GAN loss ÍUnCfIOH - -ĩ- G1111 211v TH ng ry 33Figure 3-8 Training the discriminator to distinguish between fake images generated by

the generator and real images using Backpropagation algorithm . -s «+ 34Figure 3-9 Training Generator to generate fake images with feedback from Discriminatorusing Backpropagation aÏØOTItHIT - - «<< E1 E911 11 9101930 1H in 35Figure 4-1 An overview of the framework proposed in [1] - StyleYourHarr, which is

used in our AI Hair Engine Initially, we acquire wsrc, wtrg, and Fsrc by embedding

both a source and target hair image into the W+ and FS spaces Subsequently, we

fine-tune wtrgto align with the source pose, yielding wtrgalign Using the segmentation

mask of the aligned target hair, we determine wsrcinpaint , which handles the

inpainting of source occlusions Finally, we combine Fsrc, wsrcinpaint, and

wtrgalign to produce the high-quality OUtPUL 0.0 eceeseseeseeseeeeeeeeeeeeseceeeseeseeeseeneees 38

Figure 4-2 Visualization of GAN inversion: In contrast to the typical procedure of

sampling and generating with a trained generator G, GAN inversion involves mapping a

Figure 4-3 Reconstruct an image from random latent Code Z - «+ «<< «++<++ 41Figure 4-4 Latent code optimization Process cccecsesceseeseeseceseceecesececeseeeceseeseeeeeeaees 42Figure 4-5 Reconstructed image is improved throughout epochs After epoch 1000, the

reconstructed image looked the same with the input 11àØe - 55-5555 + ++s£+s++sx++ 42

Figure 4-6 Target hair alignment phase of the proposed framework in [1] The algorithm

derives the aligned latent codes for the target hair, wtrgalign, through optimization of

the initial m vectors within wtrg This optimization process ensures that these vectors

adopt the source pose while maintaining the original ha1rstyÌe ‹- 5s «<< 43Figure 4-7 Local-style-matching loss [1] While aligning the target hair, the algorithm

incorporates a local-style-matching loss, which is employed on style regions found in

both the target hair and the aligned target hair The style regions are delineated by white

boundaries, and the red-bordered regions illustrate an instance of a style region that is

consistently fraCK€C - s11 1H HT HH TH HH nh 45Figure 4-8 Generation of an objective label [1] When it comes to source inpainting, the

algorithm establishes an objective label known as S srcobj This label serves as a

reference to direct the process of filling the occluded regions with appropriate semantic

Figure 4-9 Blending [1] (a) The w vectors derived from the earlier stages are mixed

with the fine-tuned blending weight wweight to produce wblend (b) Following this, wemerge F trgalign, Fblend, and Fsrc with their respective masks to form F final (c)

F final and wblend are then input into the StyleGAN2 generator to generate the final

output (d) The blending loss is comprised of three components: L perceptkeep,

A percepthair, and L perCe€thQÏT cccccccccccssccssssecscesseessecsnscsssesseessseessecsseesseeeessesseeesnees 49

Figure 4-10 AI-Hairstyle pipeline - - c5 11 SH HH HH kh, 51Figure 4-11 AI-Hairstyle system arChIt€CfUTC - - 6 251 + 1311 1* S3 vn cư, 52Figure 4-12 Main using flow for our mobile client appÏication -«++-«<+++ 54Figure 4-13 API structure đ1aØTaIm - c1 331135111211 193119 111111191 HH TH như, 57Figure 4-14 AI-Hairstyle use case đIaØTAI - - - G1111 HH re, 59

Figure 4-16 Activity diagram for adIm111 6 5 E3 191 33 E9 ESxESkEEkEkkrskrskree 61

Figure 4-17 Sequence diagram for generating hairstyle (Mobile client U]) 62

Figure 4-18 Sequence diagram for searching hairstyle (Admin U]) - - 63

Figure 4-19 Sequence diagram for inserting a new hairstyle (Admin UỊ) 64

Figure 4-20 Sequence diagram for updating hairstyle (Admin UD) .‹- - 65

Figure 4-21 Sequence diagram for deleting hairstyle (Admin UI) - « - 66

Figure 4-22 Wireframe of mobile UI homepagỹe - ¿+ +s + ++x£+sE+eE+sereeeseerserse 67 Figure 4-23 Wireframe of hairstyle Ï1Sf 5 + + 2213 23 1E 9 1 ri 68 Figure 4-24 Wireframe of confirming image SCT€€T 5 55 2 2+ + +sEEeeseereeree 70 Figure 4-25 Wireframe of result SCT€€TA 5 5 211 1 TH nh HH ngư nên 71 Figure 4-26 Wireframe of share image SCT€€I1 26 <6 (511230111511 81 E331 11 vn 72 Figure 4-27 Wireframe of Login page (admin pOrfaÌ) - - + «++s+++++s+se+se+ 73 Figure 4-28 Wireframe of hairstyles list (admin portal) - - «<< £+s++ecz+sx+s 74 Figure 4-29 Wireframe of adding hairstyle form (admin portal) « -«<+s+ 75 Figure 5-1 Homepage of mobile UI, users can choose hairstyle by clicking on the second button (2), and take image by clicking on first button (1), and generate image by clicking On the third button ()) (211 2111111665388 1111111111119 1111 003111111 11kg vn yy 76 Figure 5-2 List Of hairstyle SCT€€TNS - G2 S2 9211121113111 11 811191 11 HH ngư, 77 Figure 5-3 Search hairstyle SCreen - G11 HH gi 78 Figure 5-4 Taking image SCT€€IN - G5 E1 S111 HH HH Hư 79 Figure 5-5 Detecting image SCT€H - - -G G n1 TH HH HH ky 80 Figure 5-6 Confirm image SCT€€T G1231 1911 910 E911 9011111 HH HH ky 81 Figure 5-7 Homepage after choosing hairstyle and taking image of users' face 82

Figure 5-8 Loading SCreen ceecescesceseeeeceeeseeeseeeeesecsececesseescessecsaeeseceeeeaeeeeeaeeseseneeaees 83 Figure 5-9 Result screen and QR code for users to download their image 84 Figure 5-10 Administrator login page - - 5 11111112111 111911191 HH Hy 85 Figure 5-11 Hairstyle management SCT€€TI - - - E122 11911311 E931 E91 1v nh rưy 86 Figure 5-12 Update hairstyle SCT€€TN - - G3 1391119111931 5 1111111 HH thư, 87 Figure 5-13 Delete hairstyle Screen 0.0 ee es eeceeseesecesesseeeseeseecessecscesseseeeeeeaeeaeseeeeaeeeees 88

Figure 5-14 Adding hairstyle SCT€€TI - (G5 1231991 931991 911 01 vn ng ng nh 89Figure 5-15 Loading screen while the server processes the request - -©- 89Figure 5-16 Deployment of Administration Íront-enid - - «- ««+sx£+s+seeseeeseess 90Figure 5-17 MongoDB database InfOrmatiOT eee eseeeeeeceeeeeeseeeesesseeseeseaeees 91Figure 5-18 Cloudinay aCCOUII - - 6 s11 91 90191 HH HT HH nh 92Figure 5-19 Server's CPU 1nÝOTIafIOTI - - 6 5 111191 230191 911 01 0v ng ghi nr 93Figure 5-20 Server's RAM InÍOTIAfiOT 6 5 6 31918231 1 911 1 91 0v ng nh 93Figure 5-21 Server's GPU InfOrTmatiOT «+ + 3 23 21123 1 1 TH HT ng rệt 94Figure 6-1 Compare generated images of Barbershop and StyleYourHarr (1) 96 Figure 6-2 Compare generated images of Barbershop and Style YourHair (2) 97Figure 6-3 Our distributed GAN inversion approach cceeeeeseescceeceeeneeeteeeeeeseaeeeeeeee 99

LIST OF TABLES

Table 3-1 Comparison between Python and TensorFlow ccceeseeseeeeeeececeneeeeeetenees 16Table 3-2 Comparison between NodeJS and other framewOorKsS ++- «<< <+++ 26Table 3-3 Comparison between NoSQL and SQL eeceecceeseesseeseeeeceeeseeeaeeeeeseeeeeaeeeeees 29Table 5-1 Structure of Hairstyle €OlÏ€CIOHI - 5 2219112111 1E 11 1 81 91 vn cư 56Table 5-2 Structure of Admin CỌÏGCfÏOTA - 6 5 6 5E 123131 E51 1991 911 11 1 1g nh 57Table 5-3 API for sr 1 da 58Table 5-4 API for ÀAm1Ti 7 5 1119119119101 1 900111 HH nh ng nry 58Table 7-1 We measured FID score for Barbershop and StyleYourHarr framework on

hairstyle transfer task ceceeseesceseeseceseeeceseeececeesecceeseceeesecseessecsecesecseceseeseeeeesaeeeeeeaees 95

I would like to extend my heartfelt gratitude to the individuals who have played pivotal

roles in supporting and guiding me throughout the completion of this thesis

First and foremost, my sincere appreciation goes to Dr Nguyen Thanh Binh for his

unwavering commitment to this project His invaluable technical guidance and insightfulcontributions have significantly enriched the quality of this thesis Dr Binh's dedication

and mentorship have been instrumental in shaping the direction of my research, and for

that, I am truly thankful

Furthermore, I would like to express my gratitude to all the esteemed teachers at the

University of Information Technology — Vietnam National University, Ho Chi Minh

City Their profound knowledge, engaging teaching methods, and continuous support

have been integral to my academic journey The lessons imparted by these educators havenot only deepened my understanding of the subject matter but have also inspired and

motivated me to strive for excellence

In acknowledging these individuals, I recognize that their collective efforts have played acrucial role in my academic and research endeavors I am truly fortunate to have had theprivilege of learning from such dedicated mentors and educators, and their impact on myscholarly pursuits is immeasurable

Ho Chi Minh City, December 28, 2022

Author team

In the fast-paced and ever-evolving landscape of the contemporary era, personal style andself-expression have taken center stage The growing demand for beauty has becomeincreasingly essential, evident from the rising number of hair salons not only in Vietnam

but also around the world To cater to hair salons and users, we proudly present AI Hair

Style, a revolutionary system At its core lies an innovative and user-friendly mobile

application designed to transform how people explore, experiment, and transform their

hairstyles This cutting-edge app boasts a powerful feature - "Change Hair Style" - offeringusers a comprehensive and personalized hair styling experience Leveraging the support of

AI, the app goes beyond mere virtual styling and proposes high-quality looks that userscan envision as real-life transformations Embrace the future of hairstyling with AI Hair

Style, where creativity meets technology for the ultimate hair journey

CHAPTER 1 : INTRODUCTION

1.1 The need of the subject

As a result of the growing development in life and demand for beauty, the harshness ofuser aesthetics increases in proportion to the cost and competitiveness of the beauty servicemarket, so trying a hairstyle like trying other items or pretesting clothes is essential.However, many applications support users to try hairstyles like Virtual Hairstyle, Hairstyle

Changer, and Hair Makeover However, these applications only stop to the extent that they

allow the user to use a hairstyle to graft the user's face in the graft form, so the output isnot yet honest enough to help the user visualize how the hairstyle fits into his or her face

a) original image, b) result image from application Hair Makeover

Aware of the problem with existing applications that do not meet users' needs, we built anapplication called AI Hairstyle - AIHS, which uses Deep Learning technology, so that the

"AI Hairstyle - AIHS" output reaches a far higher level of realism than traditional (2D)

apps, which allows users to figure out what their face looks like after a Hairstyle is cut

Then the person can choose the best style and best fit for hair

Figure 1-2 An example of Al-Hairstyle application a) Input a picture of a user b) Hairstyle was chosen by the user c) Output of

AI-Hairstyle application that is user in figure 1-2a after applying hairstyle 1-2b

1.2 Scope

Recent evaluations of contemporary hairstyle apps reveal a consistent trend of falling short

of user expectations and satisfaction Notably, a noticeable void exists in the market withregard to apps designed to cater specifically to the Asian and Vietnamese communities.Our thesis aims to address this gap by leveraging the capabilities of AI technology andincorporating them into the development of the AI-Hairstyle App Through this endeavor,

we aspire to transform the user experience by providing a significantly enhanced andcustomized solution tailored to the unique preferences and requirements of these distinct

We will create an application for hair salons to upgrade their services that containsalgorithms matching with features Besides that, the AI-Hairstyle application combines AI,model StyleGAN2, and hairstyle data which can help users choose the best style for them.The main function for users is to see the result when they get some hairstyle, thisapplication will generate an image that shows the face of the user associated with a newhairstyle.

This thesis aims to develop a comprehensive application for hair salons to enhance their

service offerings by implementing advanced algorithms that facilitate personalizedhairstyle recommendations In addition, the AI-Hairstyle application will integrateartificial intelligence (AD technologies, leveraging the StyleGAN2 model, and harnessing

a rich dataset of Vietnamese hairstyles to assist users in selecting the most suitablehairstyle

The system comprises four core components: an administrative dashboard, a oriented tablet UI, an AI Hair Engine module, and APIs for request processing Theadministrative dashboard provides administrators with comprehensive control, enablingthem to effortlessly add or remove hairstyle images, thus ensuring system currency andorganization

client-The tablet UI, designed with a harmonious blend of aesthetics and usability, aims toenhance the user experience It boasts an intuitive and user-friendly interface, a visuallyappealing and contemporary design, and a plethora of customization options, all carefullycrafted to resonate with users and make their tablet experience a positive one

Within the context of this thesis, we embark on an exploration of the feasibility ofintegrating an AI hair engine model into the system This endeavor entails training the AImodel on a specialized dataset of hair images and subsequently subjecting it to rigorous

testing on a separate image dataset We also delve into diverse strategies for enhancing the

AI model's precision, including the adjustment of hyperparameters and the application of

data augmentation techniques

The API we develop provides an efficient means to process request statuses The Nodejs,and JavaScript frameworks are used to build API.

In summary, this system aims to streamline hair salon operations by combining advancedtechnology with user-friendly design principles Through this thesis, we seek to validatethe integration of an AI hair engine model while maintaining a keen focus on enhancingthe overall user experience and system efficiency

1.3 Objectives

Our primary goal is to develop a complete system that allows users to experiment with a

wide range of hairstyles before making a decision about their haircut

In pursuit of our primary goal, we have outlined a set of key objectives These objectivesare designed to guide our efforts and ensure the successful realization of our vision for acomplete hairstyle experiment system By addressing each of these objectives, we aim to

build a user-friendly platform that seamlessly facilitates hairstyle trials, enabling

individuals to make informed decisions prior to getting a haircut:

e Research and understand the StyleGAN2 model: Conduct an in-depth literature

review and analysis of StyleGAN2 to gain a comprehensive understanding of itsunderlying principles, architecture, and applications in image generation Moreover, ifidentified, we plan to propose a resolution to enhance the speed and performance ofStyleGAN2

e Design the Hair-Engine: Develop a detailed design and architecture for the

AI-Hair-Engine, outlining how StyleGAN2 will be integrated and utilized for generatingrealistic hairstyles

e Implement the AI-Hair-Engine: Develop the AI-Hair-Engine module, integrating

StyleGAN2 into the system, and fine-tune it to generate a plausible image of a user withthe chosen hairstyle

e Build a Server with APIs: Create a server infrastructure that hosts the AI-Hair-Engine

and processes requests from the admin side and client sides

e Develop an admin dashboard: Design and implement an intuitive admin dashboard

that allows administrators to manage hairstyles in the database

e Develop Android UI for clients: Design and develop a user-friendly Android

application interface that allows users to choose a wide range of hairstyles in our

database and receive the generated results

e Testing and validation: Conduct rigorous testing and validation of the entire system

to ensure the system can generate high-quality images, the server functions reliably,and the Android UI provides seamless user experience

1.4 Expected result

The envisioned outcome of my thesis is to create an exceptional AlI-driven Hairstyleapplication that sets itself apart from existing applications available on Google Play Thisinnovative application will have the capability to generate new images by seamlesslymerging a user's facial image with their chosen hairstyle It will achieve this by harnessingthe power of AI technology, particularly leveraging the StyleGAN2 model, andincorporating a meticulously curated database, crafted by our team for a truly unique userexperience

CHAPTER 2 : REVIEW OF LITERATURE

2.1 Current Hairstyle changing applications

In the contemporary world, attire and hairstyle go beyond mere enhancements to ourappearances; they serve as means for expressing ourselves to the world A well-chosenhairstyle or suit not only enhances your aesthetic appeal but also plays a crucial role incrafting a compelling first impression, leaving an indelible mark on how you are perceived

by others However, when choosing clothing, the option to try it on provides a clearunderstanding of its suitability prior to buying it In contrast, the realm of hairstyling lacks

an analogous pre-evaluated method Unlike the straightforward process of trying onclothing, anticipating a hairstyle’s outcome before getting a haircut is challenging Theonly way is imaginative visualization, creating an element of uncertainty Someapplications attempt to fill this gap by offering virtual hairstyle options, allowing users tostick hairstyle stickers onto their face images However, the limitations of such applicationsbecome apparent, as these stickers, inherently artificial in appearance, offer only a limitedperspective, making it difficult to accurately envision the final outcome of the intended

haircut on one’s own facial features

Several hairstyle changing applications currently offer users the ability to explore diverse

hairstyles Examples include Hairstyle Changer - Hairstyle, Hairstyle Try On: Bangs &Wigs, and Hair Makeover These applications, although widely used, share a fundamentaldrawback - they rely on static hair stickers that users superimpose onto their images.Consequently, the visual output lacks the authenticity required for an accurate assessment

of how a selected hairstyle truly complements an individual's facial features

The primary limitation of these hairstyle changing applications lies in their dependence onpre-designed hair stickers These stickers often struggle to seamlessly integrate with users'facial features, leading to an unrealistic and disconnected visual representation As a result,users may experience a gap between the virtual preview and the actual outcome whentrying the selected hairstyle in reality

{ 2 Hairstyle Try On: Bangs & Wigs x

Ratings and reviews

6 Brian Fitzgerald

November 22, 2023

| & | Hair Makeover-modiface - haircut x

Ratings and reviews

Not enough wigs (BGEMeRE GSES NOLOHEMIESTIGNE Not enough age appropriate or blonde wigs for

instance Also the long hair female wigs are just plain awful

1 person found this review helpful

® Simona I-B H

Did you tind thls helpful? Yes No

*

(0) Dylan Stuts : 27 August 2023

this app uses crops from photos, verypoor qui ‘ou get hairstyles with ears} Asks for Sstar review

* September 28, 2023 to continue | added 1 star and could go further You can't save any photo 0 stars from me and

it's ui le that they want you to pay for this _ N6fffeis oF ai uninstall immediately.

48 people found this review helpful 15 people found this review helpful

Did you find this helpful? Yes No Did you find this helpful? Yes No

4 Hair try-on - hair styling x

a Ratings and reviews

‘changing options are a joke If the wig youre

difference in colors, Would give this app 0 stars if 1 could.

71 people found this review heipful

Did you find this helpful? Yes No |

Cw) Wojciech Kapica

31 March 2022

10 peopie found this review helpful

Did you find this helpful? Yes No

Figure 2-1 Feedback of some current Hairstyle transfer applications

Figure 2-2 Result of some current hairstyle transfer applications a) original image, b) Hairstyle Try On, c) Hair try-on —

hair-styling, d) Hair Makeover application

Addressed that issue, in this study, we aim to develop a system that assists users inexperimenting with various hairstyles before getting a haircut Our system will leveragecutting-edge technology and state-of-the-art AI algorithms in general and Generativeadversarial network (GAN) in specific, this approach enables our system provides userswith a realistic depiction of their face with the chosen hairstyle Therefore, they caneliminate the worry about how their face will appear with the new hairstyle

2.2 Hairstyle transfer approaches

Currently, we have two main approaches for the hairstyle transfer problem The firstmethod is a long-established and widely used technique involving cutting, pasting, andediting images using tools such as Photoshop We can use Photoshop to cut out anyhairstyle we like from various images Subsequently, we can paste that hairstyle onto ourfacial image and perform image editing operations to make the image look more realistic.However, this method is quite complex for those unfamiliar with using Photoshop.Additionally, it requires human intervention, making it challenging to apply in practical

scenarios for creating systems or services that allow users to easily try out differenthairstyles To address the identified limitations that occurred with using Photoshop,numerous applications facilitating the transfer of hairstyles have been introduced These

applications generally offer two primary methods of utilization The initial method

involves users selecting from a repository of pre-existing hairstyle stickers provided by theapplication and overlaying these stickers onto their facial image While this method isfaster and more user-friendly compared to employing sophisticated tools like Photoshop,

the outcomes fall short of achieving the desired quality, as detailed in section 2.1

The second method entails the application autonomously detecting the user's face andsubsequently applying the chosen hairstyle automatically However, despite this automatedprocess, there has been no substantial enhancement observed in the quality of resultsachieved through this approach

But thanks to the remarkable advancement of technology, we have a more modern second

approach to the hairstyle transfer problem This involves using AI algorithms, more

specifically, employing Generative Adversarial Network (GAN) models Because thismethod does not require human intervention during the process, it is suitable for building

systems as mentioned above Presently, three noteworthy frameworks leverage GAN for

hairstyle transferring — LOHO [9], Barbershop [2], and Style YourHair [1].

Hairstyle transfer belongs to a bigger problem, that is style transfer In style transfer, ouraim is to extract the stylistic components from an image and embed those styles into a baseimage like extracting only the hairstyle that is represented in a sample image and thenembedding the hairstyle onto the user face The earliest model that can tackle style transfer

is Pix2Pix, but this model has a drawback Pix2Pix necessitates the presence of every imagewithin the training set in both source (set of images for extracting styles) and target (set ofimages that will be embedded with the styles from source images) domain [14] In otherwords, using Pix2Pix in our system is unfeasible due to the impracticality of training the

model For instance, if I have 100 users and a list of 30 hairstyles, to enable all 100 users

to try out all 30 hairstyles, I will need to request each customer to capture images with all

30 hairstyles for training the model Moreover, if I were to add a new hairstyle or have anew user, I would need to repeat this process and retrain the model This is Impractical

when considering a broader perspective, especially if we aim to launch this system for

global use However, a state-of-the model which is StyleGAN2 can aid this problem using

a technique called Latent space manipulation, so we decided to use StyleGAN2 in our

system.

Image generation using generative adversarial network (GAN) Generative adversarial

networks (GANs) are Deep Learning-based algorithms When the concept was firstintroduced by Goodfellow in [15], it has been playing a crucial role in researching highquality image generation Many state-of-the-art GAN models can produce plausible imagesthat cannot be distinguished from real images such as ProGAN [16], StyleGAN [17] andStyleGAN2 [18] Especially, when it comes to human faces, these GAN models have theability to create hyper-realistic and unparalleled quality [19, 20] When discussing thepower of GANs, it would be incomplete not to mention a vital element, which is thelifeblood not only of GANs but also of other Deep Learning algorithms - that is data.According to [StyleGAN2-ada], GANs can be trained on limited data without

compromising their generative power Datasets like FFHQ, AFHQ or LSUN provides

GANs with both necessary quality and diversity for training, significantly

contributing to the development of realistic image generation Therefore, our

research focuses on implementing StyleGAN2 to develop a sophisticated system capable of generating lifelike images portraying a user's face with a variety of hairstyles This system empowers users to experiment with different hairstyles

virtually before committing to a haircut

Latent space manipulation Recent studies, as highlighted in [1], delve into the

manipulation of latent spaces [II, 21, 22] for image editing, showcasing promisingoutcomes in this research area Notably, GANSpace [21] and InterfaceGAN [22] have

successfully altered facial attributes by manipulating the latent space of StyleGAN [5].Principal component analysis is utilized in GANSpace, while InterfaceGAN uses semanticscores for identifying disentangled directions related to desired attributes Similarly,

Viazovetskyi et al [23] and Zhuang et al [24] concentrate on editing images by shifting

latent vectors along semantically meaningful directions in the latent space of StyleGAN2[18], achievable through a pre-trained face classifier or learned transformation Cutting-edge hair-style transfer algorithms leverage latent space manipulation to generate high-quality images LOHO [9] and Barbershop [2] employ latent space optimization tomanipulate the extended latent space of StyleGAN2 for sophisticated image editing.Additionally, Barbershop introduces the innovative FS space, distinguished by largercapacity compared to the StyleGAN2 latent space, thereby guaranteeing the preservation

of the original hairstyle structure

CHAPTER 3 : BACKGROUND KNOWLEDGE

3.1 Language and Framework

3.1.1 Python

® python’

Java, C#, and C/C++ grow slower

than the developer populationNumber of active software developers, globally, in millions, Q4 2018 (n=11,519)

Desktop, AR/VR, Gaming

Desktop, lof, Gaming, AR/VR

Mobile, Gaming, Web

loT devices, ML, Web

loT devices, Machine learning

Web, Mobile, Cloud

of prolonged compile and link steps, further amplifies programmer efficiency Theportability of Python programs across major computer platforms stands as anotherprominent attribute Transcending platform-specific intricacies, Python facilitates seamless

code transfer between operating systems, exemplifying its versatility This extends to

diverse application domains, including the development of portable graphical user

interfaces, database access programs, and web-based systems Python's comprehensive

standard library adds another layer of functionality, covering a spectrum of level programming tasks Beyond the standard library, Python's extensibility is showcasedthrough in-house libraries and a rich repository of third-party application support software,ranging from website construction to numeric programming Python's adeptness incomponent integration is a notable facet, enabling seamless communication with otherapplication components through various mechanisms This interconnectedness positionsPython not as a standalone tool but as a versatile contributor to product customization and

application-of programming from a mundane chore to an enjoyable pursuit While the subjective nature

of this benefit may be intangible, its positive impact on productivity remains a Snoteworthyasset Among the myriad attributes, the emphasis on software quality and the substantial

boost in developer productivity emerge as the most compelling advantages for a majority

of Python users

3.1.1.2 Python with AI Hairstyle

In our system, the incorporation of the AI-Hairstyle module is imperative, and Pythonstands out as an ideal choice for this task Python offers numerous frameworks for imageprocessing and constructing Deep Learning models, including OpenCV, Theano, PyTorch,and TensorFlow After careful consideration, we have opted to utilize PyTorch toimplement the module due to the following reasons:

e Dynamic Computational Graph: PyTorch uses a dynamic computational graph,

allowing for more flexibility during model construction and easier debugging

e Data parallelism: The data parallelism feature in PyTorch facilitates the distribution

of computational tasks across multiple CPU or GPU cores While other

machine-learning tools also support parallelism, PyTorch makes the implementation of dataparallelism significantly more straightforward.

e Community and documentation: PyTorch has a vibrant and active community, and

its documentation is well-maintained, making it accessible for both beginners andexperienced practitioners

3.1.1.3 Comparing Pytorch with TensorFlow

Table 3-1 Comparison between Python and TensorFlow

Dynamic graph for easyadjustments, intuitive modelbuilding

Popular in research,innovative ecosystem

Dynamic graph aidsdebugging, and iterativedevelopment

User-friendly for research,prototyping

Lacks native tools, relies onthird-party

Seamless integration withPython.

Gains traction in academia,praised for flexibility

The static graph providesoptimization, may be lessintuitive

Mature tools for industrialapplications

TensorBoard forcomprehensive visualization

Extends beyond Python,multi-language support.

Dominates in industry, robustproduction capabilities

industry applications

More user-friendly, shorter

curve.

3.1.2 ReactJS

3.1.2.1 About ReactJS

ReactJS is a JavaScript library designed for constructing reusable UI components As perthe official React documentation, the definition is as follows serves as a library forconstructing adaptable user interfaces It promotes the development of reusable UIcomponents that display dynamically changing data Many individuals utilize React in thecapacity of the "V" in MVC (Model-View-Controller) React simplifies the handling of theDocument Object Model (DOM) by providing a more straightforward programming modeland enhanced performance Additionally, React has the capability to perform server-siderendering using Node and can drive native applications through React Native Theimplementation of a one-way reactive data flow by React minimizes unnecessary code andoffers a more straightforward approach compared to traditional data binding

ReactJS, a dynamic JavaScript library maintained by Facebook, plays a pivotal role in the

landscape of tablet and mobile application development This chapter delves into the

nuanced applications and advantages of ReactJS, with a particular focus on its contribution

to the creation of responsive and scalable user interfaces

Responsive Design with React Native:

An essential extension of ReactJS, React Native empowers developers to construct nativemobile applications using JavaScript and React Its cross-platform capabilities enable theseamless deployment of applications on both iOS and Android platforms, fosteringcodebase consistency and efficient development practices

ReactJS excels in promoting code reusability, allowing developers to employ the same

components across web and mobile applications For instance, consider the following

The inclusion of hot reloading in React Native significantly enhances the development

process For example, during the development of a mobile app, changes made to the code

are instantly reflected in the app interface without the need for time-consumingrecompilation, allowing developers to iterate more efficiently

React Native grants developers direct access to native features and APIs, facilitating the

seamless integration of device-specific functionalities Imagine a scenario where adeveloper seamlessly integrates the native camera functionality into a mobile applicationusing React Native, enhancing the overall user experience

Optimized Performance with Native Controls:

Through the utilization of native UI components, React Native ensures optimized performance crucial for delivering a smooth and responsive user interface In a tabletapplication, utilizing native controls for complex interactions ensures a high level ofperformance, contributing to a positive user experience

Navigation is a critical aspect of mobile application design React Navigation, a dedicatedlibrary for navigation in React Native, simplifies the implementation of intuitive userexperiences For instance, stack navigation can be utilized to create a seamless flowbetween different screens, enhancing the overall user-friendliness of the mobileapplication

3.1.2.2 Core concept of ReactJS

e Components: components are the essential building blocks of a React application

They are self-contained, reusable code units designed for rendering specific UI sections.React components come in two forms: class-based or function-based The prevalence

of function-based components has increased with the introduction of React Hooks Here

is an illustration of a basic functional component This component serves as an HTML

input accompanied by a label Subsequently, I can employ this component within formswhere user input is anticipated.

const Input = (props) => {

return(

<>

<LabeL fon=‡{props input1d†>{props LabeL}:</LabeL>

<input id={props.inputId} name={props.inputName} type="text" />

</>

Figure 3-2 Example of components in ReactJS

e JavaScript XML (JSX): JSX functions as a syntax extension for JavaScript, enabling

the incorporation of HTML-like code within your JavaScript code This facilitates the

creation and comprehension of component structures, and JSX is converted intoJavaScript during the build process In the aforementioned example, I utilized JSX by

returning a JSX element

<>

<LabeL for={props inputTd}>{props LabeL}:</LabeL>

<input id={props.inputId} name={props.inputName} type="text" />

</>

Figure 3-3 Example of JSX element

e Props: with React, information is transmitted from parent components to child

components via "props," an abbreviation for properties Props are designed as only, implying that a component must not alter its In the provided example, I employthe Input component within my Registration component for gathering name and addressinputs On each occurrence, I provide the necessary props to the Input component The

read-Input component, in turn, utilizes these props to present the input field with the

specified label, name, id, and other relevant attributes

const Input = (props) => {

<Input inputId="name" inputName="name" Label="First Name"/>

<Input inputId="address" inputName="address" Label="Address"/>

</form>

)

}

Figure 3-4 Example of Props in ReactJS

Event Handling: In React, the management of events involves the use of event

listeners attached to DOM elements These event listeners are commonly implemented

as functions within function components

3.1.2.3 ReactJS with AI Hairstyle

ReactJS offers a UI library known as Ant Design, specifically designed for Reactapplications Utilizing Ant Design, we can rapidly and easily construct the administrativewebsite for our system

Ant Design stands as an open-source design system created by the Ant Group, the parentcompany overseeing Alibaba, Alipay, Huabei, MYbank, and various other entities Thiscomprehensive component library is compatible with React, Vue, and Angular front-endframeworks

® 2

Our administration website is used for managing hairstyles in our system, so it requirestables to display data and many forms to perform CRUD action with our database Here

are the reasons why Ant Design can help use in building the admin website:

e Rich Set of Components: Ant Design offers a comprehensive collection of

components such as buttons, forms, navigation menus, modals, tables, and more These

components are designed with a consistent look and feel, making it easier to createcohesive and professional-looking interfaces

e Customization: While Ant Design components come with predefined styles, they are

highly customizable You can adjust the appearance, theme, and behavior of thecomponents to suit the specific needs of your application

e Grid System: Ant Design includes a responsive grid system that allows you to create

flexible and responsive layouts for your applications

execution of JavaScript scripts on the server side, a capability pivotal for generating

dynamic content in web pages before delivering them to user’s browsers

NodeJS, while not a framework itself, forms the backbone of web-based framework likeExpress, leveraging Chrome’s JavaScript runtime for constructing agile and scalablenetwork applications

Node.js was built and developed since 2009, supported by the company Joyent,headquartered in California, USA It is an independent development platform based on theV8 JavaScript engine It is an interpreter that executes JavaScript code, allowing thecreation of web applications such as video clips and forums, especially with the ability toquickly and easily scale or narrow down the scope of social networking sites

This is considered an advantage as Node.js can operate on various operating systems, fromWindows and Linux to OS X Node.js provides a rich library in the form of differentJavaScript modules that simplify programming and minimize the required time

The main idea of Node.Js is to utilize non-blocking, fast routing of input/output data

through real-time tasks Because Node.js has the ability to rapidly scale and handle a large

number of concurrent connections with high throughput

Most traditional web applications inefficiently use system resources when requests create

new processing threads and consume a significant amount of system memory Therefore,

the solution that Node.js proposes is to use a single thread, known as Single-Threaded inEnglish This combines non-blocking I/O to handle requests, allowing it to support tens ofthousands of concurrent connections

Advantages of NodeJS:

e Asynchronous event-driven I/O allows for the processing of multiple requests

simultaneously.

e It utilizes JavaScript, a programming language known for its ease of learning.

e Facilitates code sharing between the client and server sides

e NPM (Node Package Manager) and Node modules are continually growing and robust

e Benefits from an active and supportive community.

e Allows for streaming of large-sized files.

Disadvantages of NodeJS:

e Each callback often comes with many nested callbacks, requiring careful management

of asynchronous operations

e Proficiency in JavaScript is crucial for effective development.

e Not well-suited for tasks that demand heavy CPU usage due to its single-threaded

nature.

3.1.3.2 Core concept of NodeJS

e Module: The control panel provides a debugging method similar to the basic JavaScript

console in a web browser It will print messages to stdout (standard output) and stderr(standard error)

e Cluster: Node.js is created and developed based on the idea of single-threaded

programming Cluster is a module that enables multithreading by creating child

processes that share the same server port and run concurrently.

e Global variables: Global variables, also known as global variables in Node.js, will

exist in all modules, encompassing functions, modules, strings, and more Some

common global variables you may encounter include dirname, filename, exports,

module, require, and others

e Error Handling: When executing commands, Node.js has an error reporting

mechanism, and you may encounter errors such as Standard JavaScript errors, Systemerrors, User-specific errors, and Assertion errors Errors in Node.js are handled throughexceptions and need to be promptly and efficiently addressed for the application tooperate normally

e Streaming: this is a tool that allows you to read and write data continuously Buffers

are often used to provide permission to handle streams containing binary data Streamsare divided into 4 types:

o Readable: A stream from which data can be read

o Writable: A stream to which data can be written

o Duplex: A stream that can be both read from and written to

o Transform: A stream that can manipulate data while it is being read or written

3.1.3.3 Nodejs with AI Hairstyle

There are tremendous number of frameworks for server-side development, but Nodejs

stands out of the crowd and is the ideal choice for us to develop our system because of thetwo compelling reasons:

e Non-blocking, I/O: In the context of our system, paramount importance lies in

efficiently serving a substantial number of users who connect to the serversimultaneously Therefore, Nodejs is an ideal choice since it has non-blocking I/Omechanism that allows it to handle many requests in a asynchronous manner

Other server-side environments, such as those employing languages like PHP or Java,frequently adhere to synchronous or blocking I/O paradigms In such systems, each I/Ooperation, for example making a network request, will block the execution of the

program until the operation finishes This problem can lower the performance of thesystem, particularly during the peak hour when there is a high volume of simultaneous

requests.

Nodejs distinguishes itself by using an event-driven, non-blocking I/O model In

Nodejs, I/O operations are processed asynchronously Hence, the server can continuehandling other tasks while waiting for I/O operations to complete This is achieved byleveraging the use of an event loop, which continuously checks for events and executes

associated callbacks

Microservices: Our system is designed in a microservices approach Therefore, Nodejs

is the outstanding choice for us Nodejs is well-suited for building microservices systembecause of the following reasons:

© High Performance: Node.js relies on the high-speed Chrome V8 Engine,

enabling it to handle numerous requests per second with exceptionalperformance.

© Scalability: Thanks to its fast processing and capability to manage an extensive

volume of requests, Node.js proves to be highly scalable, making it well-suitedfor very large applications catering to millions of users

© Event-based: Assists in developing event-driven applications, as Node.js objects

have the capability to emit and listen to events These events can encompass userinteractions such as clicks or system messages, providing a versatile frameworkfor building applications based on events

3.1.3.4 Comparing NodeJS with other frameworks

Table 3-2 Comparison between NodeJS and other frameworks

mm NodeJS Other frameworks

Growing User Base

Rich Ecosystem and

applications

Single-thread, well-suited forhandling concurrent

connections, scalablearchitecture

Boasts a vibrant npmecosystem with extensivepackages

Guarantees consistentbehavior across diverse

Seamlessly integrates withpopular frontend

technologies, e.g., React.js

May have more code structure.

In contrast to synchronousframeworks such as Flask orASP.NET

Multi-threading gets difficult

in writing code, debugging,testing, and porting existingcode

favored solution for managing extensive volumes of unstructured or semi structured data.

MongoDB is a document-oriented database, a type of NoSQL database Therefore,MongoDB avoids the table-based structure of relational databases to adapt to documentslike JSON with a highly flexible schema called BSON MongoDB stores data in JSON-like documents, so each collection can have different sizes and different documents Thedata stored in JSON-like documents makes queries very fast

MongoDB was initially introduced by MongoDB Inc In its debut in October 2007, duringthe 10th generation, it was part of a Platform as a Service (PaaS) product like WindowsAzure and Google App Engine It was later open sourced in 2009

MongoDB has become one of the most prominent NoSQL databases, serving as thebackend for numerous websites such as eBay, SourceForge, and The New York Times

3.2.2 Core concept of MongoDB

e Database: A database is the primary container for collections in MongoDB Each

database has its own set of files on the system's file system A typical MongoDB serveroften hosts multiple databases

e Collection: Similar to a table in MySQL, a collection is a container for MongoDB

documents One key difference from other RDBMS is that a collection does not enforce

a specific schema Documents within the same collection can have different fields.However, it's common for documents in a collection to share some key fields and berelated to each other.

Document: A document is a set of data in key-value format, where each key

corresponds to a value Documents are quite flexible in terms of schema; as mentionedearlier, documents within the same collection do not necessarily have to share the samefields or structures Additionally, data within the same field can have different data

types.

JSON: MongoDB documents use the JSON (JavaScript Object Notation) format, which

is a standard for simple and lightweight data storage and exchange With the advantages

of being easy to read and understand, JSON is widely supported in most popularprogramming languages today, such as C, C++, C#, Java, JavaScript, Perl, Python

Data in JSON is stored in a key/value format, where each key corresponds to a value.The value can be an array, a string, an integer, a double, an array, or an object,among other data types

3.2.3 MongoDB with AI Hairstyle

Our system exclusively manages information related to hairstyles, eliminating the need for

SQL databases This is why MongoDB stands out as the ideal choice for our requirements

Besides, Moreover, MongoDB offers some advantages to our system:

Automatic indexing: MongoDB automatically creates indexes to improve queryperformance

Flexible schema design: MongoDB employs a document model that is adaptable andlacks a rigid schema, enabling dynamic and changing data structures within acollection

High performance: The query speed (find, update, insert, delete) of MongoDB issignificantly faster compared to relational database management systems (RDBMS)