mini project american sign language observer technology aslot

TABLE OF CONTENT American Sign Language Observer Technology ASLOT I9 ao n--n-iic$Ö5.... American Sign Language Observer Technology ASLOT Language plays a vital role in the first stages

UNIVERSITY OF ECONOMICS AND LAW VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY

MINI PROJECT

American Sign Language Observer Technology

Class:

Lecturer:

Class code:

Group members:

(ASLOT)

Digital Transformation and Artificial Intelligence Nguyen The Dai Nghia

222MI5216 Tran Pham Anh Duy — K214020172 (leader) Nguyen Quang Dat — K214020174 Nguyen Ngoc Thao Nguyen — K214020180 Nguyen Quang Phat — K214020181

Le Phuong Thao — K214020183

07 March 2023

IL

IH

IV

TABLE OF CONTENT

American Sign Language Observer Technology (ASLOT)

I9 ao n n-iic$Ö5 PNS-iiiaidddidiiidddẳŸ Ea e4

A, Applicaton oo Conclusion -a Teachable Machine model link 2c 2221135221122 5212111185111 11 kee Trello Observers - 2c Q1 Q n2 2011 nS 12111 K ST 111k nh 1k1 1k kh hy cà

American Sign Language Observer Technology (ASLOT)

L

H

Overview

Teachable Machine

Teachable machine is a AI generated web based GUI (Graphical User Interface) tool that makes creating machine learning models quick, simple, and available to everyone With the Teachable machine - a project by Google, people use the camera or audio to rapidly create various models There are 3 distinct options: images, sound and pose recognition It is used to teach our computer how to identify and classify different images and poses through the input images or audio used by a webcam This AI allows us to train our machine learning model with the click of a button, no coding required, and export it

to a variety of platforms such as websites, apps and more

American Sign Language Observer Technology (ASLOT)

Language plays a vital role in the first stages of brain development For the deaf and mute, sign language is considered as a “mother tongue” and is the only means for them to communicate with those around them That is why deaf and mentally disabled people were considered a form of intellectual disability until the 1970s

Deaf and mute children are especially among the most difficult subjects in special education Therefore, boosting access to and use of sign language for deaf-mute students will contribute positively to improving the quality of education and integration for this group Understanding this problem, our team is conducting a mini project to fully exploit the function of a learning machine named “ Teachable machine”

Having realized this novel and useful invention, our team has created 10 basic sign movements along with an alphabet table and given these inputs into the Teachable machine for teaching and classifying actions into categories This mini project - ASLOT (American sign language observer technology) firstly aims at applying sign language into artificial intelligence for everyone who wants to learn signs to communicate with the deaf

- mute Then we will implement and explain carefully about features, function, ways to conduct, pros and cons of the application and to conclude with the summary of this

project

Feature

The American Sign Language Observer Technique (ASLOT) is a teachable machine model that uses visual sign patterns mainly from hand and finger images to simulate how humans express and convey American sign language (ASL) In the process of researching and developing this prototype, we decided to apply a few words in the American sign

2

language alphabet and numbers as well as some “basic poses” for daily communication

by people every day in the ASLOT to experiment with its feasibility

Ý § 2 ạ 6

Ô Ø œ b$

J 8 # ø 8:

Ê- a: ge 0: gf

fot Ye oe py

ầ:

Exhibit 1: American Sign Language Alphabet

1

WW

Vi Vue Exhibit 2: American Sign Language numbers

ASLOT would be granted the ability to recognize people’s images (taken from their webcam or image files) without requiring any machine learning code Being in use, ASLOT is able to distinguish and detect which output would be matched with users’ image samples Besides, they can go to check the words to see if they have got it right when being unsure of any certain sign

As aresult, users will be able to learn and practice American sign language over time

= Teachable Machine

Preview — F _ Export Modo!

203 Image Sample: <

a — i |

Webcam — Uplssd ¬

138 Image Samples =

o L PL Pi : i ye { Model Trained Ð

HA — fe Pe te fe fe |~

°o

305 Image Samples

m +

Webcam Upload a c 5

Exhibit 3: The ASLOT preview

There are considered many different sign languages used around the world, but we came to the conclusion to experiment with the American sign language because it is widely used and accepted in the United States, and the fourth-most studied the second language at American universities As ASLOT is still in the experimental stage, we continued to filter out and choose particular gestures that is suitable and linked with our model because ease and feasibility are now our top priorities Until Teachable Machine is installed and added with more specific and outstanding features, some other complicated and multi-active movements would be later considered in ASLOT

During the experiment, we figured out two opposite sides of our model:

Advantages:

+ It is easy for everyone who wants to learn about it to access In some cases, with the completed models, people just download it on their devices and use it every time in everywhere they want

+ No coding requirements, just put the images or records into your computer, the machine will automatically learn it in a few seconds, and it will release results rapidly and accurately in the testing step

+ Perfect tool for anyone that has interest in artificial intelligence in general and machine learning in particular They can research and explore AI without requiring too much expertise beforehand

Disadvantages:

+ High resolution requirement When the input is stmple, this machine can recognize and decode based on the previous images that have just been put in By contrast, with complicated and continual gestures sequence, high resolution of the camera or devices are needed This drawback could dramatically limit the user's operation

+ Poor interface optimization: Although TM is simple to use, it lacks functions named as

“auto save’, “fast save” because it is a web-based application Honestly, unintended incidents can destroy a user's entire project if they haven't saved it yet Furthermore, when working with a variety of classes, there is no option to display all classes at once Instead, users have to continuously scroll up and down to see the outputs, and especially

in some cases, reading the output necessitates not being able to see the camera at all

3 Function

3.1 A timely “translator”

Let’s imagine that you are about to have a volunteer trip to a secondary school for disabled students, so the question is how can you understand and communicate with those children? Just like traveling to a new world, having a translation app is seemingly one of the essential things you really need, regardless of whether you have approached to the sign language before or not In a similar way, ASLOT would become the optimal choice for you to have a conversation while coming to the world of people with disabilities

Facilitating effective communication

With an ASLOT, users can get a close comprehension of the others are saying and simultaneously have better story-telling and daily communication between everyone ASLOT would also help the disabled to communicate by fully expressing themselves in both simple and complex ways to ordinary people and vice versa When the disabled are in trouble, having an understanding of sign language is a great way to solve the circumstances much faster and easier Furthermore, it is undeniable that you can still have basic communication with people with disabilities whether you have gone through sign language before or not

Boosting self-esteem

Having something to say but being unable to express it is one of the most frustrating things

If the disabled can’t get their message across, then they will never feel included in conversations or activities Consequently, this triggers a sense of self-isolation and frustration, especially for children or teenagers Having ASLOT might provide them with an alternative way to make themselves understood This extra tool enables them to express how they feel, their thoughts and wants so that they can engage in many social activities without any hesitation or worry

3.2 A learning tool for one and all

ASLOT comes into existence to create a common platform for not only the disabled but also ordinary individuals to access and use Our technology specifically aims at “one and all” for the purpose of narrowing and eliminating the barners between all groups of people Generally, the disabled would no longer suffer from contempt and alienation but be able to integrate into the community with the help of this top-notch invention

Promoting understanding and acceptance

ASLOT promotes better awareness of and sensitivity to the deaf and mute community When someone is proficient m sign language, they will develop a strong appreciation for the culture of people with disabilities, and they can promote understanding and acceptance of the language among others There would no longer exist disrespect or toxic discrimination as learning sign language might nurture and foster a thrill of sympathy to those unfortunate souls At the same time, people with disabilities would have a brighter chance to integrate into the common life without having to hide or nestle themselves behind the flow of this society anymore

Building and strengthening the relationships

Communication problems are one of the main barriers to building relationships Witnessing issues in the community for the disabled, ASLOT focuses on opening up these communication barriers and allowing the disabled to freely interact with others In addition,

6

learning how to sign for other people is also promising, as it introduces them to a brand new language This enables both parties to meet new people and make new friends as they are learning and progressing together For instance, two friends who are both deaf can teach each other how to use sign language or the mute can teach the ordinary one how to communicate, which gradually creates a bond between them

Accessing more career opportunities

One especially exciting career path open to bilingual hearing professionals is interpretation There is a great need to increase the availability of qualified American sign language interpreters in the community and mainstream programs in schools and colleges or universities Some places interpreters are in demand include hospitals, courts, governmental agencies, community activities, and local, county, and state legislatures For those passionate about any job related to sign language interpretation but cannot afford to attend training courses, ASLOT can be used as a means of self-study

Exhibit 4: ASL Interpreter Justina Miles in Rihanna’s Super Bowl Halftime

4 Application

We intend to use ASLOT in the design of a sign language translation app with the same name ASLOT This app can be used on both phones and laptops to bring certain conveniences to users and has a similar function to other translation apps such as TFlat, Cambridge dictionary, or Duolingo

First of all, we want to set up the software to learn American Sign Language as well as install additional search for languages according to the user's gestures The combination of lessons, tests, and library and settings would be added to decrease the feeling of tedium, and typically the features of making friends and calling would also be considered in the app If we are lucky enough to succeed in the market, we plan to go further by installing more sign languages so that users can flexibly change and choose the language that suits them Although people can use their phones to Google and look up different language symbols, we believe that the convenience and creativeness of this app can reach many people who really need it

Exhibit 5: ASLOT’s main theme app

HH Conclusion

In brief, Teachable machine 1s one of a breakthrough of advanced AI, especially ASLOT

is considered as one of practical experience on this web-based tool Although having inadequacy, in the future, human intelligence collaborate with the convenience and productivity of AI will enhance the aforementioned inadequacy

IV Teachable Machine model link

https://teachablemachine withgoogle.com/models/UW21mxhM3/

V Trello observers

Mini project Al - Teachable machine % & Workspace visible a

“Ở ‹PowerUps # ÄutomalfÐw

Essay structure Meeting Essay over + Table of contents Assignment topic and tasks division

Features - Fuctions of AS ®®®©®O

pp! Í

in modern society â Labels

& Checklist

â Dates

a Google Drive @ Enable Google Drive

đệ

our Drive files for a project directly from its card, or create and attach new Drive files to

& Cover

@ Upgrade

+ Add Power-Ups

Automation @

@ Thảo Lờ Phương 4 hours 299 (

@phatnguynquang okee merci pour votre feedback @

â Make template

E Archive

10