(Đồ án tốt nghiệp) research, design, manufacture of distributed control service robot system

INTRODUCTION

Overview

Currently, along with the socio-economic development, our country is more and more developed with research and achievements in the field of science and technology Our country is going from an agriculturalized country to industrialization and modernization, in which process modern technologies are widely welcomed and applied to many areas of life This contributes significantly to the development of service industries in the economic field in the current period of innovation, accounting for most of which cannot be ignored, including the food and beverage service industries Thanks to economic integration policies, the F&B industry (Food and Beverage Service) in our country has been developing strongly and increasing day by day Besides, it is not too strange to see delivery robots in shops and restaurants in China, Japan, etc., which can not only create good effects for customers to use, but also can easily bring people access 4.0 technology devices This has promoted the development of science and technology in the F&B industry in the world in general and our country in particular.

Overview of Service Robot

According to the International Organization for Standardization (ISO), a "service robot" is a robot that performs some useful tasks for humans Unlike industrial robots (which automate production processes), service robots are different in both roles and forms, designed to flexibly respond to different task situations and circumstances to serve customers good service to people.

Through the existing control system, the service robot can fully or partially operate on its own, that is, it can interact with humans to complete the work. Currently, service robots are used in a wide variety of industries including retail, hotel and accommodation, hospital and healthcare, space, security, storage and warehouse, agriculture and many more high-risk jobs.

One area that is rapidly becoming mainstream is contactless delivery Earlier this April, Venezia's, a pizzeria in the state of Arizona (USA), like other local restaurants, struggled with lockdown measures to prevent the spread of COVID-19 have received

1 some new help Those are 11 delivery robots manufactured by the US company Starship Technologies Pizzas are placed on the covered pallets of this fleet of driverless vehicles that are delivered to households in the neighborhood With the arrival of robots, Venezia's delivery operation has been upgraded to "contactless", and this type of delivery service is gaining popularity among customers amid the pandemic.

Starship Technologies said that the demand for contactless delivery has increased sharply in the past few weeks The company's robot is used in Washington, D.C., (USA) to deliver food from stores to consumers, and in Southern California, the robot has been delivering milk tea since the end of March.

In the US and elsewhere, self-driving car services that deliver "within 1 mile" to consumers' homes, which have been seriously deployed, are now being boosted in an effort to prevent the virus from spreading.

It is highly unlikely that robots will replace humans anytime soon even though the technology will evolve in the long run As it becomes more and more affordable to build and maintain robotic staff, whatever form they may be involved in, the human composition of the restaurant space can change.

Restaurant's computational management process will be simpler without being too dependent on people, because restaurant staff are now mostly part-time, difficult to work stably and you have to manage waiters, so the instability of personnel can cause the restaurant to lose quality or uneven service quality About your restaurant being a robot owner will help you take the initiative in serving problems, and serve customers better.

Research situation

The development of science and technology has received more and more attention from most countries in the world The 4.0 races continuously broke out to be able to

2 affirm the development and progress among countries At the same time, there are more and more diverse applications of science and technology in people's lives, from family to society, even to the military Besides, the development of service industries of other countries is equally exciting, since then, many researches, inventions, and production of information tasks in restaurants, hotels, hospitals, schools learn,… More specifically, in the food service industry, modern technologies have been applied in the process of making war or serving customers.

Figure 1.2 Service robots at restaurants [2]

Figure 1.3 Sacarino Service Robot in a Hotel Environment [3]

In our country today, there are also many restaurants and eateries that apply automatic technologies in the operation process The famous hot pot restaurant

"Conveyor belt" Kichi Kichi with hot pot side dishes is automatically transported to the available customer location via a pre-designed long conveyor belt Or Ho Chi Minh City University of Technology and Education is quite famous for its Synary coffee shop with the presence of a serving robot, etc And there are many other restaurants and restaurants that have been and will be applying science – technology into the system later Therefore, the topic of robots serving in restaurants and catering is always a topic of great interest and research by researchers, engineers, and students.

Reasons for choosing topic

Nowadays, people's life needs are increasing, people often spend their free time outside and integrate more Going out or eating at restaurants, bars, and shopping centers is one of the most attractive places chosen by many people on weekends, holidays or gathering with friends Or meetings and exchanges due to the nature of work in luxury restaurants We can easily come across videos and images on social networking sites about modern products in other countries, in which most robot products account for the majority of people's interest and are frequently mentioned Not too surprising, because the robot is the representative of the development of science - technology, always being watched, interested and extremely excited if there is the appearance of robots in life. Our country is in the stage of going up in many aspects such as economy, service, science - technology There have been many researches and production of robots applied in many areas of life such as massage robots, medical examination and treatment support robots, cargo transport assistance robots, assistive robots in other agricultural and industrial sectors One of the fields that are being researched, manufactured and applied in daily life by a large number of countries around the world is the application of robots in the restaurant and food service industry A delicious meal can help us become happier, relieve stress and fatigue in life, is also a time when we can spend more time chatting with family, relatives or friends his dear Therefore, in order to create a more comfortable and interesting space for customers to come and eat, the team came up with the idea to research and build a robot to serve in restaurants, catering in particular and the service sector service in general.

Objectives of the topic

Research, design and successfully manufacture restaurant service robots into reality Build the ability to self-locate, move to the required location with the optimal path, voice communicate with customers through a pre-designed interface Further develop control of the multi-robot distributed system, which is self-propelled in the working environment, communicate and transmit data to each other while the system is operating The application is artificial intelligence to help the communication between customers and the restaurant become more effective, reduce the workload of service staff, order and serve food more accurately.

Research Methods

The main research method used by the group is a combination of document research, on the basis of a theoretical foundation for design and fabrication and experimental evaluation In terms of theory, in addition to the theoretical knowledge taught at school,the team consulted more documents related to the F&B industry and service robots at home and abroad, and conducted practical surveys to monitor the process Work of a waiter at restaurants, to design a robot that functions as a waiter and is easiest to manage.

Limit of topics

Because of the limited research time and capacity, this topic mainly focuses on the design, manufacture and control of movement to the desired position with optimal path Ideal working environment: not too many people and obstacles passing when the robot is moving.

THEORETICAL BASIS

Overview

To be able to design a robot as a restaurant waiter, it is necessary to understand how the restaurant operates as well as how a waiter works, from which it can provide functions for the robot to suit the operating environment.

Definition of service robot

A service robot, also known as a service robot, is a robot that serves people, it supports people in many different jobs from simple to complex In particular, service robots are put into use with remote jobs, dangerous and dirty jobs that affect human health This robot is also applied to repetitive tasks, or daily housework.

The benefits of using Robots to serve humans:

• Bring more convenience to business establishments, reduce the cost of hiring human resources and management costs.

• More secure and will not affect human health, as well as create the best work efficiency.

• Makes it do the job faster, smarter, handle more diverse situations in the service industry.

Definition of Distributed System

“A distributed control system (DCS) is a computerised control system for a process or plant usually with many control loops, in which autonomous controllers are distributed throughout the system, but there is no central operator supervisory control. This is in contrast to systems that use centralized controllers; either discrete controllers located at a central control room or within a central computer The DCS concept increases reliability and reduces installation costs by localising control functions near the process plant, with remote monitoring and supervision”.

Also known as distributed computing and distributed databases, a distributed system is a collection of independent components located on different machines that share messages with each other in order to achieve common goals.

As such, the distributed system appears as if it is one interface or computer to the end- user The hope is that together, the system can maximize resources and information

6 while preventing failures, as if one system fails, it won't affect the availability of the service.

The most important functions of distributed computing are:

• Resource sharing - whether it’s the hardware, software or data that can be shared.

• Openness - how open is the software designed to be developed and shared with each other.

• Concurrency - multiple machines can process the same function at the same time.

• Scalability - how do the computing and processing capabilities multiply when extended to many machines.

• Fault tolerance - how easy and quickly can failures in parts of the system be detected and recovered.

• Transparency - how much access does one node have to locate and communicate with other nodes in the system.

Modern distributed systems have evolved to include autonomous processes that might run on the same physical machine, but interact by exchanging messages with each other.

Robot Knowledge and Dynamics

2.4.1 Kinematics of the Robot Platform

The kinetic model of the mobile robot is represented by a system of equations that allows to predict the position of the robot in its orbit, determine the position and direction of the robot along with the control process, all the movements of the mobile robot can be divided into translational and rotational motion Translation is the displacement of the moving robot's center Rotation is the rotation of an axis of the vehicle The basic element of any mobile robot is the wheel Kinematic parameters include wheel radius, wheel axle length, center of gravity Simplified kinematic model consisting of 2 degrees of freedom (DoF) with 3 wheels including 2 driving wheels and 1 self-selecting passive wheel (caster) The wheel configuration is completely described by 4 variables: position coordinates (x, y) on the plane relative to the fixed axis system,

7 the direction of rotation of the wheel relative to the x axis is w and the angle w is the angle formed by any of the radial and vertical axes, r is the wheel radius.

Simplify the moving components to calculate, here the wheel is a non-slip rotating vertical disc Expressed through 4 variables in the coordinate system, which is the position (x, y) on the plane relative to the fixed axis system and the wheel rotation direction for each axis system in the Oxy plane Because the ability to slip has been controlled, the vehicle speed meets the following controls:

The position of the robot's center is represented by the reference axis system (x, y, ) Here x and y are the robot position and is the direction of the robot relative to the reference axes Suppose the robot moves on a plane with a linear velocity and an angular velocity that can be represented as a vector as: q = x y

Controlled by 2 differentially active motors, the robot's trajectory is a straight line given the wheel's bound velocity and the trajectory is a defined arc with a deviation. speed of 2 drive wheels The robot state is determined by the position and direction on the two-wheel speed (x, y, θ, ωl, ωr) Straight and angular velocities ω are given by the control programmer for the left and right wheel angular velocities.

The kinematic model is developed on the robot's movement model according to the time function from which it is possible to predict the position and direction of the robot based on the initial values of the robot body, through determining the position and motion of the robot speed of robot body by analyzing wheel speed and direction Each robot element is built in 2D space of the Decartes coordinate system The position of the robot can be determined on the global reference axis system (X, Y) by the center of the wheel hub connection and the angle θ The rotation matrix is used to plot the trajectory of movement in the global system through the rotation around the z-axis. cos −sin 0

The rotational speed of the left and right wheels, the wheel radius r and the distance l/2 from the robot center to the wheel are known, so the robot's velocity model can be predicted With the robot moving with differential wheel, the 2 drive wheels are fixed so that the positive rotation creates a positive displacement around the axis The rolling and sliding constraints for a fixed standard wheel ensure that all rotations are generated by the wheels producing simultaneous translation and rotation of the robot.

In the case of differential transmission, to avoid slippage and have pure rotational motion, the robot must rotate around a point on the straight line connecting the two wheels This point is known as the instantaneous center of rotation (ICR) When changing the speed of the two wheels, the instantaneous center of rotation will move and create a different trajectory [27]

Figure 2 1 Base model From the above model, the long speed of the platform is determined as the average speed of the two wheels The action of the robot depends on the values of the control variables v and v =( l + r ) r

The action of the robot depends on the control variables v and

Robot movement can be described with two modes, pure rotation and pure translation Newspaper travel paths consist of circular arcs with distinct radii and

9 rotation angles along with straight tangent lines Curves and straight tangent lines are used to avoid sudden stops and provide continuous operation for the robot To achieve the controlled trajectory, the linear and rotational velocities v and θ are calculated through the fuzzy controller as follows: x v.cos y = v.sin

In general, we can describe the position of the robot based on its velocity in each instantaneous time interval t : x(t) = ∫ 0 ( ) ( ) y(t) = ∫ 0 ( ) ( )

Given the special case of the differential wheel robot as done, the formula becomes: x(t) = 1 ∫ [ ( ) + ( )] ( )

From there we can calculate the position of the robot through the instantaneous velocity of each wheel.

Controlling the robot is creating a moving path and the motion controller must follow the movement route as accurately as possible The two drive wheels are independently controlled by the motor and the self-selecting wheel will ensure the robot balance when moving on a flat surface Assume that the movement of the self- aligning wheel can be ignored when calculating the dynamics problem The model of the mobile robot is shown in Figure 2.2, where l represents the distance between the two wheels and r is the radius of the wheel.

We have 2 kinematics: no horizontal and vertical sliding between the wheel and the platform According to these two constraints, the robot can be described as follows:

Where (x C , y C ) are the coordinates of the robot center is the angle formed by the robot direction and the x-axis, l and r are the rotation angles of the left and right wheels The kinetic equation of the mobile robot on wheels can be written as follows: (2.15)

It is suitable to represent the dynamic equation of motion with internal velocity. When the robot is the goal of nonholonomic control then:

DESIGN OF MECHANICAL SYSTEM

Overview

At a time when robots are developing strongly like today, many mobile robots are created with diverse designs and multi-fields In order to create restaurant robots, the mechanical design process must be reasonable, synchronous and systematic with the original goals as well as convenient for the installation of electrical equipment, control systems, operating space The challenge of the project is not to be obstructed by tables,chairs, and some other obstacles in reality Precise mechanical design will help achieve better results.

Design option

Determining the problem is that we need to design a device with the following functions as analyzed in Chapter 1:

Screen Transportation of food and Maintain energy interaction water

Figure 3.1 Functional block diagram of robot Function 1: The device is capable of self-propelled movement on flat ground such as in restaurants, cafeterias, families , these floors are flat without leveling Therefore, this self-propelled device has the function of a self-propelled mobile robot, capable of moving on a flat surface.

Function 2: The restaurant service robot interacts through the touch screen with a menu-shaped graphic interface, possibly accompanied by audio instructions. Interactive screens must be able for customers - staff / users to easily observe and communicate with activities such as: presenting menus, being able to give analysis and advice on dishes to customers, communicating customer information to order inside. Function 3: The mobile robot is capable of transporting the ordered dishes, besides it can transport drinks and other desserts.

Function 4: The robot must operate independently of the managed and monitored energy source.

Requirements in mechanical design

Aiming to create a characteristic design, possessing the necessary requirements of the serving robot such as viewing the menu, communicating with the user as well as ensuring the size constraints must also be kept in mind Peripherals supporting the robot such as touch screens, microphones and sensors are also arranged appropriately to ensure design requirements and operability The solid mechanical part will help support the entire weight of the robot, in addition, it also contributes significantly to the flexibility and accuracy of the robot.

After converting the requirements in the problem definition into functionality we need to define the requirements in the design The outside shape must be aesthetically pleasing, user-friendly, with a height of 1.2 to 1.6m with an interactive screen oriented in accordance with the customer's viewing angle Weight < 60 kg to be able to move the robot easily to many locations The robot ribs must ensure factors such as:firmness, firmness, compactness, easy disassembly and repair, so aluminum is used for the outer frame The energy used for this self-propelled robot must be independent, so the electrical energy is stored in the battery, capable of operating for a period of 8-10 hours Dimensions and functional blocks must be suitable for Vietnamese anthropometry The overall function of this robot is to order and move food to the table for customers with 4 sub-functions.

Matrix of ideas to determine the design option

Based on functional analysis, we have options (ideas) to design each sub-function as (Table 3.1) With the moving platform, there are many options to create self-propelled motion: wheel movement, leg movement, flying movement Thus, there are 4 options to adjust the screen's interactive orientation Another function of this robot is to be able to transport food and water, move to a required place, so the robot must have at least one compartment or container for food and water and must ensure proper storage, bridge With

4 options offered: Mount the container on the pedestal, the container is in the pedestal, the container moves according to the pedestal, drawer storage However, for ease of design and ample space, the team chose the option of a container placed above

13 the movable platform, making food and water independent, unaffected by moving equipment With the charging function, there are also 4 options given, depending on the needs, the appropriate choice is as shown in (Table 3.1).

Table 3.1 Matrix of Design Ideas Matrix of Design Ideas

Moving platform differential sprocket-wheel Multi-directional Kind of like a scroll wheel car transmission

Detachable with Attached to the

Drag the system Drag from the pedestal pedestal pedestal

Wireless charging Wiring in operation

With 4 functions and 4 ideas for each function, we can combine them into 44 design options However, with the weights set in the design with the design requirements set out in the above section in order to meet the reasonable price, moderate size, flexibility, 19 independent operations, we choose the option with the following options: Moving platform with 2 degrees of freedom with drive on 2 wheels with differential active control and 2 with self-selecting wheels (idea 1), fixed screen, to keep it fixed, avoid fields In case of damage to equipment, the robot can come into contact with many different types of customers The power supply will use the battery,because it has to operate for a long time, so it needs a large power supply, the user can charge it at the end of the day, so it can ensure the battery life stable in use Wireless charging options are too expensive, pulling the cord when moving is not suitable for operating in large environments, or replacing the battery when it runs out is also difficult and requires a technician.

Design of the movable pedestal

The design pedestal is shown in (Figure 3.2) Made of aluminum alloy to minimize the weight of the robot while ensuring the rigidity of the entire structure. The moving platform of the robot is divided into three floors (Figure 3.2):

Figure 3.2 Moving platform Floor 1: Because it is the lowest floor, it is responsible for containing the motor and actuators.

Floor 2: Because the program computer as well as the central controller are quite complicated, the battery will be left on the second floor.

Floor 3: Contains the central controller and the entire control circuit.

A mobile robot on wheels is a mechanical device that can move in an environment with 2 fixed degrees of freedom To reduce the area of the robot platform as well as streamline the space of the platform, we choose the belt transmission as (Figure 3.3).

In the early stages of the design process, the team identified key requirements for the robotic platform The differential wheel platform is a two-wheel drive system with independent actuators for each wheel A differential drive system requires only two motors, one for each self-selecting steering wheel Usually the wheel is directly connected to the motor with the reducer inside After reference and selection, the team has selected the transmission diagram as well as the image of the pedestal simulation on the software (Figures 3.2, 3.3)

CHAPTER 4: ENGINE SELECTION CALCULATION AND TRANSFER

Overview

For the system to operate stably, it is necessary to calculate the parameters so that it is reasonable with the function, affordable price, to meet the needs of the user In this chapter, we present the calculation parameters based on the theory, from which to select the appropriate components according to the requirements.

Calculation of engine selection

- Active wheel diameter:d = 145 (mm) Calculation of engine selection:

- Number of revolutions of the wheel:

- Force acting on one wheel: + Weight on 1 wheel:

+ Friction force on one wheel:

+ According to Newton's second law, for the car to move: F k - F qt - F ms > 0

- The drive mechanism is arranged as shown below (belt drive).

We have: + đ = 0,96 : Belt drive efficiency

+ = 0,99 : Transmission efficiency of 1 pair of bearings =>

Mechanical transmission system with belt drive we choose usb = uđ 1 => = u sb n lv = 40 1 = 40 (v/ph)

Engine name P đc (W) n đc Torque η Mass

Transmission Ratio Distribution

Number of revolutions through the impeller: đ 2500/62,5 (4.11) n1 = đ = 1 = 40 (rpm)

Power through the working shaft:

Table 4 2 Table of calculation results

Belt transmission

+ Power on the drive shaft: P đc = 55,56 (W)

+ Number of revolutions on the drive shaft: ndc = 2500/62,5 (rpm)

+ Transmission ratio of belt drive: u d = 1

+ Static load, slight vibration, impact.

+ The belt is periodically adjusted to the belt tension.

Select belt type and cross-section: Choose to fix the ladder belt with teeth

A Determine the diameter of the small pulley:

Determine the axis distance a and belt length L:

- We preliminarily choose the shaft distance a based on the gear ratio ud and the diameter of the pulley d2

Therefore u đ = 1 so choose the ratio a =1,5 2 a sb = d 2 1,5 = 31,4.1,5 = 47,1 (mm)

We choose the standard belt length as: = 254( )

Belt life test according to belt laps per second: i = = 0,254 0.07 ≈ 0,28 (time/s) ≤ i max = 10 (time/s)

Calculate the exact - axis distance according to the standard L:

Calculate the force acting on the belt drive:

But = 2 (periodically adjust the tension), with = 0,105( / )

- Select K d = 1,2 (due to 2 shift working mode, light impact load)

- Force acting on the shaft:

Calculation and testing of shafts

Force from belt drive and wheel:

Inside: + Torque I shaft (working): T (N.mm)

+ Radius of belt divider: r (mm) + Vehicle mass: m (kg)

+ Gravity acceleration: g (m/s 2 ) Choose: l 1 = 30 (mm) l 2 = 40 (mm) l3 = 80(mm) T 1 = 12606 Nmm

Figure 4.2 Force analysis on transmission Projection on the zOy axis:

Figure 4.3 Force analysis when projecting on the xOy axis

Projection on the zOx axis:

Figure 4.5 Qy and My graph

Figure 4.6 Qx and Mx graph

Diameter of dangerous cross-section B, C:

- Shaft diameter at cross-section at B and C:

So we choose the shaft diameter = 21( )

4.5.1 Test shaft for fatigue strength:

+ The axes of the reducer are all rotating, the bending stress changes with the period of symmetry, so according to the formula we have:

+ Because the shaft rotates in one direction, the torsional stress changes with the dynamic circuit period, so according to the formula we have:

+ Based on the shaft structure and the shaft moment chart, we can see that the following sections are dangerous sections that need to be tested for fatigue strength:

Cross-section for mounting the pulley (section 12).

+ Selection of mounting: Rolling bearings are mounted on the shaft according to k6 ; pulley, shaft connection according to k6

+ Determine the coefficients for hazardous sections according to the formula:

Axes are machined on lathes, at required dangerous cross-sections = 2,5 ÷ 0,63( ), so according to table (10.8), stress concentration coefficient due to surface state = 1,06

No surface strengthening methods are used, so the strength coefficient = 1,6 According to table (10.12), when using end mills, the stress concentration coefficient at the keyway corresponding to the material with = 600( ) is: =

According to the table (10.10), look up the dimension coefficients and corresponding to the diameters of the hazardous sections, thereby determining the ratio and at the keyway on these sections.

According to the table (10.11), corresponding to the selected mounting type,

= 600( ) and the diameter of the hazardous section, the ratio and due to tension mounting at these sections, on the base, can be found The facility uses the larger of the two values of to calculate and the larger of the two values of to calculate × 3 21 3

Factor of safety considering only normal stress:

The factor of safety considering the shear stress only:

Finally calculate the factor of safety:

So the cross sections on the shaft are safe in terms of fatigue strength.

To prevent the possibility of excessive plastic deformation or failure due to sudden overload (such as when turning on the machine) it is necessary to carry out a test of the shaft for static strength.

Cross section at dangerous cross section d = 21 mm

Tangential stress at d = 21 mm; = = 6,8 (MPa)

Steel C45: sb = 600 MPA, sch = 360 MPa

Satisfy the condition of stability.

Select bearing

Depending on the working conditions, at the bearing positions only the ring force

F t and the radial force F r So at bearings 1 and 2, choose a single row ball bearing.

Provided that all rolling bearings are oil-lubricated.

On axis I, supports are located at B and D

We choose 1000905 ultra-lightweight single row ball bearings with parameters:

Forces acting on the bearing:

F y11 = 557,19 N Thus, the total reaction on each drive is:

So we test at the drive with a load greater than F l10 = 2048,4 N

4.6.1 Test drive according to dynamic load capacity

The dynamic load capacity Cd is calculated according to the formula (11.1)

For ball bearings, calculated by the formula: Q = (X.V.Fr +Y.Fa)kt.kđ (4.56)

+ V = 1 coefficients taking into account the revolutions in rotation.

+ k t =1 factor taking into account the effect of heat when working θ = 150 o C.

+ k đ = (1÷1,2) factor taking into account the load characteristic.

We have L h = 10 6 L/(60.n) (4.57) With: L h life in hours.

10 10 m = 3 degree of fatigue curve in rolling bearing test (for ball bearings).

So according to table (P2.1), it is reasonable to choose a 306 medium-sized ball bearing.

4.6.2 Test drive according to static load capacity:

To prevent residual deformation, ball bearings need to satisfy the following conditions:

Q t ≤ C o With: Qt conventional static load (kN).

Calculated according to the formula: Qt = Xo.Fr + Yo.Fa (4.60)

With: + Xo = 0,6 radial load factor

We choose 1000905 ultra-lightweight single row ball bearings, which is reasonable

ELECTRICAL-CONTROL SYSTEM DESIGN

Overview

A complete mechatronic system includes mechanical part, electrical system and control system The mechanical system of the service robot has been presented,designed and calculated in the previous chapters In this chapter, the report designs and calculates the electrical system as well as the control system applied in the topic The electrical system consists of a central control computer, a master microcontroller and a slave microcontroller, along with an actuator system and a sensor system In order to carry out the self-propelled process, the robot needs to process the input signals of the sensors, which are ultrasonic sensors and encoders Through the processing of peripheral devices that give output suitable for self-propelled process.

Electric power system

5.2.1 Communication between computer-microcontroller and microcontroller- microcontroller

The problem of communication between the computer and the microcontroller or the microcontroller with the microcontroller is very important in control and measurement applications In practice, central computers have much faster processing speed than microcontrollers The central computer takes on the role of interacting with humans and sends the service data to the restaurant and navigation to the robot In order for the computer to communicate with the microcontroller, we must adopt a serial communication standard - UART by connecting the computer and the microcontroller via the COM port with the same data transmission rate (baud) between the computer and the microcontroller There are many built-in data types in microcontroller families, including:

USART Universal Synchronous Asynchronous Receiver and

Transmitter SPI Serial Peripheral Interface

2-wire data transmission (I2C: Inter-Integrated Circuit):

In synchronous serial transmission, there are 1 data line and 1 clock line, the device that provides the clock pulse, the device acts as the master, the device that receives the clock acts as the slave, the data transfer rate depends on the clock frequency In asynchronous serial transmission, there is 1 data transmission line and 1 data receiving line, no clock signal, so it is called asynchronous In order to transmit data, both the transmitter and the receiver must generate their own clock with the same frequency and often called the data rate (baud), for example 2400baud, 4800baud,

RS stands for Recommended Standard, and RS232 serial port pairing is one of the widely used techniques to pair peripheral devices with computers It is a serial communication standard using an asynchronous format, connecting at most 2 devices, the maximum connection length allowed to ensure data is 12.5 to 25.4m, the speed is 20kbit/s sometimes is 115kbit/s with some special devices A few advantages of the RS232 serial interface:

- High anti-interference ability of serial ports.

- Peripherals can be removed even when the computer is powered up.

- Simple circuits can receive power supply voltage through series work.

- Commonly used standard data rate values: 9600, 19200, 28800, 38400 56600,

➢ RS232 is very suitable for computer communication with microcontrollers

5.2.2 Communication between actuator microcontroller and sensor

Slave microcontrollers control two BLDC motors, receive data from Master microcontroller and PC to give motor control states, is connected to the sensor block, which is responsible for collecting signals from the environment such as desks, chairs, people,

The Master microcontroller has the function of communicating with the central computer block, sending data as a position string to the Slave microcontroller to perform the migration process.

Just like humans have senses, the sensor system is the same as the senses in automation systems Therefore, the sensor system plays a very important role.

Working principle: The ultrasonic sensor has a principle of operation based on the giving and receiving process, which means that the sensor system continuously emit short sound waves with a frequency higher than that of humans listen and have strong transmission speed When these sound waves encounter an obstacle that is solid or liquid, they create reflected wavelengths Finally, the sensor device receives, analyzes and accurately determines the distance from the sensor to the obstacle.

Figure 5.3 Working principle of ultrasonic sensor [4]

An encoder, also known as a rotary encoder or shaft encoder, is an electromechanical device that converts the angular position or motion of a shaft or shaft into an analog or digital output signal Encoder is used to detect the position, movement direction, speed of the motor by counting the number of revolutions of the shaft.

The encoder's structure includes a circular optical disc, on the disc is punched with small holes divided equally over 360 , the resolution of the encoder depends on the number of these small grooves The larger the number of holes on the disc, the higher the resolution.

The principle of operation of the encoder: When the disc rotates around the axis, there are grooves on the disc for the optical signal to shine through (Led) Where there is a groove, light can pass through, where there is no groove, light cannot penetrate. With the signal yes/no, it is recorded whether the LED shines through or not The light receiver sensor turn on and off continuously, thereby generating square pulse signals, the pulse signal is transmitted to the central processor to measure, determine the position and speed of the motor.

To control the motor to operate stably and at the desired speed, we used two drivers for the two motors The control circuit receives data from the computer to calculates and output the desired PWM pulse values.

As mentioned, the service robot in the restaurant have many obstacles such as people, tables and chairs,… so it is necessary to have the feature of detecting obstacles during movement to easily avoid collisions that damage the robot or collision hurt people, reduce the quality of restaurant service Obstacles are usually of two types: fixed obstacles and movable obstacles To detect obstacles, there are many methods that can be used such as lidar, camera, radar sensor, In this topic, the team used ultrasonic sensors as introduced above to detect obstacles The reason for choosing this sensor is because this is a sensor that is less susceptible to environmental interference, has moderate accuracy, is easy to program on a microcontroller, and so on In order to ensure coverage as well as avoid interference from each other, we arranged 3 sensors in front of the robot with an angle range of 15 degrees each and arranged 3 sensors to increase the range of the robot Large-area sensors help detect obstacles during the robot's approach When an obstacle is detected, the robot temporarily stops.

5.2.7 Design of the center console

In general, this robot consists of 3 controllers that take turns taking on certain roles in the system The sensor structure is handled by the Slave microcontroller Not only that, the robot system also has to build an interface, receive data from customers and transmit it to the restaurant's data center The interaction will be done through the screen, mic and speaker on the robot.

Figure 5.5 Structure of control system

DISTRIBUTED SERVICE ROBOT SYSTEM

Overview

In order to be able to apply more as well as create a working environment completely served by robots, the team researches distributed control algorithms The system consists of 3 robots with 3 different functions, supporting working and exchanging data with each other In this chapter, the control algorithms used in the system will be presented.

Proposed Distributed Service Robot System (3 robots)

We propose a distributed control system in restaurant service including a PC server and 3 mobile robots with the following tasks and functions:

Robot 1 (Kiri): welcome customers at the entrance At the same time, display information about available tables on the screen.

Robot 2 (Isarine): act as a waiter to help customers order food.

Robot 3 (Bella): act as a waiter with the role of bringing food to customers.

Operation procedure of Distributed Service Robot System

Figure 6.1 Distributed service robot system flowchart First, customers choose the available table position displayed on the screen of robot 1 (Kiri) After the customer is directed to the table Robot 2 (Isarine) receives the command from robot 1 (Kiri) At this time, the process of calculating the optimal path

35 will be created based on the A* algorithm with the current position, the position of table that the customer has selected as inputs.

Next, robot 2 (Isarine) displays the menu screen so that customers can proceed with the ordering process The ordering process will be the process of exchanging information by voice between the customer and robot 2 (Isarine) After the customer has finished ordering, the robot 2 (Isarine) synthesizes information about the dish that the customer has selected and send it to the server PC (Reception PC).

After the chef completes the dish, the staff at the reception counter chooses the table position to bring the food to the guest, robot 3 (Bella) receives this information about the table needed and apply the A* algorithm to can move to the desired position. The operation of the system will be repeated when there are new customers Moreover, in the process of moving between robot 1 and robot 2, a collision is inevitable To avoid this unwanted collision when moving two robots 1 and 2 continuously exchange position information for each other for the dodging process between the two robots.

Data transmission methods between robots

An extremely important short-distance communication technology is Bluetooth. Currently, Bluetooth appears in most devices such as computers, phones / smartphones, and it is expected to be the key to special IoT products, allowing device communication with smartphones - a "mighty force" today.

However, Bluetooth is not really designed for file transfer applications and will be more suitable for small data blocks It has a huge advantage in the current context, smartphone is an indispensable device for everyone According to Bluetooth SIG, more than 90% of smartphones are now embedded with Bluetooth, including iOS, Android and Windows operating systems, and it is expected to be "Smart Ready" by 2018.

Some technical information about Bluetooth:

Socket is a network application programming interface used to transmit and receive data over the internet Between two programs running on the network, there needs to be a two-way communication link, or two-way communication, to connect the two processes to talk to each other The endpoint of this link is called socket The socket includes the Internet protocol (IP) address, as well as the host or port that the computers are using to transmit the data All applications that take part in the transmission use the socket to send and receive information.

The purpose of a TCP/IP socket is to identify the computer or network server's unique IP address, along with its port number Port numbers are sometimes referred to as hosts, and they tell the receiving computer where to send the requested data. Together, the IP address and port number make up a socket.

The purpose of the TCP/IP socket is to identify the IP address for a computer or server.

A port address is the number of the port that will be used to send and receive data. Each computer device has several different ports that it uses to send and receive communications Since some of those ports may be used by certain devices such as printers, the computer needs to tell other devices where to send information In Internet communication, the port may be an Ethernet port assigned to a specific device or the port address of a server that is connected to several individual computers.

A computer socket, including a TCP/IP socket, is used to facilitate communication between two separate devices It provides instructions for where that communication should be sent The IP address and port address contained in the TCP/IP socket tells the receiving computer where to send the requested data It also helps establish some sort of uniform language between the machines.

When a socket is created, the sender and receiver both test the communication port to ensure that the data is going through The sender may send a request that tells the receiver what its identity is A response is then sent back to the sender confirming the sender's identity, while also communicating the identity of the receiver As long as the communications go through and the socket is confirmed to be working, further data requests continue.

Sockets are created each time a communication protocol is established between two or more devices Once the communication process has ended, the TCP/IP socket is closed If for some reason the port requested is not available, the socket will not be created An error message will usually prompt both the sender and receiver that the socket has failed.

Applications that are used in communications, including web browsers, also use TCP/IP sockets to send and receive data During the time that the socket is in use, all communications and applications are bound to the port address that is identified in the socket As applications send and receive data, they look for the socket's specific port address or host.

In this report, we use the TCP Socket data transfer method because of the following advantages:

• Is reliable: Packets dropped in the network are detected and retransmitted by the sender.

• Has in-order data delivery: Data is read by your application in the order it was written by the sender.

• Built a client and server that can handle multiple connections simultaneously.

• Economically, we don't need to pay for a connection module circuit like Bluetooth.

Localize robot in 3D space

The team's robot positioning idea will be briefly summarized as follows:

We train the YOLO model with the images of the robots as custom data Then, we use the trained YOLO model above to detect the robot pixel coordinates Then we convert the coordinates in the image to the coordinates of the robot's operating environment But the signal operation from the camera is not always stable, for example, in low light, there are times when the signal is lost while the robot is running. During that time, it will affect the efficiency of the YOLO algorithm, to overcome we have used more kinematic equations to be able to control the position of the robot, which is Dead-Reckoning By combining them, we can ensure that the robot's position is continuously updated in real time for control purposes.

In recent years, object detection is one of the hot topics of deep learning because of its high applicability, easy-to-prepare data, and extremely large application results. New object detection algorithms such as YOLO, SSD are quite fast and accurate, so Object Detection can perform tasks like real-time, even faster than a human without reducing accuracy In addition, an object detection algorithm can create very diverse applications such as: Counting the number of objects, paying money at counters, automatic timekeeping, fire detection in firefighting… and many other applications It can be said that any field can be applied object detection.

Besides, the source of image data is extremely diverse and available It is also an advantage to train model object detection Because of its high applicability, it's easy to prepare data and train, so we will come to learn an object detection state-of-art algorithm, which is YOLO.

YOLO in object detection means “You only look once” That is, we only need to look once to be able to detect the object.

In terms of accuracy, YOLO is not the best algorithm, but it is the fastest in the class of object detection models It can achieve almost real-time speed without sacrificing accuracy compared to the top models.

YOLO is an object detection algorithm, so the goal of the model is not only to predict labels for objects like classification problems, but also to determine the location of objects Therefore, YOLO can detect many different labelled objects in an image instead of only classifying a single label for an image.

YOLO network architecture includes: base networks are convolution networks that perform feature extraction The following are the Extra Layers applied to object detection on the feature map.

YOLO's Base network using only convolutional layers and fully connected layers. YOLO architectures are also quite diverse and can be customized into versions for many different input shapes.

Figure 6.2 YOLO network architecture diagram With the above model, the Darknet Architecture component called the base network has the effect of feature extraction The output of the base network is a 7x7x1024 feature map that will be used as input for the Extra layers that predict the label and bounding box coordinates of the object.

The size of the feature map will depend on the input For input 416x416, the feature map has dimensions of 13x13, 26x26 and 52x52 And when input is 608x608 will generate 19x19, 38x38, 72x72 feature map.

The output of the YOLO model is a vector that will include the main components = [ 0 ,〈 , , , ℎ 〉,〈 1 , 2 , … , 〉]

• 0 is t he predicted probability of the object appearing in the bounding box.

• 〈 , , , ℎ 〉 helps define the bounding box In which , is the coordinates of the center and , ℎ are the width and length dimensions of the bounding box.

• 〈 1 , 2 , … , 〉 is the predictive probability distribution vector of classes Understanding the output is quite important for us to configure the correct parameters when training the model through open sources like darknet Thus the output will be determined by the number of classes according to the formula (n_class + 5) If training 2 classes, the output will be 7 In case of applying 3 anchors/cell, the number of output parameters will be (n_class + 5) x 3 = 7 x 3 = 21.

Figure 6.3 YOLO network architecture diagram The original image is a 13x13 feature map On each cell of the feature map, we select 3 anchor boxes with different sizes of Box 1, Box 2, Box 3 so that the center of the anchor boxes coincides with the cell Then the output of YOLO is a concatenate vector of 3 bounding boxes.

YOLO v3 forecasts on multiple feature maps Small initial feature maps help predict large objects The following feature maps have a larger size while the anchor box is kept fixed in size, so it helps to predict small objects.

Figure 6.4 Feature map With an input shape of 416 x 416, the output is 3 feature maps with dimensions of 13x13, 26x26 and 52x52 respectively.

On each cell of the feature maps we will apply 3 anchor boxes to predict the feature Thus the number of different anchor boxes in a YOLO model will be 9 (3 feature map x 3 anchor boxes).

Also on a square feature map S x S, the YOLO v3 model generates a number of anchor boxes: S x S x 3 Thus, the number of anchor boxes on an image will be: (13 x

This is a very large number and is the reason why the training of the YOLO model is extremely slow because we need to predict the label and bounding box simultaneously on 10647 bounding boxes.

To find the bounding box for the object, YOLO will need the anchor boxes as the basis of the estimation These anchor boxes will be predefined and will surround the object with relative precision Later, the regression bounding box algorithm will refine the anchor box to create a predicted bounding box for the object In a YOLO model:

• Each object in the training image is distributed about an anchor box In case there are 2 or more anchor boxes surrounding the object, we will determine the anchor box that has the highest IoU with ground bounding box.

Figure 6.5 Defines the anchor box for an object

• From Cell I we can identify 3 green-bordered anchor boxes as shown in the figure All three of these anchor boxes intersect with the bounding box of the object. However, only the anchor box with the thickest blue border is selected as the anchor box for the object because it has the highest IoU compared to the ground truth bounding box.

• Each object in the training image is distributed to a cell on the feature map that contains the mid point of the object.

Thus, when determining an output object, it will be necessary to identify 2 components associated with it (cell, anchor box) Not just the cell or just the anchor box.

YOLO's loss function is divided into two parts: (localization) measures the error of the bounding box and (confidence loss) measures the error of the probability distribution of classes.

• : The indicator function has a value of 0.1 to determine if cell contains an object or not Equals 1 if it contains the object and 0 if it doesn't.

• : Indicates whether the jth bounding box of cell is the bounding box of the predicted object.

• : Cell confidence score , P(contain object) * IoU.

• ( ): Conditional probability, whether or not cell contains an object of class.

• ℒ loc is the loss function of the predicted versus actual bounding box.

• ℒ is the loss function of the probability distribution Where the first sum is the loss of predicting whether there are objects in the cell or not? The second sum is the loss of the probability distribution if there is an object in the cell.

Path Planning

It is possible to classify ways to solve the problem of finding a way for a robot according to two fields fit as follows:

Case 1: Robot does not know the information of the map in advance.

Some popular route detection algorithms are as follows [5]:

Wall-following algorithm: This algorithm is applicable provided that seamless, unbroken maze With the labyrinth that's satisfied, the robot only needs to continuously follow the left or right wall then reach the final destination.

Figure 6.12 Algorithm according to the right-following rule [5]

Pledge Algorithm: The implementation method of this algorithm is a combination of suitable for sticking to the wall but taking into account the rotation angle Hence, 51 robot can reach the destination (escape from the maze) in complex maps more, like the robot is placed in the starting position inside but separate out of the rest of the map.

Figure 6.13 Pledge Algorithm get rid of complicated obstacles

In addition, there are many other algorithms that can be applied to detect the path such as stochastic algorithm, Tremaux algorithm, dead-end filling algorithm However, these methods generally have the following disadvantages:

• Pathfinding is fully automatic without manual case.

• The path found is the most accurate and shortest.

Case 2: The robot already knows the map information in advance After obtaining information about the map, the automatic pathfinding algorithms can be applied to find the way Automatic pathfinding algorithms are algorithms that help find the way automatically based on known data This automatic route finding is based on divide the map into nodes, then apply algorithms on these nodes to find the path Therefore, in order to apply the automatic pathfinding algorithm, the data about the map must be known, from which information about the nodes can be obtained, for example such as which node belongs to the goable path and which node does not belong to the goable path Many automatic pathfinding algorithms have been developed such as Grass Fire, Greedy Best First Search, Dijkstra or A* [5]:

- Grass Fire Algorithm: In the Grass Fire algorithm, nodes are neighbors to the under consideration will all be selected as the next node under consideration, if the

52 neighboring node belongs to about obstacles, it is ignored The algorithm iterates until the destination node is one of the nodes under consideration The advantage of this algorithm is that it is simple the shortest path can be found, but the speed is slow because of the number of nodes considered large and increases rapidly as the number of nodes increases.

- Dijkstra's algorithm: the idea of the algorithm is selective search i.e is that at the neighboring nodes of the node under consideration, the algorithm prioritizes choosing a node as the next node under consideration based on the cost value from the neighbor node it with the currently considered node The node with the lowest cost is selected. Dijkstra algorithm is more complex than Grass Fire but it find the path faster because the number of nodes to consider is less.

- Greedy Best First Search algorithm: This algorithm is also an algorithm selective search algorithm But instead of prioritizing cost value choices between the neighboring node and the node under consideration, the algorithm prioritizes choosing the value of estimated cost between the neighbor node of the node in question and the destination node (heuristics) The advantage of this algorithm is that it is faster than Dijkstra but the downside is that the path found may not be the shortest.

- Algorithm A*: Algorithm A* can be considered as a combination of Dijkstra algorithm and Greedy Best First Search algorithm when it uses the same both the cost value from the neighbor node relative to the current node and the cost value estimate from the neighbor node to the destination node as the criterion for selecting the node under consideration next As a result, algorithm A* has the ability to find the shortest path like Dijkstra's algorithm In most cases, A* has the same the path finding results but faster than Dijkstra's algorithm, Figure 7.9.

Figure 6.14 Route finding results by A* algorithm (left) and Dijkstra's algorithm

In this report, information about the map can be considered as completely known in advance The robot pathfinding is automatically done by the A* algorithm with the advantages summarized as follows:

• Pathfinding is fully automatic without manual case.

• The robot does not need to track the route.

• The path found is the most accurate and shortest.

The A* Algorithm is a popular graph traversal path planning algorithm A* operates similarly to Dijkstra’s algorithm except that it guides its search towards the most promising states, potentially saving a significant amount of computation time [7]. A* is the most widely used for approaching a near optimal solution [8] with the available data-set/node.

It is widely used in static environments; there are instances where this algorithm is used in dynamic environments [9] The base function can be tailored to a specific application or environment based on our needs A* is similar to Dijkstra in that it works based on the lowest cost path tree from the initial point to the final target point The base algorithm uses the least expensive path and expands it using the function shown below f(n) = g(n) + h(n) (6.12) where g(n) is the actual cost from node n to the initial node, and h(n) is the cost of the optimal path from the target node to n.

The A* algorithm is widely used in the gaming industry [9], and with the development of artificial intelligence, the A* algorithm has since been improved and tailored for applications, including robot path planning, urban intelligent transportation, graph theory, and automatic control [10–14].

It is simpler and less computationally-heavy than many other path planning algorithms, with its efficiency lending itself to operation on constrained and embedded systems [12,17] The A* algorithm is a heuristic algorithm that uses heuristic information to find the optimal path The A* algorithm needs to search for nodes in the map and set appropriate heuristic functions for guidance, such as the Euclidean distance, Manhattan distance, or Diagonal distance [15,16] An algorithm is governed by two factors for efficiency resources used for performance of the task and response time or computation time taken for performance of the task.

There is a trade off between speed and accuracy when the A* algorithm is used.

We can decrease the time complexity of the algorithm in exchange for greater memory, or consume less memory in exchange for slower executions In both cases, we find the shortest path One simple application for the A* algorithm is to find the shortest path to an empty space within a crowded parking lot [17].

Importantly, the A* algorithm is computationally efficient [18,19] This makes it suitable for applications deployed in static environments The computational speed and efficiency of A* and its variants depends on the accuracy of the heuristic function.

6.6.2 Most Common Types of Heuristic Functions Used In Path Planning Algorithms

Diagonal h(n) = D * (dx + dy) + (D2 - 2 * D) * min(dx, dy) distance

Figure 6.17 Output of A* algorithm with Manhattan distance [22]

Figure 6.18 Output of A* algorithm with Diagonal distance [22]

Figure 6.19 Output of A* algorithm with Euclidean distance [22]

Since Euclidean distance is shorter than Manhattan or diagonal distance, we still get shortest paths, but A* take longer to run This is a worth trade off.

6.6.3 Application of A* algorithm in path planning

After selecting the destination node (destination) and the starting node (starting position of the robot), the A* algorithm is basically implemented as follows:

- Select the starting node as the node under consideration and add it to the list of considered nodes.

- Update the cost value compared to the starting node g(n) of the nodes neighboring to the node under consideration.

Human interactive system

Speech is a basic human ability, we use it to communicate with each other every day Voice is formed and developed throughout human existence, it helps us to communicate, exchange information, chat, entertain or learn To create a feeling of closeness and naturalness with customers, robots need to have a voice Voice helps the robot communicate and exchange basic information with guests Moreover, voice also helps the robot to communicate and convey information more effectively The database is an important part of helping the robot communicate in the most natural way, it is like the process of people being born, growing up and learning so that each person's own database gradually grows The larger the database, the smarter the robot becomes Speech recognition is receiving requests or exchanging information from which the robot hears and understands what the interactors need to meet their needs.

In this chapter, we present about the robot-human interaction by applying voice recognition to perform intelligent functions of the robot in the topic.

6.7.2 Structure of the interactive system

The following human-robot interaction system is built with the aim of improving and enhancing the user experience during contact with robots The goals include: First, the system must be able to recognize speech From there, the system is able to recognize what the user's intentions are and categorize them to perform according to a specific task This allows the robot to interact and communicate with customers like a real waiter, thereby improving and enhancing the user experience.

Figure 6.20 Human-robot interaction system flowchart The structure of the human-robot interaction system is shown in Figure 6.20 Input is the user's voice However, the structure of the audio data is very complex to process.

To solve this problem, the data is converted from audio to text allowing for easier processing Next, these texts are classified to determine the user's intention so that the robot can give the appropriate response This is an important process to increase the interaction between the user and the robot Text is divided into two types: conversational text and order text Where conversational text is defined as normal conversational text, it does not require much knowledge to generate a response. Finally, the order document is the text that belongs to the process of classifying the intention of the dish that the customer wants to order.

When the input data is classified as conversational text The system starts responding based on the operating principle of a chatbot The robot system is capable of

60 answering questions about the restaurant, the food as well as information about itself: name, ability,

When the input data is classified as order text The system begins with three processing processes: confirming the customer's identity, retrieving information, classifying the ordered dish and starting to place the order Similar to humans communicating, robots also need to determine the identity of the interlocutor This allows the system to focus on processing information relevant to the interlocutor, while eliminating unnecessary information for the communication process Once the identity has been confirmed, the data is transmitted to the client's retrieval process The process allows the system to detect important information related to customers such as: name, frequently ordered dishes, etc From there, the system starts to suggest dishes that customers have ordered before or proceed with other dishes according to the needs of the customer.

The resulting block is the final process of the human-robot interaction system. Results from previous tasks are aggregated by this block Written responses are converted back to voice (audio) to continue the communication process Finally, the data about the information and dishes selected by the customer will be aggregated by the system and sent to the restaurant's management center to process the dish.

Most modern speech recognition systems are based on the Hidden Markov Model (HMM) This approach works on the assumption that a speech signal, when viewed over a sufficiently short period of time (e.g., ten milliseconds), can be reasonably approximated as a static process — means a process in which statistical properties do not change over time.

In a typical HMM, the speech signal is divided into segments of 10 milliseconds long The power spectrum of each fragment, which is essentially a graph of the signal's power as a function of frequency, is mapped to a vector of real numbers called the cepstral coefficient The dimension of this vector is usually small - sometimes as low as 10, although more precise systems can have dimensions 32 or more The final output of the HMM is a sequence of these vectors.

To decode speech into text, groups of vectors are matched against one or more phonemes — a basic unit of speech This calculation requires training, as the sound of a phoneme is different for each speaker and even varies from one pronunciation to another by the same speaker Then, a special algorithm is applied to determine the word (or words) that is most likely to produce a given phoneme sequence.

One can imagine that this whole process could be computationally expensive In many modern speech recognition systems, neural networks are used to simplify speech signals using feature conversion and dimensionality reduction techniques before HMM recognition Voice Activity Detectors (VADs) are also used to reduce the audio signal to only those parts that are likely to contain speech This prevents the recognizer from wasting time analysing unnecessary parts of the signal.

6.7.4 Speech to Text with Google API

Currently, there are many libraries that support speech to text conversion Such as wit and apiai - offer built-in features, like natural language processing to determine speaker intent, that go beyond basic speech recognition Or like google-cloud-speech, which only focuses on speech to text conversion.

However, we use the SpeechRecognition library because of its outstanding ease of use.

Speech recognition requires audio input, and SpeechRecognition makes retrieving this input really easy Instead of having to build scripts to access microphones and process audio files from scratch, SpeechRecognition gets you up and running in minutes.

The SpeechRecognition library acts as a wrapper for several popular speech APIs and is therefore extremely flexible One of these - Google's Web Speech API - supports a default API key hard-coded into the SpeechRecognition library The flexibility and ease of use of the SpeechRecognition package make it a great choice for any project However, it also has the limitation that any features of each API it ends up in are not guaranteed.

A word embedding is a learned representation for text where words that have the same meaning have a similar representation It is this approach to representing words and documents that may be considered one of the key breakthroughs of deep learning on challenging natural language processing problems.

Word embeddings are in fact a class of techniques where individual words are represented as real-valued vectors in a predefined vector space Each word is mapped to one vector and the vector values are learned in a way that resembles a neural network, and hence the technique is often lumped into the field of deep learning.

Key to the approach is the idea of using a dense distributed representation for each word.

EXPERIMENT AND ASSESS THE RESULTS

Overview

After studying and designing the structure, the frame for the robot In this chapter,some results are presented on the mechanical model, motion control of the robot as well as the accuracy of the moving position and the part, user interface and interaction.

Basic parameters of the robot

Figure 7.1 Complete restaurant service robot model After designing and manufacturing, the results of hardware and robot specifications are presented in Table 7.1 The robot frame is solid, stable, has good transportability and bearing capacity, the engine is stable, the moving platform has good bearing capacity, meeting the initial requirements.

6 Motion kinematics of the 2 differential wheels, 2 self- pedestal selecting wheels

8 Robot operating time 4 hours (45Ah)

Mechanical system

Because the operating environment of the robot is mainly crowded places, with food and water, it is necessary to ensure the safety of users as well as to ensure that the electrical and induction systems are not affected by external factors Because the base,like the brain of the robot, should be carefully wrapped, each food compartment is emptied inside to stabilize food and water to avoid breakage As for the display screen,the design team placed it in an elevated position in order to minimize the influence of children's curiosity

Distributed service robot system

Camera calibration data and results

Figure 7.2 Camera calibration results After we have done the camera calibration, we have the result as shown below

Figure 7.3 Results before and after camera calibration

Conclusions: Results after calibration are not completely correct ratio between pixels The value is at a relative level It is necessary to re-align the pixel ratio in the positions Calibration matrix applies only to stationary and a fixed angle camera When the camera angle is shifted, recalibration must be performed.

Once we have calibrated the camera, we align the pixel ratio at the positions and conduct experiments

Determine the coordinate angle under the camera viewing area, from which the pixel value is used to convert to the actual distance according to the coordinate angle At the same time, we also check the actual values to compare with the converted value.

After conducting, we get the actual distance value measured from the camera and the real distance value measured shown in the following chart:

Figure 7.4 Diagram showing the camera position and the actual location

- From the origin -> 300cm: The position deviation from reality is negligible, the error is about 0.1 -> 1 (cm)

- From 300 -> 500 cm : This deviation increases from 1-3 cm

⇒ The position from the coordinate angle back after pixel scaling has an error, so the overall position is deviation from 0.1-> 3 cm.

The training process is performed for about 10000 epochs with a total time about 2 hours mAP during training also increases with each epoch up to 80% and log loss converge from epoch 9000.

Figure 7.5 YOLO Loss and mAP

Figure 7.6 Diagram showing the position of the robot moving from Home to table 1 Table 7.2 Error value through experiments when the robot moves from Home to table

1 n Xset Yset XMeasured YMeasured ∆X(mm) ∆Y(mm) ∆Error(mm)

Figure 7.7 The graph shows the error when the robot moves from home to table position 1

Figure 7.8 Diagram showing the position of the robot move from Table 1 to Table 4 Table 7.3 Error value through experiments when the robot moves from table 1 to table

4 n Xset Yset XMeasured YMeasured ∆X(mm) ∆Y(mm) ∆Error(mm)

Figure 7.9 The graph shows the error when the robot moves from Table 1 to Table 4

Conclusions on the robot's movement:

- The error is less than 0.05(m) for the desired position when reaching the predetermined points.

• Accumulated pixel ratio errors while system is running.

• The signal from the camera is slow due to the network signal, this affects the positioning process.

7.4.5 Communication and data transmission between robots

Figure 7.11 Data transmission of robots

Conclusions: the robot was able to create the most optimal path using the A* algorithm Since the system can communicate by transferring data between robots, helping robots to have status and position information of other robots, thereby preventing collisions between robots during operation In the figure 7.10, it is easy to see that the most optimal path for robot 2 is to go straight from the point (30,40) to (30,10) But then there will be a collision with robot 1, so robot 2 creates another optimal path to move.

Figure 7.12 LSTM model accuracy after training 500 epochs The process of training the LSTM model does not take too much time because the model structure is quite simple and the actual amount of data is not much We can see that the model converges with 100% accuracy after only 330 epochs.

Figure 7.13 Naive Bayes predict on example 1

Figure 7.14 Naive Bayes predict on example 2

Figure 7.15 LSTM model predict on example 1

Figure 7.16 LSTM model predict on example 2

As we can see in Figures 7.13 - 7.16 the Naive Bayes classifier predicted 2 different examples both of which are class: “plus_chicken_1” Because basically Naive Bayes classifier predicts based on the number of occurrences of words in a sentence. Considering the two examples, we can see that the number of times the words appear in the sentence is the same Obviously, we can see that LSTM has outperformed Naive Bayes classifier completely because LSTM predicts semantically, not simply counting the number of occurrences of each word like Naive Bayes.

CONCLUSIONS AND RECOMMENDATIONS

Conclusions

The project presented an overview of Distributed Service Robot System along with related research works, building a distributed control robot system, solving the robot positioning problem, creating the optimal trajectory, the ability identify customer intent After the experiment, the group obtained the following results:

• The system operates relatively stably, the data transmission rate between robots is relatively high Transmitted data will always be full due to TCP Socket's loss detection and retransmission.

• The robot system always has information of other robots: what the customer ordered, the robot's trajectory, etc to help with the robot movement and system operation.

• Ability to recognize customer intent with high accuracy with trained data set But still not able to recognize the untrained speech.

• Ability to detect and calculate robot position with high accuracy Although there are still many shortcomings such as the control algorithm is still basic, the processing speed of the program is not high, but in general, the program has achieved the requirements of the proposed project.

Recommendations

Based on the work done, the project implementation team found that the system can further develop the following contents:

- Improve the object detection by increase more data and try to use different model’s architecture to increase the efficiency of the positioning process Research using other positioning algorithms such as: ROS, to compare and choose a more effective method.

- Research solutions that the system can be used in many different spaces.

- Design facial recognition check-in software, create a database to store each customer's information such as name, favorite dishes, from there to suggest suitable dishes.

[5] Trịnh Tuấn Dương (2019), “Tìm đường cho Mobile robots dựa trên việc tìm đường biên áp dụng thuật toán A*”, Cơ khí Việt Nam, số 3.

[25] Bong-Su Cho, Woo-sung Moon, Woo-Jin Seo and Kwang-Ryul Baek , “A dead reckoning localization system for mobile robots using inertial sensors and wheel revolution encoding”, Bài báo khoa học đại học Hàn Quốc.

[26] Le Duc Hanh & Nguyen Duy Anh, “HOẠCH ĐỊNH VÀ BÁM ĐƯỜNG ĐI CHO MOBILE ROBOT ỨNG DỤNG ĐIỀU KHIỂN HỒI TIẾP ẢNH” Báo khoa học đại học Bách Khoa TP.HCM

[27] Nguyễn Trường Thịnh, “Giáo trình kỹ thuật robot”, Nhà xuất bản ĐHQG.

[28] Trịnh Chất, Lê Văn Uyển Tính toán thiết kế hệ dẫn động cơ khí tập 1, NXB Giáo dục

[1] Chen, Cheng, et al "The adoption of self-driving delivery robots in last mile logistics." Transportation research part E: logistics and transportation review 146 (2021): 102214.

[2] Seo, Kyung Hwa, and Jee Hye Lee "The emergence of service robots at restaurants: Integrating trust, perceived risk, and satisfaction." Sustainability 13.8 (2021): 4431.

[3] Pinillos, Roberto, et al "Long-term assessment of a service robot in a hotel environment." Robotics and Autonomous Systems 79 (2016): 40-57.

[4] “How to measure distance using Ultrasonic sensor”, https://mbedgeek.blogspo t.com/2018/12/ultrasonic-sensor-arduino-HC-SR04.html [Accessed 9/12/2022].

[6] Ferguson, D.; Likhachev, M.; Stentz, A A Guide to Heuristic-based Path Planning.

In Proceedings of the International Workshop on Planning under Uncertainty for Autonomous Systems, International Conference on Automated Planning and Scheduling (ICAPS), Monterey, CA, USA, 5–10 June 2005; pp 9–18.

[7] Gass, S.I.; Harris, C.M Near-optimal solution In Encyclopedia of Operations Research and Management Science; Gass, S.I., Harris, C.M., Eds.; Springer: New York,

[8] Hernández, C.; Baier, J.; Achá, R Making A* run faster than D*-lite for path- planning in partially known terrain In Proceedings of the International Conference on Automated Planning and Scheduling, ICAPS, Portsmouth, NH, USA, 21–26 June 2014; Volume 2014, pp 504–508.

[9] Cui, X.; Shi, H A*-based Pathfinding in Modern Computer Games Int J Comput. Sci Netw Secur 2010, 11, 125–130.

[10] Gunawan, S.A.; Pratama, G.N.P.; Cahyadi, A.I.; Winduratna, B.; Yuwono, Y.C.H.; Wahyunggoro, O Smoothed A-star Algorithm for Nonholonomic Mobile Robot Path Planning In Proceedings of the 2019 International Conference on Information and Communications Technology (ICOIACT), 2019; pp 654–658 Available online: https://icoiact.org/ (accessed on 9 October 2020).

[11] Zhou, Y.; Cheng, X.; Lou, X.; Fang, Z.; Ren, J Intelligent Travel Planning System based on A-star Algorithm In Proceedings of the 2020 IEEE 4th Information

Technology, Networking, Electronic and Automation Control Conference (ITNEC), Chongqing, China, 12–14 June 2020; Volume 1, pp 426–430.

[12] Belhaous, S.; Baroud, S.; Chokri, S.; Hidila, Z.; Naji, A.; Mestari, M Parallel implementation of A* search algorithm for road network In Proceedings of the 2019 Third International Conference on Intelligent Computing in Data Sciences (ICDS), Marrakech, Morocco, 28–30 October 2019; pp 1–7.

[13] Dong, G.; Yang, F.; Tsui, K.L.; Zou, C Active Balancing of Lithium-Ion Batteries Using Graph Theory and A-Star Search Algorithm IEEE Trans Ind Inform.

[14] Liu, H.; Shan, T.; Wang, W Automatic Routing Study of Spacecraft Cable based on A-star Algorithm In Proceedings of the 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China, 12–14 June 2020; pp 716–719.

[15] Kider, J.T.; Henderson, M.; Likhachev, M.; Safonova, A High-dimensional planning on the GPU In Proceedings of the 2010 IEEE International Conference on Robotics and Automation, Anchorage, AK, USA, 3–7 May 2010; pp 2515–2522.

[16] Cheng, L.; Liu, C.; Yan, B Improved hierarchical A-star algorithm for optimal parking path planning of the large parking lot In Proceedings of the 2014 IEEE International Conference on Information and Automation (ICIA), Hailar, Hulun Buir, China, 28–30 July 2014; pp 695–698.

[17] Yao, J.; Lin, C.; Xie, X.; Wang, A.; Hung, C.C Path Planning for Virtual Human Motion Using Improved A* Star Algorithm In Proceedings of the 2010 Seventh International Conference on Information Technology: New Generations, Las Vegas, NV, USA, 12–14 April 2010; pp 1154–1158.

[18] Chen, T.; Zhang, G.; Hu, X.; Xiao, J Unmanned aerial vehicle route planning method based on a star algorithm In Proceedings of the 2018 13th IEEE Conference on Industrial Electronics and Applications (ICIEA), Wuhan, China, 31 May–2 June 2018; pp 1510–1514.

[19] Baker, Z.K.; Gokhale, M On the Acceleration of Shortest Path Calculations in Transportation Networks In Proceedings of the 15th Annual IEEE Symposium on

Field-Programmable Custom Computing Machines (FCCM 2007), Piscataway, NJ, USA, 23–25 April 2007; pp 23–34.

[20] Seo, W.; Ok, S.; Ahn, J.; Kang, S.; Moon, B An Efficient Hardware Architecture of the A-star Algorithm for the Shortest Path Search Engine In Proceedings of the 2009 Fifth International Joint Conference on INC, IMS and IDC, Seoul, Korea, 25–27 August 2009; pp 1499–1502.

[21] “A* Search Algorithm”, https://www.geeksforgeeks.org/a-search-algorithm/ [Accessed 9/12/2022].

[22] “A*’s Use of the Heuristic”, http://theory.stanford.edu/~amitp/GameProgrammi ng/Heuristics.html [Accessed 9/12/2022].

[23] “Understanding LSTM Networks”, https://colah.github.io/posts/2015-08-

[24] “LSTM for Text Classification in Python”, https://www.analyticsvidhya.com /blog/2021/06/lstm-for-text-classification/ [Accessed 9/12/2022].