Luận văn thạc sĩ VNU UET study on supervision and control of robot over computer network, nghiên cứu vấn đề giám sát và điều khiển robot qua mạng máy tính

Introduction

Introduction to networked robot systems

Appeared in 1994, the first networked robot system (NRS) had received over 2.5 million accesses for seven months The system namely Mercury project permitted Internet users to operate a manipulator to excavate artifacts buried in a nuclear con- taminated region to look for evidence of ancient water flows [27] In the next seven years, over forty such systems had been developed allowing users to remotely visit museums, tend gardens, navigate undersea, float in blimps, and handle protein crys- tals [25, 26, 48, 63, 76] This new trend attracted the IEEE Society of Robotics and Automation and the name Networked Robots was created in 2004 with the following definition [62]:

“A networked robot is a robotic device connected to a communications network such as the Internet or LAN The network could be wired or wireless, and based on any of a variety of protocols such as TCP, UDP, or 802.11.”

To reflect a broader set of problems and applications, the definition was extended to include two subclasses of networked robot systems (NRSs): the autonomous systems and the teleoperated (manual) systems

• Autonomous NRSs: In autonomous systems, the controller generates control signals to operate the robot The robot (including actuators and sensors) com- municates with the controller via the network The network extends the effective operating range of the robot, allowing multiple robots to communicate with each other over long distances to coordinate their activity

• Teleoperated NRSs: In teleoperated systems, human operators send commands to the robot The robot executes the commands and returns feedback to the operators via the network Input devices and user graphic interfaces are used to support the operator They act as a passive controller to translate operator’s commands to control signals

NRSs call for the integration of several fields: robotics, perception (sensor systems), ubiquitous computing, artificial intelligence, and network communications

The topics of NRSs transcend ‘‘conventional’’ robotic problems such as localization and motion control to the type of distributed systems over heterogeneous communications networks The challenging issues include the guarantee of system reliability and performance under the influence of time-varying transmission delay, message loss, out-of-order delivery, and non-guaranteed transmission bandwidth Many new applications are now being developed ranging from automation to exploration.

Applications of networked robot systems

Applications of NRSs can be classified into five groups including industrial robots, educational robots, medical robots, service robots, and other various robots

Networked robots are applicable in industry In [36], a teleoperated robot system for coal mining was developed The robot is equipped with two cameras, two laser scanners, and some other sensors to extract the environmental parameters [figure 1.1(a)] These parameters are then transmitted via the network to the human operator The human operator, in a safe location, controls the robot to complete tasks such as shoveling and breaking [figure 1.1(b)] Advantages of the system include the iso- lation of human labor from the harsh environments and the endurance of the robot in hard work Difficulties relate to the problem of sensor fusion and localization

Figure 1.1: The coal mining networked robot system[36]: (a) The robot with actuators and sensors; (b) The operator with input devices and user graphic interfaces

Networked robots enable the development of telelaboratory In [82], a telelaboratory was developed for the training of robotics Via the Internet and world wide web, students are able to connect to the laboratory to operate a real manipulator (figure 1.2) The students also have access to other facilities such as the teach pendant and control computer to fulfill the practical lesson Advantages of telelaboratory include the sharing of expensive systems among educational units, the ease in evaluation process, and the flexibility in practice

Figure 1.2: A telerobotic laboratory platform[82]: (a) The telelaboratory with robotic devices; (b) The web-based interface to interact with the telelaboratory

Medical networked robots bring users chance to be diagnosed and treated remotely by experienced doctors In [53], Masuda developed a telerobot for echography Re- mote operators use two joysticks to control: one moves the robot; the other adjusts the ultrasonic probe’s angle At each stop position, the robot captures an ultrasonic image, compresses it, and sends it over the Internet to the examiner In [14], a telesurgical system was developed Two slave manipulators are placed at patient’s site

They are controlled by two joysticks located at operator’s site The interaction between the operator and patient is conducted with the support of tactile and force feedback sensors (figure 1.3)

Figure 1.3: The telesurgical system [14]: (a) Setup of slave manipulators around the operating table; (b) An experiment with the surgeon and patient

Networked robots can supply services at home, office, and public areas In [62], Sanfeliu et al developed new ways of cooperation between networked robots and human beings Robots have tasks of guidance, assistance, transportation, and surveillance in urban areas Their architecture integrates cooperating urban robots, intelligent sensors (video cameras, acoustic sensors, etc.), intelligent devices (PDA, mobile telephones, etc.) and communications (figure 1.4)

Figure 1.4: Service networked robots in urban areas [62]: (a) A service robot; (b)

The robot interacts with human to supply information; (c) Robots cooperate in a guidance task

Networked robots can be applied in other various applications In [50], Luo introduced a robot for remote surveillance The robot has five sensory subsystems for the fire detection, intruder detection, motor diagnosis, obstacle detection, and environment detection In [4], the robot named GARBI was developed for the tasks of observation and manipulation in shallow water environments It has a 3D vision system to support the remote control In [65], a prototype rover was designed as a test- and-validation vehicle for a Mars technology program Other applications of networked robots include pet robot, tour-guide robot, intelligent wheelchair, etc [49,

In Vietnam, several works on NRSs have been proposed recently In [1], Cong Thanh developed a telemanipulator system The system consists of two manipulators (master and slave) communicated through a local area network During the operation, human operators act on the master manipulator At remote site, the slave manipulator mimics the master based on signals received via the network In [78], a telehealthcare system was developed In the system, cameras are arranged at patient’s house to record and transmit the vision data to a remote processing central If abnormal behaviors are detected, the system will send messages to doctors In [37], a NRS for searching high radioactivity regions in a disaster area was introduced

The system uses a partial swarm optimization algorithm to enable the cooperation between the robots to fulfill the task

Though the proposed works are mainly preliminary results, they prove the applicability of NRSs in Vietnam In fact, NRSs would be also applicable to emerging fields in Vietnam such as traffic control, ubiquitous healthcare, and underwater exploration.

Related works

Adapting to this emerge field of robotics, there have been a number of projects dealing with problems involved in networked robots This section reviews studies on the localization, stabilization control, and navigation which are topics covered in this thesis

1.3.1 Study of NRSs on localization

Localization is fundamental to the functioning operation of robotic systems In order to complete a given task, the robot needs to know its own state in the environment In NRSs, there are two approaches in localization including advance interface techniques and optimal filters

A Localization using advance interface techniques

Advance interface techniques locate the robot by reconstructing the operating environment In [86], a virtual environment which is proportional to the real dimension of the working environment is constructed at client site Before commands are sent to the server program, they are processed in the virtual environment to predict the upcoming position of the robot Based on it, the system can tolerate a certain amount of time delays as well as allow users to experience the interaction with the robot as in the real environment In [74], the model-based virtual reality technique was used to construct the polygonal model of the complete 3D world The operator controls the robot by issuing commands to the model An extension of this technique called ecological interface paradigm was introduced in [55] It uses multiple sets of information to create a 3-D virtual environment that is augmented with the real video information

On another note, Hou and Su put four cameras in the robot field as external visual sensors to formulate four adjacent grids of vision without the dead zone [34] A recognition algorithm is implemented to recognize and give out the relative location of the robot, target, landmark and obstacle symbols In [31], the robot pose is estimated at the local site using odometry, sonar, and compass sensors This information is then transmitted to the remote site as the pose of the robot without consid- ering the its change during the communications In [41], a map-based localization method was proposed The absolute position of the robot is determined by comparing a reference map of the local site with the one built by the GPS map building technique at the remote site

The limitation of advance interface techniques is that they avoid dealing with the key issues of communications networks, the network delay, message loss, and out-of-order delivery The data transmission between the remote controller and the actuator is often treated as a given condition and rarely touched From the viewpoint of estimation theory, significant delay and loss are equivalent to the inaccuracy in state estimation and control that can easily downgrade the system performance

The optimal filtering method incorporates parameters of the network into the state- space representation of the NRS and then uses the estimation theory to locate the robot In [71], Sinopoli et al introduced an augmented Kalman filter for state estimation with intermittent observations The arrival of observations is modeled as a binary random variable λ t Its variance is defined as the variance of output noise if λ t is 1, and as σ 2 I otherwise, where σ is a real constant and I is the identity matrix The

Kalman filter is then reformulated using a “dummy” observation (if a real observation does not arrive) and takes the limit as σ → ∞ This approach was expanded in

[64] to simultaneously deal with random delay and packet drop In this work, the arrival of observations is also defined as a binary random variable, but an infinite buffer is introduced to store and rearrange delayed measurements The estimation is then computed from the initial to the latest available measurements at each step

Due to the iteration, this filter is very computationally demanding A finite buffer can be used to lighten the problem, but it requires the time delay to be bounded

Recently, Moayedi et al [54] have introduced an adaptive Kalman filter for networked control systems with mixed uncertainties of random delays, packet dropouts, and missing measurements This work uses a set of matrices to represent the mixed uncertainties and add them into the system model The filter gain is then derived by solving a set of recursive discrete-time Riccati equations On a similar note, Ma and Sun [52] proposed an optimal filter to deal with random sensor delays, packet dropouts, and uncertain observations Three Bernoulli distributed random variables with known distributions are introduced to generate a unified model of the mixed uncertainties Based on it, a Riccati equation and a Lyapunov equation are solved to retrieve the estimator Both filters have advantages in dealing with network induced problems They are however not optimal for the out-of-order delivery

In addition, those filters are designed for linear systems Further modifications are required to extend them to nonlinear systems such as NRSs

1.3.2 Study of NRSs on stabilization control

Similar to classic robot systems, stability is an important criteria of NRSs In the literature, several works have been proposed for this problem Wargui et al developed a stable controller for NRSs with nonholonomic constraints [88] In this work, control laws that stabilize the system in delay-free scenario are first derived They are then extended to work with the time delay by adding a multistep predictive estimator between the system output and the controller In [15], the maximum time delay allowed at the control input without losing the system stability is estimated

Based on it, a single layered neural network is designed to stably control the networked robot In [47], Luck used a time buffer which was longer than the worst delay to make the system to be time-invariant and then applied the classic control theory In [89], Xi and Tarn proposed an event-based (non-time) control scheme to reduce the impact of time delay on the system operation

In addition to the above main approaches, some other interesting works were also introduced In [72], the minimum jerk motion model is employed to predict the input from the user a time into the future This predicted input is then used to stabilize the remote robot In [35], multiple commands with all possible happened delay are sent in one packet to ensure the relevant of control signal

It is recognizable from the above approaches that they mainly focus on dealing with the time delay However, communications over the network is influenced by not only the time delay but also other various problems such as the message loss, out-of-order delivery, and limited transmission bandwidth These problems affect the control in such a complicated way that may cause the overall system to be un- stable

1.3.3 Study of NRSs on navigation

Robot navigation includes different interrelated activities such as perception, localization, cognition, and motion control In NRSs, studies on navigation include two methods: direct and behavior-based navigations

In direct navigation, teleoperators set the primitive force or velocity commands to perform the remote control In [61], haptic interfaces were introduced to provide force feedback to the teleoperator A force sensor which is a two-way full bridge strain gauge is fixed at the robot’s front center When the teleoperator controls the robot to push an object, the sensor measures the force and transmits it over the In- ternet to the teleoperator’s haptic joystick The joystick generates proportional forc- es to act to the teleoperator’s hands In [85], a haptic device was also implemented, but the author focused on the communications A compensative parameter that reflects the time delay was proposed to continuously adjust the feedback transfer function of the control loop As the result, the force feedback to the operator was as exact as at the time of measurement Advantage of force feedback is the supply of operators with sense of interaction However, control architectures still need to be developed to ensure the functioning operation of the whole system

In [31], a direct control architecture for Internet-based personal robot, which is insensitive to the time delay is introduced Main components of the architecture includes a command filter to recover the loss information of control commands and a path generator and a path-following controller to reduce the time difference between the real and virtual robots In addition, a posture estimator is employed to overcome the difference between the two environments In another approach, Wang and Liu introduced the telecommanding concept [87] In telecommanding, each command is designed for an independent task and is defined with multiple events so that the robot can deliberately respond to expected events while reactively respond to unexpected events

Advantage of the proposed approaches is the flexibility in control However, direct transmit of control signals to the actuator over the network without modification may cause problems in some circumstances For instance, those signals can be delayed or lost so that the actuator does not perform as expected In addition, direct navigation requires the operator to have experience in handling unpredicted situations caused by the network

The goal of the research

Motivated from the wide applicability and active research, this work addresses the problem of supervision and control of NRSs The goal of the research is to realize new and effective algorithms for the localization, stabilization control, and navigation which are fundamental problems of NRSs As networks are in general very complex and can greatly differ in their architecture and implementation depending on the medium used, and on the applications they are meant to serve, this work uses the Internet as the communications network and limits its influence factors to the time delay, message loss, and out-of-order delivery

The localization algorithm will be designed using optimal filter approach A unified state-space presentation of the system is constructed to describe the influence of networked induced problems Based on it, estimation theory is applied to formulate a new filter for the localization of NRS

The stable controller will be developed using the Lyapunov stability theory and optimal filter The Lyapunov stability theory is employed to derive the control laws that stabilize the non-networked robot system Those control laws are then extended to the NRS by using optimal filters

Finally, the navigation will be addressed using the behavior-based approach

This approach combines the results of localization and stable control into a single architecture This architecture divides the complex navigation task into separate behaviors to execute The coordination between the behaviors will be handled by the fuzzy logic The accuracy of the localization and control is required to be 30 cm in position and 10 0 in orientation for the functioning operation of the navigation.

The organization of this thesis

The thesis consists of six chapters organized as shown in figure 1.5 Chapter 1 gives a brief overview of the NRS It discusses applications and reviews related works

The chapter ends with the research goal and thesis structure

Chapter 2 describes the system model A state-space representation of the NRS is introduced with the existence of network parameters Details of each parameter are then analyzed Finally, the kinematic model, hardware configuration, and data communications of the NRS are presented

The problem of localization is described in chapter 3 This chapter first discusses related methods with the evaluation of their strengths and weaknesses It then presents our algorithm from the theory to the simulations and experiments

Chapter 4 Motion control using Lyapunov theory and predictive filter

Chapter 5 Navigation using behavior- based model

Chapter 3 Localization using optimal filter

Figure 1.5: Schematic structure of the thesis

Chapter 4 describes the design of the stable controller It starts with the derivation of the control laws that stabilize the non-networked robot system It then explains the expansion of those control laws to the NRS The chapter ends with simulation and experiment results

The results from previous chapters are combined in chapter 5 to develop a navigation model This chapter first highlights some navigation strategies It then describes our behavior-based navigation with details of each behavior and experimental results

The thesis ends with chapter 6 which lists summary of the research, declaration of main contributions, and recommendation for future work.

System Model

State-space representation of the NRS

Consider a general robotic system with fundamental components including the controller, actuator, process, and sensor Its state-space representation in discrete-time domain is given by:

( , ) k =h k k z x v (2.2) where k∈ is the time index, x k ∈ n is the state vector, u k ∈ q is the input vector, k ∈ m z  is the output, f and h are the state and output functions, and w k and v k are the process and measurement noises, respectively These noises are assumed to be independent, zero-mean, and white-noise processes with normal probability distributions: w k ~ (0, )N Q k , v k ~ (0, )N R k , (E w v i j T ) 0= This assumption is sufficient and widely adopted due to the central limit theorem [10, 70]

In our study, the robot is the type of differential-drive mobile robot with two driving wheels and two passive wheels (figure 2.1) The driving wheels regulate the position and orientation of the robot The passive wheels, one at front and one at rear, keep the robot balance These passive wheels are designed and manufactured so that they are flexible and do not influence the kinematics of the robot The pose of the robot is defined as the position of the wheels axis center (x, y) and the orientation θ of the chassis with respect to the X axis Figure 2.1 shows the coordinate systems and notations for the robot where (XG, YG) is the global coordinate, (XR,

YR) is the local coordinate attached to the robot center, R denotes the radius of driving wheels, and L denotes the distance between the wheels It is assumed that the robot is rigid moving on the plane with non-slipping and pure rolling

Figure 2.1: The robot’s pose and parameters

The kinematic model of the robot is then given by [30]:

(2.3) where ω L and ω R are the angular velocities of the left and right wheels at its center of rotation, respectively Let

L ω ω ω = − (2.5) be the tangential and angular velocities of the robot, equation (2.3) is rewritten as: cos sin c c c x v y v θ θ θ ω

As the system operates over digital communications network, equation (2.6) need be discretized Let T s be the sampling period, the discrete time model is obtained from Taylor's expansion at k as:

T T dx dx dx x x T dt dt n dt

Taking n=1, the discrete time Euler approximation of the robot is given by:

This approximation is valid provided that the sampling period is sufficiently small

More specifically, the choice of the sampling period is based on the approximation error given by the Taylor’s theorem with the Remainder Term [79] as follows:

= x (2.9) where e is the approximation error and t c ∈[ ,k k+1] In our system, the sampling period is 100 ms resulting in the maximum approximation error of 0.5 cm This value definitely satisfies the accuracy requirement of the system which is 30 cm (radius of the robot chassis)

From the kinematic equation (2.8), vectors in the state equation (2.1) are defined for our system as follows:

• The state vector x includes the position and orientation (pose) of the robot:

• The input vector u contains setting angular velocities set by the controller to the left and right driving wheels: u = [ ω ω L R ] T As angular velocities of the wheels linearly relate to the translational and angular velocities of the robot by equations (2.4) and (2.5), we sometimes use [ v c ω c ] T instead of [ω ω L R ] T for the convenience of notation

• The state function f is represented by equation (2.8) It relates the state vector

• The process noise w is modeled as the noise influencing the input This approach is adequate provided that the systematic errors are eliminated by the calibration process [77] Let ω L a and ω R a be the real angular velocities (noise included) of the driving wheels, the process noise w k is defined as: n n T k = ω L ω R  w (2.10) a n

In our system, sensors can directly measure the position and orientation of the robot More specifically, the position ( , )x y is measured by the optical encoders attached to the driving motors and the orientation θ is measured by the compass sensor Detailed implementation of the sensors is presented in section 2.3.1 Based on it, vectors in the output equation (2.2) are defined for our system as follows:

• The output vector z includes the position and orientation of the robot:

• The output equation (2.2) can be simply rewritten as: z x v = + (2.13)

• Noise v are the measurement noise of sensors This noise and the process noise are assumed to be independent, zero-mean, and white-noise processes with normal probability distributions The determination of their covariances is presented in section 3.4

When distributing over communications networks, the robot system is decentral- ized and its functioning operation depends on a number of network parameters The networks are in general very complex and can greatly differ in their architecture and implementation depending on the medium used, and on the applications they are meant to serve

Figure 2.2: Model of the NRS

In this work, a network is modeled as a module between the process and controller which delivers input signals and observation measurements with possible delay, loss, and out-of-order (figure 2.2) The delay is assumed to be random, but measurable at each sampling time The out-of-order is considered as a long delay The packet arrival is modeled as a binary random variable λ k defined as follows [64]:

1, if a packet arrivesduring time 1to

Let n be the network delay (in number of sampling periods) between the controller and the actuator, m be the network delay between the sensor and the controller, k ca λ be the binary random variable described the arrival of inputs from the controller to the actuator, λ k sc be the binary random variable described the arrival of measurements from the sensor to the controller The NRS and its signal timing can be described as in figure 2.3 The controller, which is a software program installed at the remote computer, first generates setting velocities u=[ω ω L R ] T in three ways depending on the control purpose: pre-determined as an array of values in localization; computed in real time by a control law in stabilization control; and inputted by users via joystick in navigation The values of velocities are then transmitted over the network to the actuator On the other side, the output z=[x yθ] T of the system are measured by sensors The values, which suffer from measurement noise, are then transmitted over the network to the controller The delivery of these data are in packets with the time stamp and sequence number included for the determination of delay, loss, and out-of-order The timing of signals can be explained as follows:

(Delay, Loss, Out of Order)

Figure 2.3: Timing of signals in the NRS

At time k, the controller sends a control input u to the actuator Due to the network delay or out-of-order n, the control signal arrives to the actuator at time k+n

After one sampling period to the time k+n+1, the system state changes The sensor updates this by taking the measurement z This measurement is transmitted over the network and arrived the controller at time k+n+m+1 The controller employs the measurement as feedback information to start a new control cycle During the operation, there always exist probabilities that the control input and measurement are lost

From the analysis, the robot state at time k is driven by the control input at time k-n-1 while the received measurement at time k is actually taken at time k-m The system (2.1) – (2.2) becomes time-varying and can be rewritten as follows:

Equation (2.15) describes the robot state under the influence of the network It is the model of the NRS and will be used throughout this study In the following sections, we discuss more details about components of the model including the communications network and the robot.

The communications network

Industrial communications networks were introduced in the 1970’s in car industry in order to reduce cost for cabling, ease the system setup, and modularize the system design Since then, various types of network have been developed to serve numerous demands of applications In NRSs, networks used vary in communications protocols and topologies For industrial and transport applications, it is convenient to use fieldbuses (e.g FIP and PROFIBUS) and automotive buses (e.g CAN) For service, education, and some others, it is more appropriate to employ general purpose networks (e.g IEEE LAN’s and ATM-LAN) and the Internet A short summary is now given to describe some popular networks and their characteristics

Four types of network including the Foundation Fieldbus, CAN, Ethernet, and Inter- net will be briefly presented They are commonly used in NRSs

The Foundation Fieldbus was developed by the organization of Fieldbus Foundation in 1994 [83] It was intended for replacement of traditional 4 − 20 mA standard, but today it is widely installed in many heavy process applications such as oil refineries, chemical plants, paper mills and power generations Two communications protocols of Foundation Fieldbus have been released to meet different needs of control and monitoring: H1 and HSE The first, H1, works at 31.25 Kbit/s and is used to connect the field devices such as sensors, actuators, and controllers The second, HSE (High- Speed Ethernet), uses 10 or 100 Mbps Ethernet as the physical layer and provides a high-speed backbone for the network A typical Foundation Fieldbus network topology has HSE connections between computers, and runs slower H1 links between the devices themselves By using bridges, more devices can be connected and a hierar- chical network topology can be built Access control in a Foundation Fieldbus network is controlled by link masters The link master that is currently controlling the bus is called the Link Active Scheduler (LAS) During runtime, the LAS will send commands to other devices, telling them when to broadcast data to the bus using a schedule The LAS also publishes time information and grants permission to devices to allow them to broadcast unscheduled messages such as alarms and events

Development of CAN started originally in 1983 by the German company Bosch

[84] It was designed specifically for automotive industry, but now also used in other areas such as aerospace, industrial automation and medical equipment CAN is defined in the ISO standards 11898 and 11519-1 The bit rate can be 1 Mbit/s if the network length is not longer than 40m, and 500 Kbit/s if the network length is longer than 40m In a CAN network, devices such as sensors and actuators are not directly connected to the bus, but through a node Each node requires three components: a transceiver to receive or transmit signals at bit level, a CAN controller to convert bits into messages and vice versa, and a host processor to process messages There is no limit to the number of nodes Each node is able to send and receive messages, but not simultaneously If the bus is free, any node may begin to transmit If two or more nodes begin sending messages at the same time, the message with the highest priority gets the right to use the bus There are 229 different priority levels for messages

Messages with numerically smaller values of IDs have higher priority and are transmitted first

Ethernet was standardized in 1985 as IEEE 802.3 [73] Since then, it has constantly evolved to meet new requirements and become one of the most used local area network (LAN) technologies Ethernet can transmit data with the speed from 10 Mbit/s to 100 Gbit/s, but the most commonly used speeds are 10 Mbit/s, 100 Mbit/s, and

1000 Mbit/s An Ethernet segment is usually limited to 100 m in length Multiple Ethernet segments can be joined together by hubs or switches An almost unlimited number of stations can be connected to an Ethernet Each station has a unique Ether- net address defined as a sequence of six numbers separated by colons Each number corresponds to 1 byte The first three bytes are used as a vendor ID, and the last three bytes are defined by the vendor Ethernet uses a bus access method called CSMA/CD, Carrier Sense Multiple Access with Collision Detection The "carrier sense" means that all the stations can distinguish between an idle and a busy link

The "collision detection" means that a station can detect when a transmitting frame has interfered with a frame transmitted by another, and if this happens, the colliding stations will back off, and try a retransmission after a random wait An Ethernet frame, or packet, is between 64 and roughly 1500 bytes in length A frame begins with 64-bit preamble for delimitation and synchronization, followed by a header fea- turing source and destination MAC addresses The middle section of the frame contains payload data which is between 46 and 1500 bytes The frame ends with a 32-bit cyclic redundancy check

Starting from ARPANET, the Internet was brought online in 1969 which initially connected four major computers at universities in the southwestern United State

[28] Through developments and evolutions, today, the Internet became a global data communications system that links millions of private, public, academic and business networks via an international telecommunications backbone that consists of various electronic and optical networking technologies As a network of networks, the Inter- net is heterogeneous; for instance, data transfer rates and physical characteristics of connections vary widely The Internet has no centralized governance in either tech- nological implementation or policies for access and usage Every device on the In- ternet, such as a computer or mobile phone, is identified by a unique number called

IP address Each IP address consists of 6 bytes for IP version 6, and 4 bytes divided into five classes A, B, C, D, and E for IP version 4 In order to transmit data between hosts, the Internet uses the Internet protocol suite, also known as TCP/IP It is a model architecture that divides methods into a layered system including the application, transport, Internet, and link layers As the user data is processed down through the protocol stack, each layer adds an encapsulation at the sending host At the receiving host, the encapsulation procedure is reversed

Though networks are different in design and structure, and are in general complex in implementation, they share several common characteristics when being used for NRSs In this study, the network delay, message loss, and out-of-order delivery are investigated Addressing those characteristics allows control algorithms to be effective for not only a specific type of network but also generic communications networks

Sending a message over a network typically takes some time Depending on the network and scheduling policy in the system, this transfer time can have different properties In some setups, the transfer time can be nearly constant, but in many cases it is time-varying For instance, the length of the transfer delay can depend on the network load, priorities of the other ongoing communications, and electrical disturb- ances

The network delay can be measured It is often carried out by reading the timestamp added to the sending message and then comparing it with the receiving time This method requires internal clocks of the sender and receiver to be synchro- nized Typically, the synchronization is based on the concept of estimating the difference between the clocks of two nodes by sending clock-requests back and forth In this study, the clock synchronization is carried out as follows

Let S be the node which wants to estimate its clock difference to node R Let t S be the local time in node S, t R be the local time in node R, and t * be the absolute time

The local clocks in S and R have skews to the absolute time as:

= + (2.16) where δ S and δ R are the clock mismatches The clock offset, δ, is defined as:

From equations (2.16) and (2.17) it follows that:

In order to estimate the clock offset δ, the process starts with a synchronous re- quest from node S to node R at time t S a As node R receives the message from node S, it instantly returns a message containing the local clock value t b R This message arrives node S at time t S c Figure 2.4 explains the process

Let ∆t SR and ∆t RS be the transfer times from S to R and from R to S, respectively

The transfer times are given by:

Assuming that E t(∆ − ∆ SR t RS ) 0= , where E denotes the expectation operator, we get:

By repeating the clock synchronization experiment we can find an accurate estimate of δ The network delay is then determined by equation (2.19)

Messages may arrive in wrong order This phenomenon especially dominates in packet switching network In that network, messages are packed in datagram packets before being sent Depending on the link state and router configuration, packets can be delivered through different routes and arrived in wrong order

The out-of-order can be detected by adding sequence numbers to sending messages For instance, if sequentially numbered messages arrive as 1, 3, 4, 2, 5, then message 2 is out of order From the time view, the out-of-order can be considered as a sudden long delay Message 2 in the above example is delayed two sampling periods more comparing to the remaining ones More formally, an out-of-order message with sequence number i arrived at time k (i k< ) equivalents to a delayed message with the time delay:

The Robot

Robots used in NRSs can be any types from manipulator to humanoid robot depending on research purposes and applications In this work, a real NRS was developed to serve as the platform for study and experiment Figure 2.5 shows an overview of the system Its details including the hardware configuration and data communications are described as follows

Figure 2.5: Overview of the developed NRS

The hardware configuration consists of two parts: actuators and sensors, and user- interaction devices Details are described as follows

Figure 2.6: Hardware configuration of the robot

The robot has the size of 60 cm x 60 cm x 110 cm It contains basic components for sensing and navigation The components include drive motors for motion control, sonar ranging sensors for obstacle avoidance, compass and GPS sensors for heading and global positioning, and laser range finder (LRF) and vision system for mapping and navigation (figure 2.6)

The drive system uses high-speed, high-torque, reversible-DC motors Each motor is attached a quadrature optical shaft encoder that provides 500 ticks per revolu- tion for precise positioning and speed sensing The motor is controlled by the Motion mind PID controller It is a microprocessor-based electronic circuit with an embedded firmware that permits to control the motor by a PID algorithm

Motion mind 3 Laser range finder

The heading module contains a CMPS03 compass sensor operating based on the Hall effect (figure 2.7) The module is used to determine the robot orientation with the heading resolution of 0.1° It has two axes, x and y Each axis reports the strength of the magnetic field component paralleled with it The microcontroller connected to the module uses synchronous serial communications to get axis measurements [20]

North y-axis x-axis angle θ angle θ=arctan(-y/x)

Figure 2.7: Operation of the compass sensor

The GPS is mainly applied for positioning in the outdoor environment A HO- LUX GPS UB-93 module is used [33] Due to the presence of networking in our system, an Assisted GPS (A-GPS) also can be used in order to locate and utilize satellite information of the network in the poor signal condition

The system provides eight SFR-05 ultrasonic sensors split into four arrays, two on each, arranged at four sides of the robot [21] The measuring range is from 0.04 m to 4 m

The visual system is detachable and mounted on the head of the robot It can be switched between a Sony EVI-D100 pan-tilt-zoom (PTZ) color camera and an omni- directional digital camera Hyper-Omni Vision SOIOS 55 The CCD camera gives a clear view of the environment in front of the robot ranging from -100 0 to +100 0 in horizon and from -25 0 to +25 0 in vertical The omni-directional camera consists of a hyperbolical mirror placed a short distance above a normal camera to acquire a 360- degree view around the robot (figure 2.8)

Figure 2.8: Operation of the omni-directional camera

On front side of the robot, a 2D SICK-LMS 221 laser range finder (LRF) is implemented [68] Its range is from 0.04m to 80m Its operation is based on the time- of-flight measurement principle A single laser pulse is emitted out and reflected by an object surface within the range of the sensor The lapsed time between emission and reception of the laser pulse is used to calculate the distance between the object and the sensor By an integrated rotating mirror, the laser pulses sweep a radial range in its front so that a 2D field/area is defined (figure 2.9)

Figure 2.9: Operation of the laser range finder [68]: (a) Direction of scanning angle; (b) The scanned image

The communications between actuators and sensors is transferred via several channels: low-rate channels with standards of RS-485 and RS-232 and high-rate channels with USB ports (figure 2.10) Devices using the RS-485 are managed by an on-board 60MHz Microchip dsPIC30F4011-based microcontroller with independent

Figure 2.10: Communications scheme between sensors and actuators controller boards A RS-485 bus is established to maintain the communications between controllers Devices using the RS-232 are connected the USB-to-COM modules Control commands and short messages are transferred via low-rate channels

Images from camera are directed to a high-rate USB port In the system, the optical

Motor for vertical rotation of LRF

Motor for driven wheel encoders are used to measure indirectly the robot position (x, y) whereas the compass sensor is employed to directly measure the robot orientation θ with absolute value

Control devices are located at user’s site They consist of a personal computer and a joystick The computer is an ASUS notebook computer with 1.5GHz M-Centrino processor, 500Mb RAM and Windows XP operating system A control software is developed to allow users to retrieve feedback information of the remote site and navigate the robot (figure 2.11) To support users with a more convenient way of control, a joystick is added It is a 3D Logitech Extreme series with 10 bit resolution in hori- zontal and vertical axes and 12 functioning buttons In the system, the joystick interprets user’s inputs to a sequence of control parameters and forwards them to the control software for processing

Figure 2.11: Graphic user interface of the control software written in Microsoft

As shown in figure 2.5, there are two configurations for the robot to connect to the Internet The first one uses a 802.11 wireless access point The other uses a 3G mobile network In both configurations, the data communications is handled by a multi- protocol model The model utilizes different protocols for each type of data exchanged so that the overall performance is enhanced The choice of protocols is based on the analysis of each protocol in conjunction with the data exchanged in a NRS

The Internet protocol suite consists of four abstraction layers: data link, network, transport, and application layers Each layer includes several communications protocols to fulfill the function of that layer In NRSs, most studies focused on transport protocols [44, 45, 46, 59] Among various transport protocols such as TICP, ALCAP, STCP, etc [23], the transmission control protocol (TCP), user datagram protocol (UDP), and real-time transport protocol (RTP) are widely used and standardized to be the Internet standards [57, 58, 66]

TCP is a sophisticated protocol which was originally designed for the reliable transmission of static data such as e-mails and files over low-bandwidth, high-error- rate networks [58] In each transmission session, TCP establishes a virtual connection between the sender and receiver, performs the acknowledgment of received data packets and implements the retransmission mechanism when necessary In NRSs, several attempts to use the TCP have been proposed [59, 60] However, they are hardly appropriate since the timeliness of transmission outweighs the reliability con- cerns In addition, the retransmission mechanism is ill-suited for real-time data delivery and the strict congestion control mechanism incurs higher delay jitter which rapidly degrades the quality of service (QoS) in a congested network

UDP, on the other hand, is based on the idea of sending a datagram from a device to another as fast as possible without due consideration of the network state

[57] This protocol does not maintain a connection between the sender and receiver

It also does not guarantee the arrival of transmitted data as well as the chronological order of data at the receiving end The main advantage of UDP is the relatively min- imized transmission delay and delay jitter achieved under good network conditions

UDP is often used for real-time applications [18, 44, 45]

RTP is a relatively new protocol designed for delivering real-time multimedia data [56, 66] It bases on UDP, but adds facilities for jitter compensation and out-of- sequence detection RTP is often used in conjunction with the RTP Control Protocol (RTCP) While RTP carries the media streams (audio and video), RTCP is used to monitor transmission statistics and information relating to the quality of service

Conclusion

It is able to summarize the system model as follows:

The state-space representation that describes the overall NRS is presented in equation (2.15) The network is modeled as a module between the process and controller which delivers input signals and observation measurements with possible delay, loss, and out-of-order (figure 2.2) The delay is assumed to be random, but measurable at each sampling time The out-of-order is considered as a long delay described in equation (2.21) The packet loss is modeled as a binary random variable given by equation (2.22)

The robot is the type with two wheels, differential drive Its continuous and discrete kinematic models are described by equations (2.6) and (2.8), respectively The hardware configuration consists of actuators and sensors for sensing and navigations, and remote control devices for user interaction The data communications is carried out by a multi-protocol model This model utilizes advantages of different transport protocols to better support the NRS by enhancing properties such as real time and reliability.

The result of this chapter are published in [1, 2, 3, 4, 5, 10].

Localization Using Optimal Filter

Robot localization

Localization is the estimation of robot’s location relative to the environment When we talk about the robot’s location, pose, or position, we mean the x and y coordinates and the heading direction of the robot in the global coordinate system Locali- zation is important in mobile robotics If the robot knows its own pose in the environment, it is truly autonomous in executing given tasks By some authors, the robot localization has been stated as the “most fundamental problem to provide robots truly autonomous capabilities” [19] Consequently, localization has received much research attention in the past decade and significant advances have been made on this front [9, 67, 80] It is able to classify localization methods into three approaches including the dead reckoning, absolute positioning, and sensor fusion

The dead reckoning, also referred to as relative positioning, is the estimation of robot’s position based on the speed and direction of travel and the time that passed since the last known position Odometry is an example It works by integrating in- cremental information over time By using wheel encoders to count the number of revolutions of each wheel, the robot measures the distance it traveled and the direction it changed Those displacements are then combined with the previous position to update the present position Odometry is widely used because it gives good short- term accuracy, allows high sampling rates, and is inexpensive It however accumu- lates errors over time so the error may grow unboundedly if appropriate compensation is not considered For this reason, it is often used along with absolute positioning to offset the error after a certain period of time

The absolute positioning employs measurements that are relative to the global frame to locate the robot Map matching is a popular technique This approach uses geometric features of the environment to compute the location of the robot The features can be lines and points that describe walls and corners or specific shapes such as rectangular or triangular depending on the structure of the environment The main advantage of the absolute positioning is its independence from previous esti- mations and accumulative errors Its limitation includes the complexity in implementation and the dependence on the environment structure

The method that employs both relative and absolute measurements is called the sensor fusion Since not all sensors are able to sense the same, some features may be occluded for some sensors while visible to others Thus, fusing information from multiple sensors is a good way to overcome the problem Together the sensors can provide a more accurate and more complete picture of the environment Several fusion techniques have been proposed such as the central limit theorem, Kalman filter, Bayesian networks, and Dempster-Shafer Among them, the Kalman filter is a pow- erful tool to combine all available information from multiple sources to give an optimal estimate in a statistical sense [10] This method has advantages when applying for NRSs.

Localization of NRSs

In NRSs, the localization faces new challenges related to communications networks

The challenges include but not limited to the inevitable communications delays, out-of-order data arriving, partial intermittent of observations, and non-guaranteed bandwidth As discussed in section 1.3.1, a number of studies have been introduced to address those problems They use either advance interface techniques or optimal filters

In advance interface techniques, the robot’s pose is determined at the local site before transmitting to the network At the remote site, this pose is extrapolated to the being estimate based on the network state This method is simple to implement, but its accuracy is low In addition, it is insufficient to deal with the mixed uncertainties of delay, loss and out-of-order

Optimal filters can overcome those limitations In this approach, the influence of the network is modeled and incorporated into the system model Optimal estimation theory is then applied to obtain the robot state Current filters mainly focus on a specific problem such as the network delay or message loss Several studies addresses multiple problems, but they are not optimal for the out-of-order delivery In addition, most filters are designed for linear systems Further modifications are required to implement them to nonlinear systems such as NRSs

In this study, we employ the optimal filtering approach for the problem of localization A new filter called past observation-based extended Kalman filter (PO- EKF) is introduced It can simultaneously deal with the delay, loss, and out-of-order caused by the network The derivation of the filter is based on the Kalman filter’s theory The main idea is the determination of a “relevance factor” which describes the relevance of observations from the past to the present This factor is employed as a multiplier to enable the incorporation of delayed or out-of-order measurements to the posteriori estimation In case of missing the measurement, the filter performs a subsequent prediction step in which the kinematic model and input signal are fused.

Localization of NRSs using past-observation based extended Kalman filter 45

This section first briefly introduces the Kalman filter which will be used as funda- ment for our algorithm The derivation of the optimal filter for linear NRS is then described Finally, the expansion of the filter to nonlinear system is presented

The Kalman filter by definition is a set of mathematical equations that provides an efficient computational (recursive) means to estimate the state of a process, in a way that minimizes the mean of the squared error The implementation of the filter is executed through two steps: the time update to predict the state before taking the measurement and the data update to correct the predicted estimate after the measurement is available

Consider the following discrete time linear stochastic system:

= + x x u w z x v (3.1) where k∈, x, w∈ n , z, v∈ m , u∈ l , A∈ n n × , B∈ n l × , H∈ m n × , ( , , )x w v 0 are Gaussian, uncorrelated, white, with mean ( ,0,0)x and covariance ( , , )P Q R 0 respectively The steps to calculate the Kalman filter can be summarized as follows:

• The time update equations (prediction phase):

P − = A P A − + − − +Q − (3.3) where ˆx k − ∈ n is the priori state estimate at step k given knowledge of the process prior to step k, and P k − denotes the covariance matrix of the priori estimate error

• The data update equations (correction phase):

P + = −I K H P − (3.6) where ˆx k + ∈ n is the posteriori state estimate at step k given measurement z k , K k is the Kalman gain, and P k + is the covariance matrix of the posteriori estimate error

More details of the Kalman filter can be referred in [10, 70]

3.3.2 Optimal filter for linear NRSs

Consider the NRS described in equation (2.15) If we define the state vector x as [x yθ] T x= where (x, y) is the position and θ is the orientation of the robot, then the localization becomes the problem of estimation of the state vector x If the functions f and h are linear, the system (2.15) can be rewritten as:

(3.8) where u k n − − 1 , z i k , H k m − , v k m − , and i are defined by the above equations We now derive the optimal filter to estimate the state of the linear system given in equations (3.7) and (3.8)

Priori State Estimate: The priori state estimate is defined as the expectation of the state x k given all measurements up to and including the last step k-1 By taking the expected value of both sides of equation (3.7), we can express the priori state estimate ˆx k − as:

As E(x k − 1 ) is the posteriori state estimate at time k-1, u k n − − 1 is a known input and

1 k − w is Gaussian zero-mean white noise, equation (3.9) becomes:

Priori Error Covariance: Let e k − and e k + be the priori and posteriori estimate errors, respectively: k k ˆ k

From equations (3.7) and (3.10) we obtain:

Taking the expected value of both sides of equation (3.13) and noting that k + 1 e − is uncorrelated with w k − 1 , we obtain the covariance of the priori estimate error:

Posteriori State Estimate: From equation (3.8) the measurement z k would update the system state at a previous time i rather than the present time k Due to the delay or the out-of-order problems, this measurement could not reach the estimator until time k We therefore construct the data update equation of the form: ˆ k + = ˆ k − +K k ( i k −H i i ˆ − ) x x z  x (3.15)

That is, the correction term is the past residual at which the measurement is taken

This can be explained by computing the mean of estimation error:

From equations (3.7) and (3.10) we get:

Substituting equation (3.17) to (3.16) and noting that w k − 1 and v i are zero-mean obtain:

Equation (3.18) implies that if the initial estimates of x are set equal to the expected value of x, the posteriori estimate ˆx + , on average, will be equal to the true value x

This is the desirable property of our estimator

Kalman gain and Posteriori Error Covariance: Next, the value of the Kal- man gain K k and the posteriori error covariance are derived to ensure the optimality of the filter The optimal criterion is to minimize the sum of the variances of the estimation errors The cost function is:

Assume that the measurement is fused using equation (3.15) with an arbitrary gain K k From equation (3.16), the covariance of the posteriori estimate error, P k + , is determined as:

Due to the independence between e − and v , equation (3.20) can be simplified to:

As the matrix K k is chosen to be the gain or blending factor that minimizes the cost function J k This minimization is accomplished by taking the derivative of J k with respect to K k , setting that result equal to zero, and then solving for K k Apply- ing this process with noting that tr A tr A( )= ( ) T , tr XA( ) A T

∂ , the derivative of J k is obtained as:

Setting equation (3.22) to zero gives:

In order to compute L, the priori state estimate at time k needs determining from the estimate at time i Through the time update equation (3.10) and the data update equation (3.15), e − becomes:

After m updating steps, the estimation error becomes:

=∏ −  (3.27) and ξ 1 and ξ 2 are the functions of noise sequences w and v From equation (3.26) and the independence between e − and noise sequences, the covariance L becomes:

Substituting equation (3.28) in (3.24) and (3.23) yields: k k k i i T

□ Summary: The optimal filter for the linear NRS can be summarized as follows:

We call this filter past observation-based Kalman filter (PO-KF)

Remark 1: When the delay n is zero, the time update equation (3.31) reduces to the standard form (3.2) When the delay m is zero, F I= and the Kalman gain (3.30) reduces to the standard form (3.4) and the error covariance (3.29) reduces to (3.6)

Remark 2: Equation (3.30) can be rewritten as: k i *

K =FK (3.37) where K i * is the Kalman gain at time i of the standard Kalman filter (3.4) It turns out that the past residual (z k −H i i xˆ − ) in equation (3.15) can be normally updated to the posteriori estimate at time k as at time i, but the Kalman gain needs to be changed by the factor F This factor describes the relevant of the measurement updated at time i to the state at time k

Remark 3: From equations (3.33) – (3.36), the computation of the PO-KF depends on values of the random variables λ and m As analyzed in chapter 2, these values are determined and measurable at each sampling time: λ k is determined by equation (2.14) and m is measured by the time stamp included in the receiving message (see section 2.2.2) Thus, the implementation of the PO-KF is feasible

Remark 4: Equation (3.35) can be rewritten as: ˆ k + = ˆ k − +λ i K k ( i k −H i i ˆ − ) x x z x (3.38)

It means that if a measurement is not arrived (λ i =0), a “dummy” measurement will be employed, but the estimator does not convey any information of this “dummy” measurement for the estimate It simply sets the posteriori estimate to the value of the priori estimate In another word, there is no correction phase The estimator becomes open-loop and the accuracy is reduced This result is logical as if the feedback measurement is lost, we have no information to adjust the prediction

On the other hand, the network delay m does not cause the loss of information

Instead, it adds extra complication to the system, but as far as the network delay is measurable, it can be integrated into the computation process to retrieve relevant information for the estimate (equations (3.33) – (3.36)) As will be shown later in simulations, the network delay causes estimate errors at initial steps, but when the delayed measurements arrive, the errors quickly reduce and finally reach the steady state

3.3.3 Optimal filter for nonlinear NRSs

As NRSs are often nonlinear, modifications needs to be accomplished to extend the PO-KF for nonlinear systems Our main idea is the linearization of a nonlinear system around its previous estimated states

Performing a Taylor series expansion of the state equation (2.15) around

(3.39) where A k − 1 , W k − 1 are defined by the above equation The known signal u * k n − − 1 and the noise signal w * k − 1 are defined as follows:

Similarly, the measurement equation (2.15) is linearized around ˆ( ,0)x i − to obtain ˆ ˆ

(3.42) where h i , H i , V i are defined by the above equation The known signal ε * i and the noise signal v * i are defined as follows:

The system (3.39) and the measurement (3.42) now become linear and the PO-

KF can be applied to obtain the optimal filter for networked robot localization as follows:

• The time update equations at the prediction phase:

• The data update equations at the correction phase:

We call this filter past observation-based extended Kalman filter (PO-EKF) It is noted that the PO-EKF is suboptimal rather than optimal due to the linearization process.

Implementation of the PO-EKF for the differential-drive network robot

The PO-EKF introduced in previous section is in general form and applicable to various network control systems In this section, the implementation of the PO-EKF for the differential-drive NRS is described

In order to implement the PO-EKF, it is required to know the network state including the message loss, delay and out-of-order at each sampling time In our system, the message arrival λ k is determined by equation (2.14) in which the value equals

“1” if a message is received during time k-1 to k and equals “0” otherwise The network delays n and m are measured by comparing the sending time included in the receiving message with the receiving time Noting that a clock synchronization is required to use this method (see section 2.2.2) Finally, the out-of-order is determined by the sequential number included in the receiving message The corresponding time delay is given by equation (2.21)

3.4.2 Implementation of the prediction phase

The estimation of the priori estimate ˆx k − in equation (3.45) is carried out via the kinematic equation (2.8) of the robot where the function f is defined The input

[ω ω L R ] T u= is pre-determined depending on the trajectory we want the robot to move In order to compute the priori covariance P k − , matrices A k − 1 , W k − 1 and Q k − 1 need be identified From the kinematic equation (2.8) and equation (3.39), A k − 1 and k 1

Q k − is determined by the process noise According to section 2.1, the process noise is modeled as being proportional to the angular velocities ω L and ω R of the left and right wheels, respectively The variances equal to δω L 2 and δω R 2 , where δ is a constant determined by experiments as follows: the deviations between the true robot position and the position estimated by the kinematic model when driving the robot straight forwards several times (from the minimum to the maximum tangential speed of the robot) is determined The deviations between the true robot orientation and the orientation estimated by the kinematic model when rotating the robot around its own axis several times (from the minimum to the maximum angular speed of the robot) is also determined Based on those errors in position and orientation, the parameter δ is calculated In our system, the δ is estimated to be 0.01 The input-noise covariance matrix Q k is then given by:

In the implementation, the initial estimate xˆ 0 + is set to zeros implying that the origin of the global coordinate system is set at the starting position of the robot The initial error covariance P 0 + is chosen to equal to the measurement error R 0 (equation (3.50)) In practice, the value of P 0 + does not influence the functioning operation of the PO-EKF as the error covariance P k + will quickly converge to the steady value

3.4.3 Implementation of the correction phase

Implementation of the correction phase is based on equation (3.46) The required parameters include z, R, H and V From equation (2.14), if the data packet is lost (λ i =0), these parameters will equal to zeros and there will have no correction phase This property has been discussed in remark 4 of section 3.3.2 In contrast, if the packet arrives (λ i =1), the parameters will become z, R, Hand V that we now determine

The output z includes the position and orientation of the robot Its value is measured by two kinds of sensors: optical encoder for position and compass sensor for orientation The covariance matrix R k of the measurement noise is determined similarly to the constant δ in which the measurements are compared with the true values under different configurations in many times to estimate the expectation and variance of the noise In our system, the covariance matrix of measurement noise is identified as:

Finally, matrices H k and V k are identity matrices due to equations (2.13) and (3.42): k k

In next sections, the performance of the PO-EKF will be evaluated via simulations and experiments.

Simulations

In order to evaluate the efficiency of the PO-EKF for the localization of NRSs, simulations have been carried out in MATLAB [24] The setup is as follows

The robot is identified by its motion and sensory system In our simulation, robot motion is represented by its kinematic equation (2.8) The parameters are set based on the real NRS as follows:

• Radius of the driven wheel: R = 5 cm

• Distance between the driven wheels: L = 60 cm

• The input is the angular velocities of the left and right wheels, u=[ω ω L R ], ranging from -26 rad/s to 26 rad/s Their values at each step are pre-determined and stored in the controller so that the robot follows trajectories shown in figures 3.1 and 3.2 The choice of trajectories will be discussed in the “simulating scenario” section At each execution step k, a value u k will be got out of the controller and sent over the network to the actuator

• The output includes the position and orientation of the robot, z = [ x y θ ] T Its initial values are chosen to be zeros, z 0 = 0 0 0 0  T , corresponding to the origin of the global coordinate system

In MATLAB, the representation of robot motion by its kinematics is carried out as follows:

% Compute the tangential and angular velocities (noise included)from the angular velocities (without noise) wL=wL(i-1)+PNoise_L(i); wR=wR(i-1)+PNoise_R(i); v=r_wheel*(wR+wL)/2; w=r_wheel*(wR-wL)/b_wheel;

% Time-based propagation of robot pose by its kinematics x(i)=x(i-1)+ dt*v*cos(theta(i-1)); y(i)=y(i-1)+ dt*v*sin(theta(i-1)); theta(i)=theta(i-1)+ dt*w;

In the simulation, it is assumed that the robot has a sensory system that can measure the output z similarly to the real system Values of sensors are then identified as the system output adding measurement noises:

% Values measured by sensors pos_mes_X(i)=pos_true_X(i)+ONoise_x(i); pos_mes_Y(i)=pos_true_Y(i)+ONoise_y(i); pos_mes_Theta(i)=pos_true_Theta(i)+ONoise_theta(i);

The network is modeled as a module that introduces the delay, loss and out-of-order to the system as in equation (2.15) The values of these parameters at each sampling time are set based on experimental measurements of the Internet as follows: the time delay varies from 100 ms to 800 ms; the data loss rate is 2%; and the out-of- order rate is 5%

In MATLAB, the loss rate is initialized as: lamdaCA = binornd(1,0.98,N,1); % Controller to actuator lamdaSC = binornd(1,0.98,N,1); % Sensor to controller where N is the total number of execution steps The out-of-order and delay are defined through two steps A binary random variable is first created with 5% of value

“1” corresponding to the occurrence probability of the out-of-order: lamdaOCA = binornd(1,0.05,N,1);

Then, at the time at which the lamdaOCA = 1, the time delay will be added an extra delay (j i T− ) s as in equation (2.21):

% Network delay a1=1;a2=8; % range of random delay b1=5;b2; % range of out-of-order

Delay(i)=round(a1+(a2-a1)*rand()+lamdaOCA(i)*(b1+(b2- b1)*rand()));

In some simulations, those network parameters will be set to be higher in order to simulate extremely bad network conditions

Implementation of the PO-EKF

Parameters for the PO-EKF such as A W Q H k , , , k k k and R k are chosen as presented in section 3.4 The definition of these matrices in MATLAB are straightforward by their formulas The input and measurement noises are defined via the covariant matrices For instance, the component x of measurement noise is defined as:

Var_ONoise_x = R(1,1); %1x1 Std_ONoise_x = sqrt(Var_ONoise_x)'; %1x1 ONoise_x = Std_ONoise_x* randn(1,N+2); %1x1 * 1xN = 1xN

In simulations, a set of input velocities (without noise) is set to the kinematic model of the robot at each sampling time to construct a theory trajectory Figures 3.1 and 3.2 show two theory trajectories that will be used The first one corresponds to an ordinary trajectory whereas the second one describes a difficult movement Due to the input disturbance and network influence, the robot actually follows a different path called the true trajectory (corresponding to x) Measurements are performed to generate an observation trajectory which represents the output z This trajectory suffers from the measurement noise Figure 3.3 compares the trajectories in X direction (noting that only 100 sample points are shown for the convenience of comparison) The localization algorithm is then performed to create the estimate trajectory

(corresponding to ˆx + ) This trajectory is expected to close to the true trajectory x

The difference between the estimate and the true trajectories determines the accuracy of the localization algorithm

In our evaluation, the PO-EKF will be compared with two popular localization algorithms including the extended Kalman filter (EKF) and the optimal filter proposed in [64] The EKF is implemented with the assumption that it does not know if a measurement is delayed or not It incorporates every measurement it receives as there is no delay On the other hand, the filter proposed in [64] is specifically designed for the system subject to random delay and packet drop It is called Lucas-Extended Kalman filter (LEKF) in this thesis The LEKF uses an infinite buffer to store and rearrange the delayed and out-of-order measurements Loss measurements are stored as zero values At each estimation step, the filter iteratively executes the Kalman equations from the initial to the being estimated state This method was proven to be optimal

Figure 3.1: An ordinary theory trajectory of the robot in the motion plane

Figure 3.2: An extreme theory trajectory of the robot in the motion plane

Figure 3.3: The theory, true and observation trajectories in X direction

The first simulation is conducted with the network parameters as follows: the time delay is between 100ms and 800ms; the out-of-order rate is 5%; and the loss rate is 1% Figure 3.4 show the estimate and true trajectories in the motion plane

The overlap between the trajectories implies the good estimation In order to com- pare between algorithms, the root mean square error (RMSE) was calculated Fig- ures 3.5–3.7 give the comparative curves in X, Y and θ directions simulated by 100- time Monte Carlo tests We see that EKF has the largest error while the PO-EKF and LEKF introduce equivalent small error

EKF PO-EKF LEKF True value

Figure 3.4: The EKF, LEKF, PO-EKF, and true trajectories in the motion plane

Figure 3.5: RMSE of the EKF, LEKF,

PO-EKF estimation and the true trajectories in X direction

Figure 3.6: RMSE between the EKF,

LEKF, PO-EKF and the true trajectories in Y direction

LEKF, PO-EKF and the true trajectories in orientation

In addition to the accuracy, the computational burden is also considered Stand- ard MATLAB functions such as tic, toc and flops [24] were used to calculate the amount of floating point operations and execution time of the filters Table 3.1 shows the result scaled with respect to the EKF It shows that the PO-EKF is around two times higher than the EKF, but hundred times smaller than the LEKF

Table 3.1: Normalized computational burden of filters

Parameter EKF LEKF PO-EKF

Figure 3.8: The EKF, LEKF, PO-EKF, and true trajectories of the robot in the motion plane

LEKF, PO-EKF and the true trajectories in X direction

LEKF, PO-EKF and the true trajectories in Y orientation

LEKF, PO-EKF and the true trajectories in orientation

In the second simulation, an extreme scenario is considered in which the time delay is between 800ms and 1500ms, the out-of-order rate is 15%, and the loss rate is 10% The robot follows a sinusoidal path The LEKF uses a finite buffer with 50 slots The purpose of the finite buffer is to reduce the computational burden of the LEKF under the assumption that the time delay is bounded Figure 3.8 presents the estimate and true trajectories in the motion plan It shows that the EKF introduces large drifts at curvature positions whereas the PO-EKF and LEKF still present good tracking performance This evaluation is clearer when comparing the RMSEs of algorithms as shown in figures 3.9–3.11 Table 3.2 shows the computational burden of the filters Though the LEKF reduces the computation (due to the finite buffer), it is still high compared to the PO-EKF

Table 3.2: Normalized computational burden of filters

Parameter EKF LEKF PO-EKF

It is concluded from the simulations that the EKF, which was not designed to deal with network induced problems, is seriously affected by the network On the other hand, the PO-EKF and LEKF both introduces good performance, but the PO-EKF is less computationally demanding.

Experiments

Experiments have been conducted to evaluate the performance of the PO-EKF The hardware for experiments is the NRS described in section 2.3 The software is written in Microsoft Visual C++ [43]

In the experiments, two network configurations are considered The first one, called local configuration, is setup within the local internet service providers (ISP) as shown in figure 3.12 The robot is connected to the Internet via the 3G network of the ISP Viettel At the remote site, a control computer with the localization algorithms installed is connected to the Internet through the ADSL network of the ISP VNPT The second configuration, called VPN configuration, is setup as shown in figure 3.13 In this configuration, the robot and control computer are connected to remote servers located at the United State The connection is established through the virtual private network (VPN) The purpose these two configurations is to capture as closely as possible the good and bad conditions of the network

Figure 3.12: Experimental configuration with local Internet service providers

Figure 3.13: Experimental configuration with VPN connections to the servers located at the United State

During experiments, the robot is controlled in the manual mode to follow predefined paths Its position is measured relatively by the optical encoders while its orientation is measured absolutely by the compass sensor The localization is then performed at two sites: the robot with direct input and measurement data, and the remote controller with delayed data Due to the fact that every attempt to measure the real trajectories subjects to inevitable measurement errors, the determination of true trajectories in experiments is impossible Thus, the estimation using the EKF with no-delay data will be used as the reference for the comparison and evaluation

In the experiment with local configuration, the network parameters were measured as follows: the time delay was between 300ms and 500ms; the out-of-order rate was 2.4%; and the loss rate was 1.3% (table 3.3) Figure 3.14(a) shows the estimate trajectories in the motion plane Figure 3.14(b)–(d) present errors of the EKF, LEKF, and PO-EKF trajectories in X, Y, and θ directions in which the reference is the no- delay estimation It shows that the PO-EKF introduces better accuracy than the EKF and the same accuracy as the LEKF at steady state This result is encouraging as the LEKF was proven to be optimal in dealing with network induced problems We also see that the estimation error (approximately 10 cm) in this experiment is higher than the error in simulations (approximately 5 cm) This difference can be explained by the influence of unexpected factors to the system such as the wheel slip, robot dynamic and non-Gaussian noise component Even we tried to minimize those factors during the calibration process, they are not our main concern in this study In addition, the accuracy of 10 cm is sufficient for our system which requires the accuracy of 30 cm for the functioning operation of control and navigation tasks

Figure 3.14: Comparison between the EKF, LEKF and PO-EKF in local configuration: (a) Trajectories in the motion plan; (b) Errors in X direction; (c) Errors in Y direction; (d) Errors in orientation.

In the experiment with the VPN configuration, the network parameters were measured as follows: the average time delay was 600ms and 800ms; the out-of- order rate was 2.9%; and the loss rate was 1.4% (table 3.3) The estimation results are shown in figure 3.15 They are consistent with the previous results in term of accuracy

From the results, it is able to observe that the PO-EKF and LEKF introduce relatively high estimation errors at initial steps This can be explained by the lack of feedback measurements at those steps due to the network delay During this time, the filters just perform the open-loop estimation by propagating the time update equations After some initial steps, the delayed measurements arrive The filters start to adjust the estimation and finally reach the steady state

Table 3.3: Network parameters during experiments Parameter Local configuration VPN configuration

Time delay 300ms – 500ms 600ms – 800ms

Figure 3.15: Comparison between the EKF, LEKF and PO-EKF in VPN configuration: (a) Trajectories in the motion plan; (b) Errors in X direction; (c) Errors in Y direction; (d) Errors in orientation.

Discussion

In the experiments, we recognize that the rate of message loss and out-of-order are relatively high compared to typical network performance which is often below 1% [75] The reason comes from the sampling rate In our system, the sampling rate is 100 ms During this time, the system has to complete the sensor update as well as other tasks such as motion control, data store, and network communications Due to the sensor adaptation, timing error, and clock mismatch, those tasks are not always completed in time resulting in missing measurements It is able to reduce this problem by increasing the sampling time However, the efficiency of the filter is also reduced because of the slow update of measurements

(b) Figure 3.16: Estimate by the PO-EKF with uniform distribution noise: (a) RMSE in X and Y directions; (b) RMSE in orientation

(b) Figure 3.17: Estimate by the PO-EKF with non-zero mean noise: (a) RMSE in X and Y directions; (b) RMSE in orientation

We have also evaluated the performance of the PO-EKF with non-Gaussian and non-zero mean noises Figure 3.16 shows the estimation result with the uniform distribution noise It shows that the accuracy is similar to the case with Gaussian noise (figures 3.5–3.7) Figure 3.17 shows the estimation result with the noise having the mean of 20 cm In this case, the accuracy is reduced comparing to the estimate with zero-mean noise (figures 3.5–3.7) It is concluded that the PO-EKF still works with different types of noise but the accuracy may be slightly affected

Finally, it is emphasize that our main concern in this study is to deal with the network induced problems From this view, the better accuracy than the EKF, which was not designed for networked system, the same accuracy as the LEKF, which is optimal for networked system, and the low computational demanding compared to the LEKF confirms the efficiency of the PO-EKF.

Conclusion

The localization of NRSs faces challenges caused by the communications network

In order to deal with those problems, we propose a new filter called the PO-EKF

This filter combines knowledge of the system kinematics, network state, and feedback measurements to give an optimal estimate in term of minimizing the mean squared errors The optimality of the combination is theoretically proven A number of simulations and experiments have been conducted The result confirms that the proposed algorithm gives accurate estimate while not computationally demanding

The results of this chapter are published in [12, 13].

Motion Control Using Lyapunov Stability Theory and Predictive

Introduction

The stable movement from one point to another is essential for the efficient operation of a control system and is basic for the development of real-world applications

In non-networked robot systems, a number of studies have been introduced and the problem of stabilization control has been solved in both theoretic and experimental aspects [2, 16, 40] NRSs however have differences The occurrence of the communications network influences the accuracy of the state estimation and control so that directly applying previous control methods is no longer practical

As discussed in section 1.3.2, some new approaches have been proposed to stabilize the NRS Wargui used a multistep prediction mechanism to predict the state that will be used for the controller [16] Luck used a time buffer which was longer than the worst delay to make the system to be time-invariant [47] Xi and Tarn proposed an event-based (non-time) control scheme to reduce the impact of time delay

[89] Kosuge and Murayama introduced the notion of virtual-time delay to keep the apparent time-delay constant [42] Nevertheless, those works mainly focus on the time delay, hardly address other issues such as the out-of-order delivery

In this study, we deal with the problem of stabilization of NRS under the influence of three network parameters including the delay, loss, and out-of-order Our approach is the development of a state estimator that is able to predict the robot’s pose several steps ahead so that the control laws designed for non-networked robot system can be used to stabilize the NRS.

Problem formulation

Consider the control scenario shown in figure 4.1, with an arbitrary starting position and orientation of the robot and a predefined goal position and orientation Let the difference between the present pose ( , , )x y θ and the final pose ( , , )x y 2 2 θ 2 given in the robot reference frame { , , }X Y R R θ R be the error vector, e=(x 2 −x y, 2 −y,θ θ 2 − ) T The task of the controller layout is to find a control constraint, if it exists, of the tangent and angular velocities so that the error e is driven toward zero: lim ( ) 0 t t

Figure 4.1: The goal of the controller

In order to solve this problem, control characteristics of the robot system need be discussed as prerequisite conditions

In this study, the kinematics of the robot is considered instead of its dynamics

There are two reasons to clarify First, the kinematics is sufficient for the current configuration of our robot system Specifically, the robot with the weight of 25 kg moving at a moderate speed (0.5 m/s on average) does not introduce much dynamic properties Another reason is that the main concern of this work is to dealing with problems caused by communications network The ignorance of dynamics stands out the network properties while simplifies mathematic burdens

The controllability property is required in order to stabilize a system As our system is nonlinear, the controllability is addressed using the Hermann and Krener theory

[32] which can be summarized as follows:

C = [ g , ,g , [g , g ], [ad ,g ], , [f,g ], , [ad ,g ], ] (4.2) has rank n where n is the rank of x and [g , g ] and i j [ad ,g ] are Lie brackets gi k j From the kinematic equation (2.6), the system can be rewritten as:

(4.5) and thus sin 0 cos cos 0 sin

C has rank 3 everywhere so the system is controllable

In order to use the feedback measurement in the system, the observability need be assured According to Hermann and Krener [32], the system is locally observable at x 0 if there exists a neighborhood of x 0 such that every x in that neighborhood other than x 0 is distinguishable from x 0 A test for local observability is that:

∂ x x x u x u x (4.7) must have rank n, where n is the rank of x and

  x u (4.8) where L h n f ( ) 1 is n th -order Lie derivative, h is the measurement function, and px1 is the dimension of the output z

From the kinematic equation (2.6) and the measurement equation (2.13), we have:

O has rank 3 everywhere so the system is observable

From equation (2.6), one degree of freedom of the robot is constrained by: sin cos 0 x θ −y θ = (4.11)

Consequently, the system is nonholonomic (a system whose state depends on the path taken to achieve it) According to the work of Brockett [11], Cartesian state- space representation of the robot cannot be asymptotically stabilized via smooth and time invariant feedback laws We therefore define a new coordinate system shown in figure 4.2

Figure 4.2: The robot poses and navigation variables

The new coordinate system consists of three parameters (ρ,a,φ) called navigation variables Let OXY and O X Y 2 2 2 be the local coordinate frames attached to the present and final poses of the robot, respectively, ρ is then defined as the distance between O and O2, φ is the angular made by the vector connecting O and O2 and the vector connecting O2 and x 2, and α is the angular made by the vector connecting O and O2 and the vector connecting O and x 1:

Without lost of generality, we assume that the goal desired pose of the robot is

( , , ) (0,0,0)x y θ = which can also be expressed by ( , , ) (0,0,0)ρ a φ 2 2 2 = (This can be done by setting the origin of the global coordinate frame at the final position)

The kinematic equation (2.6) is then written in the navigation variables domain (ρ,a,φ) as: cos sin sin v v v ρ a a ω a ρ φ a ρ

The goal now is to establish smooth control laws that drive the navigation variables (ρ,a,φ) toward zero

Our approach consists of two steps First, control laws that stabilize the non- networked robot system are derived Then, a predictive filter is introduced to extend those control laws to the NRS.

Stabilization of non-networked robot system

The Lyapunov technique is employed in this study to stabilize the non-networked robot system This technique was proven to be simple and effective [2] Consider the robot with the kinematic model described in equation (4.13) We define the Lya- punov function in the positive definite quadratic form as follows:

Its time derivative V along equation (4.13) is given by:

Let the tangent velocity has the form:

( cos ) ; 0 v= γ a ρ γ > (4.16) The first term of equation (4.15), corresponding to V 1 , becomes:

V = − γ a ρ ≤ (4.17) and the second term, corresponding to V 2 , becomes:

V = −aω γ a+ +hφ a a (4.18) Setting the angular velocity ω to the form: cos sin (a a h ) ω λa γ a φ

According to the Lyapunov stability theory, equation (4.21) proves that by choosing the control laws as in equations (4.16) and (4.19), the final state of non-networked robot system (4.13) goes to zero The system is asymptotically stable

As our system operates in the discrete time domain, we discretize the kinematic model (4.13) and the control laws (4.16) and (4.19) by performing the Taylor expansion and ignoring high order terms Equations (4.13), (4.16), and (4.19) can respectively be rewritten as:

These approximations are valid provided that the sampling period is sufficiently small as discussed in equation (2.9).

Stabilization of NRS

Consider the NRS described in equation (2.15) Due to the network, the system is time-varying in which the control input at time k would not reach the actuator until time k+n whereas the measurement at time k actually reflects the system state at time k-m Consequently, the control laws (4.23) no longer assure the asymptotic convergence of the system

Figure 4.3: NRS with the presence of a state estimator

Our approach for this problem is the development of a state estimator that is able to predict the system state at time k+n based on the delayed or out-of-order measurement taken at time k-m This prediction value, ˆ(x k n k m+ | − ), will be used at the controller to generate the control signal u k n + according to equation (4.23)

This control signal will arrive the actuator at time k+n as expected to stabilize the system (figure 4.3) This approach is similar to [88] The difference is the structure of the state estimator We employ the PO-EKF with some modifications

The first modification is the time update equation (3.45) As the controller sends the input signal u k+n at time k, equation (3.45) is rewritten as:

The data update equation (3.46) remains unchanged as it bases on the measurement at time k-m

The second modification is the supplementary of an extrapolated phase to predict the system state at time k+n As there is no additional measurement, this phase is an open-loop propagation:

The state estimator for the stabilization controller can be realized as follows:

• The time update equations at the prediction phase:

• The data update equations at the correction phase:

• The predictive equation at the extrapolated phase:

We call this filter predictive filter.

Simulations

In order to evaluate the proposed control algorithm, simulations have been conducted in MATLAB Its setup and results are presented as follows

4.5.1 Simulation setup Implementation of the network and robot models

Basically, parameters for simulations including the robot model, network state, and PO-EKF are set similarly to simulations in section 3.5.1 However, the trajectory of the robot is not pre-determined Instead, only the initial and final poses of the robot are set The input velocities are then generated based on equation (4.23) at each step to navigate the robot to the final pose In MATLAB, the generation of input velocities is carried out as follows:

% Input tangential and angular velocities u_theory(k) = gama*cos(alpha(k))*ro(k); w_theory(k) = m*alpha(k) + gama*cos(alpha(k))*sin(alpha(k)) /alpha(k)*(alpha(k)+h*phi(k));

% Real velocities (noises included) set to the robot kinematic u(k) = u_theory(k) + r_wheel/2*(PNoise_R(k)+PNoise_L(k)); w(k) = w_theory(k) + r_wheel/b_wheel*(PNoise_R(k)-PNoise_L(k)); where the robot pose is defined in the navigation variables (ρ,a,φ) The conversion from the Cartesian coordinate to the navigation variables is carried out in MATLAB based on equation (4.12) as:

% Cartesian coordinate to navigation variables ro(k+1) = sqrt((x_target-pos_est_X(k+1))^2 +(y_target-pos_est_Y(k+1))^2); phi(k+1) = atan2(y_target-pos_est_Y(k+1), x_target-pos_est_X(k+1))-theta_target; alpha(k+1) = phi(k+1)-pos_est_T(k+1)+theta_target;

As the input velocities are generated online, the shape of robot trajectories may vary depending on the initial and final poses as well as the control parameter (λ, h, γ)

(equation (4.23)) In simulations, the initial pose can be chosen arbitrarily, for example (-4, -3, 90 0 ), but the final pose is always set to zeros Parameters for the controller are chosen as: λ=6, h=1, γ=3 Details of this choice is discussed in section

In order to evaluate the validity of the proposed approach, three simulations have been carried out The first simulation verifies the asymptotic stable of the control law (4.23) for non-networked robot system In this simulation, the network and predictive filter are not implemented Thus, the robot operates as in classical systems

In the second simulation, the network module is added, but the predictive filter is not executed The purpose is to verify if the control laws (4.23) itself can stabilize the NRS

In the last simulation, the predictive filter is executed to provide augmented feedback measurements This simulation evaluates the validity of the proposed approach in stabilizing NRS

The first simulation verifies the asymptotic stable of the control law (4.23) in non- networked robot system Figures 4.4 and 4.5 present the control results with the initial pose (-4, -3, 90 0 ) and the final pose (0, 0, 0 0 ) The (x, y) coordinate and θ orientation go toward the final state (0, 0, 0 0 ) while the tangent and angular velocities go toward zeros Thus, the system is asymptotically stable

(b) Figure 4.4: Stable control of the non-networked robot system: (a) Trajectory of the robot in the motion plane; (b) Variation of the direction of the robot

(b) Figure 4.5: Velocities during the stable control of the non-networked robot system:

The second simulation applies the same stable control laws to the NRS Simula- tion results with 500ms network delay, 5% out-of-order, and 1% message loss are shown in figures 4.6 and 4.7 The robot’s pose (x, y, θ) does not go to (0, 0, 0) The angular speed goes to -1.5 rad instead of 0 rad as expected The system is therefore not asymptotically stable

(b) Figure 4.6: Stable control of the NRS without using the predictive filter: (a) Tra- jectory of the robot in the motion plane; (b) Variation of the direction of the robot

(b) Figure 4.7: Velocities of the robot during the stable control without using the predictive filter: (a) Tangent velocity; (b) Angular velocity

Figures 4.8 and 4.9 present the control results with the use of the predictive filter It is able to see that the robot’s pose and velocities go to zero The system is asymptotically stable again This result confirms the success of the predictive filter in ensuring the stability of the NRS

(b) Figure 4.8: Stable control of the NRS with the use of the predictive filter: (a) Tra- jectory of the robot in the motion plane; (b) Variation of the direction of the robot

(b) Figure 4.9: Velocities of the robot during the stable control with the use of the predictive filter: (a) Tangent velocity; (b) Angular velocity.

Experiments

Experiments have been conducted to validate the proposed approach The NRS is the system described in section 2.3 The experimental environment was a rectangular shaped flat-wall environment constructed from several wooden plates surrounded by a cement wall (figure 4.10)

Figure 4.10: Setup of the experimental environment

In experiments, the controller was required to respectively navigate the robot from starting points (-4,-4,0 0 ), (-4,-4,45 0 ), and (-4,-4,90 0 ) to reach the destination (0,0,0 0 ) The parameters of the controller were chosen as: λ 0, h=3, γ0 Figure 4.11 describes the trajectories of the robot Figure 4.12 shows the tangent and angular velocities of the robot during the operation They converges to zeros as the robot reaches the destination In another word, the system is asymptotically stable

O rient at ion ( degr ee)

(b) Figure 4.11: Stable control of NRS with the use of the predictive filter: (a) Trajectory of the robot in the motion plane; (b) Variation of the direction of the robot

Tangent v el oc ity (m /s ) 0 degree 45 degree

A ngul ar v el oc ity (r ad/ s)

Time (s) A ngul ar v el oc ity (r ad/ s) 0 degree

(b) Figure 4.12: Velocities of the robot during the stable control with the use of the predictive filter: (a) Tangent velocity; (b) Angular velocity.

Discussion

We have carried out many experiments with different initial positions, network configurations, and control parameters (λ, h, γ) At all time, the robot succeeded in reaching the destination However, we also recognize that the control parameters (λ, h, γ) play an important role in determining the way the robot reaches the destination

For example, the high values of λ and γ speed up the goal reaching process but introduce more fluctuation to the trajectory while the high value of h implies faster change in robot’s direction Figure 4.13 compares trajectories of the robot during the stable control from point (-4,-4,0 0 ) to point (0,0,0 0 ) with different control parameters It is recognizable that the configuration with λ 0, h=3, γ0 gives the best result in term of travelling distance and time response It suggests that tuning these parameters need be carefully taken into account when implementing practical systems as they can significantly enhance the control performance

Figure 4.13: Trajectories of the robot during the stable control with different control parameters

Another role of the control parameter (λ, h, γ) is to determine the range of the input speed In our system, the robot speed ranges from -1.1 m/s to 1.1 m/s This range can be complied by setting λ 0, h=3, γ0 as shown in figure 4.12 Cur- rently, the choice of those parameters is carried out manually In future work, adaptive algorithms such as neural network can be used to automate the process

The next discussion is about the acceptable error of the system at the destination position More specifically, we stated from the problem formulation that the goal of the controller is to drive the error e between the present and final poses toward zero as the time goes infinite: lim ( ) 0 t t

→∞ e = It means that it may take a very long time for the robot to reach the destination exactly In practice, this condition can be relaxed as each control task often requires a certain accuracy For example, the time for our robot to reach the destination (0,0,0 0 ) from point (-4,-4,0 0 ) with the accuracy of 1cm (not include the estimate error) is 75 seconds If the acceptable error is 5cm, the goal reaching time reduces to 15 seconds In conclusion, there is a trade-off between the accuracy and time response Depending on the control requirement, a proper accuracy can be chosen so that the time response is optimal

Finally, it is noted that our approach to stabilize the NRS is not a mathematical approach Instead, it bases on our logical analysis of the NRS and the result published in [88] Ideally, if the estimation is 100% accurate, the robot’s pose will converge to zero as proven in section 4.3 In practice, the robot will reach the destination with possible error depending on the accuracy of the estimate As presented in chapter 3, the accuracy of the PO-EKF was proven to meet the system requirement

When extending to the predictive filter, the extrapolated phase will add extra error

This error is accumulated over time and can become worse if the time delay grows unboundedly To avoid this, a time-out value is set It is the maximum delay at which the estimate error still meets the system requirement The choice of the time- out value is carried out by evaluating the performance of the predictive filter similarly to the evaluation of the PO-EKF In our system, the time-out value is set to 5 seconds This value ensures that the estimation error is within the required range (figure 4.14) In addition, a delay longer than 5 seconds over the Internet would indicate a network congestion or interruption which stops the NRS from functioning operation

R M S E in O rient at ion ( rad)

(b) Figure 4.14: RMSE of the predictive filter with the time-out value set to 5 seconds: (a) RMSE in X and Y directions; (b) RMSE in orientation.

Conclusion

Stabilization control of NRS faces uncertain factors caused by the network Our approach for this problem consists of two steps First, the Lyapunov stability theory is employed to derive control laws that stabilize the non-networked robot system

Those control laws are then extended to the NRS by implementing a predictive filter between the sensor and controller The filter compensates influences of the network to acquire accurate estimate of the system state and consequently ensures the convergence of the control laws The structure of the filter includes the PO-EKF and an extrapolation phase Many simulations and experiments have been conducted The results not only prove the validity of the proposed approach but also reveal some characteristics that we have discussed to improve.

The results of this chapter are published in [8, 9].

Navigation Using Behavior-based Model

Introduction

The final goal of most mobile robot systems is the ability to determine its own position in the environment and then drive towards some goal locations to complete given tasks This process is navigation Depending on the application, navigation can be cooperative, autonomous, semi-autonomous or manual A structured and large scale area is suitable for cooperative and autonomous operation while complex and uncertain tasks in hazardous and less structured environments, such as space, nuclear plants, battlefield, surveillance, and underwater often require semi-autonomous and manual control

In general, navigation requires four functioning processes: perception, the robot interprets its sensors to extract meaningful data; localization, the robot determines its position in the environment; cognition, the robot decides how to act to achieve its goals; and motion control, the robot modulates its motor outputs to achieve the desired trajectory (figure 5.1) Each process has been separately discussed in this study: the perception in chapter 2 with the implementation of sensory system, the localization in chapter 4, and the motion control in chapter 5 We now combine those processes together to generate a navigation system

Figure 5.1: General navigation scheme for mobile robot systems

As discussed in section 1.3.3, there are two methods of navigation corresponding to the direct and behavior-based navigation In direct navigation, operators set the primitive force or velocity commands to perform remote control Environmental information and robot states are transmitted in real time for displaying on the operator’s monitor Direct navigation increases control flexibility but it depends on user’s experience so the system may be at risk if unpredictable situations such as moving obstacles are unaware

Behavior-based navigation, on the other hand, uses the concept of designing sets of behaviors that together result in the desired robot motion For instance, moving from room A to room B may first follow the wall, then discover new area, avoid obstacles, and finally reach the goal position During the process, the data communications is running at background Figure 5.2 shows the architecture of this approach In networked robot, behavior-based navigation has advantages in dealing with network induced problems

Follow right/left Discover new area Reach goal position Avoid obstacles Communicate data

Figure 5.2: An architecture of the behavior-based navigation

In this study, the behavior-based navigation is employed with two improve- ments Firstly, a supervisory module is introduced to fuse output of each behavior to give a best possible motion decision Secondly, fuzzy logic is employed to adjust the control parameter so that the system is more adaptive to the network state.

Behavior-based navigation for NRSs

Figure 5.3 shows the architecture of the behavior-based navigation for NRSs It has three behaviors including the user following, obstacle avoidance, and goal reaching, and one supervisory module Details of each module are as follows

Figure 5.3: The architecture of our behavior-based navigation system

The user following behavior translates the high level command of the teleoperator to the low level control signal so that the robot moves accordantly with the teleoperator’s desire This behavior also updates the network state during the operation to tune control signals so that the system is more adaptive For example, if the command is forward and the network latency is high, the robot will move forward at low speed This adaptation is transparent to the operator

Fuzzy logic is chosen to implement the user following behavior Introduced by Lotfi Zadeh in the proposal of fuzzy set theory (1965), fuzzy logic uses the concept of the degrees of truth rather than the crisp logic for sorting and handling data It accepts vague, ambiguous, imprecise, noisy, or missing input information and presents them in form of membership functions to render a definite conclusion based upon an inference process Fuzzy logic offers several unique features for control system such as the robustness again imprecise input, the extensibility to incorporate new sensors, the capability to control nonlinear systems, and the simplicity in implementation With those features, fuzzy logic was employed in many control systems ranging from simple, small, embedded micro-controllers to large, networked, multi-channel data acquisition and control systems

In the user following behavior, the fuzzy logic is implemented through four steps characterized for fuzzy systems: defining the problem, defining the linguistic variables and membership functions, building the fuzzy rules, and defuzzification

The behavior has two inputs: the network delay and command state The range of the delay is chosen from 0 ms to 1000 ms The command state has two values 0 and 1 corresponding to the loss and non-loss of the command Output of the fuzzy system is the adjusting ratio of the robot speed The value is within the range of [0.5 1.0]

The value of 0.5 corresponds to the adjustment under the worst network condition (very high delay or the command is lost) whereas the value of 1.0 corresponds to the adjustment under the best network condition (low delay and no command loss)

B Defining the linguistic variables and membership functions

The input data is converted into linguistic values which are presented as labels in fuzzy sets The network delay is divided into three subsets: Small (S), Medium (M) and Large (L) The command state has two binary values 0 and 1 corresponding to the state of loss (L) and non-loss (NL) The adjusting ratio of the robot speed has three subsets: Low (L), Medium (M) and High (H) The membership functions of above variables are defined in the Gauss shape as in figures 5.4–5.6

Figure 5.4: Membership function of the network delay input

Figure 5.5: Membership function of the command state input

Figure 5.6: Membership function of the adjusting ratio output

Each rule in the fuzzy knowledge base corresponds to a fuzzy relation In this behavior, 6 rules were defined as shown in table 5.1 Each rule is a combination of two inputs and one output as the following statement: “If delay is small and command is non-loss then the adjusting ratio is high”

Table 5.1: Fuzzy rules of the user following behaviors

Rules Delay Command Adjsuting ratio

To get the crisp output value, the centroid defuzzification with the min-max inference was used The formula is given as:

∑ (5.1) where φ is the output value, x i is the i’th domain value, and à(x i ) is the truth membership value of the i’th domain point

The obstacle avoidance behavior tends to avoid collisions with obstacles that are in the vicinity of the robot In the literature, a number of methods have been proposed to avoid obstacles such as the bug algorithm, gradient method and vector field histo- gram [69] In this study, the fuzzy logic is employed with details of the implementation including four steps as follows

The robot is equipped with eight ultrasonic sensors with the operational range from

0 m to 4 m Those sensors are eventually arranged at the robot front so that the scanning area is from 10 0 to 170 0 In order to reduce the complexity of the fuzzy system, the sensors are divided into three groups: right, forward and left The distance used by each group is the minimum value measured by sensors in that group:

Dleft = min (D6, D7, D8), where D i is the distance acquired by the sensor i Those distances are inputs for the obstacle avoidance The outputs are the rotational speed of the left and right wheels

B Defining the linguistic variables and membership functions

Three inputs Dright, Dforward, and Dleft have the same linguistic values with three subsets: Near (N), Medium (M) and Far (F) The outputs ω L and ω R are divided into Low (L), Medium (M) and High (H) The membership functions of the inputs and outputs are defined in figures 5.7 and 5.8

Figure 5.7: The membership functions of input variables D right , D forward , and D left

Figure 5.8: The membership functions of output variables ω L and ω R

Each rule is a combination of inputs and outputs such as the following statement:

“If Left is Far and Forward is Near and Right is Near then ω L is Low and ω R is High” With three inputs and two outputs, we have defined 27 rules in total Some most common rules are shown in table 5.2

Table 5.2: Fuzzy rules of the obstacle avoidance behavior Rules Left Forward Rigth ω L ω R

The output value of speed of each wheel is obtained by using the centroid defuzzification with min-max inference similarly to equation (5.1)

The goal reaching behavior tends to drive the robot from a given initial position to a stationary or moving target position It is equivalent to the stable control problem that we have addressed in chapter 4 Thus, this module uses the control algorithm with the predictive filter that we have presented

The supervisory module determines the priority to execute of each behavior and decides the control signals the actuators shall receive Inputs of the supervision module include the network states (delay, loss, out-of-order) and values of three sonar sensors (front, left, right) Its outputs are the rotational speeds of the left and right wheels The operation of this module is based on fuzzy rules such as “If context then behavior” Table 5.3 shows details of the rules

Table 5.3: Rules for determination of behaviors Rules Left Forward Right Timeout Behavior

Simulations

We have conducted simulations to evaluate the proposed approach In simulations, the robot is navigated by the behavior-based controller to explore a laboratory with narrow space and many obstacles

The simulation setup is conducted via the operating environment, robot and network models, fuzzy modules and simulating scenarios

The environment in which the robot operates is built based on the real working space of our laboratory (figure 5.10) It is a squared room with the size of 20 m x 20 m divided into six areas by four walls There are six obstacles with different shapes corresponding to facilities such as table, chair, or network box In MATLAB, polygonal obstacles and walls are defined by the coordinates of vertices, for instance:

% Rectangular obstacle in operating environment x_vertices = [10 20]; y_vertices = [11 11.5]; x_obstacle_1 = x_vertices([1 1 2 2 1]); y_obstacle_1 = y_vertices([1 2 2 1 1]);

Round obstacles are defined by their centers and radiuses:

% Round obstacle in operating environment xc = 12; yc = 6; r = 1.5; x = r*sin(-pi:0.1*pi:pi) + xc; y = r*cos(-pi:0.1*pi:pi) + yc;

This representation of obstacles enables the integration with the sensory model in- troducing below

The system model including the robot, network, and PO-EKF is implemented similarly as in section 3.5.1 One additional component is the eight sonar sensors using for obstacle avoidance In simulations, each sensor is modeled as a circular sector with the central angle of 10 0 and the radius of 4 m They are equidistantly placed in front of the robot as shown in figure 5.9

Figure 5.9: Configuration of sonar sensors in simulations

In MATLAB, the circular sector of sensor is implemented as:

% Sonar sensor = circular sector with center (a,b),

% radius R, and central angle from angle theta1 to theta2 t = linspace(theta1,theta2); x = R*cos(t) + a; y = R*sin(t) + b;

During the operation, the distance measured by a sensor is defined as the intersection between the circular sector and objects in the environment

The fuzzy modules are implemented using the Fuzzy Logic Toolbox™ distributed with MATLAB Parameters required for each fuzzy module such as linguistic variables, membership functions, and fuzzy rules are defined in section 5.2

In simulations, the robot is set to start at position A and is navigated by the behavior- based controller to reach the target G (figure 5.10) During the operation, different behavior modules will be activated to control the robot depending on the surround- ing environment The navigation is successful if the robot reaches the target without colliding obstacles

During the operation, the robot starts from the position A with the user following behavior At position B, the robot acknowledges a behavior switch and changes to the goal reaching behavior At positions C and E, due to the appearance of obstacles, the robot switches between the goal reaching and the obstacle avoidance behaviors The robot finally reaches the destination at position G

G Goal wall 1 wall 2 wall 3 wall 4

Figure 5.10: Result of the behavior-based navigation

Figure 5.11 presents the network state during the simulation The average time delay is 382ms; the out-of-order delivery is 5%; and the loss rate is 1% Figure 5.12 shows control signals including angular velocities of the left and right wheels sent to the actuator From A to B, the robot follows user commands in term of movement direction such as turn right, turn left, go backward, and go forward However, the user following behavior tunes the speed of the movement to adapt to the network state At C and E, the sensory data indicate the appearance of obstacle Control signals indicate the switches between the goal reaching and obstacle avoidance at these points Those results confirmed the functioning operation of the navigation model

Delay Loss Out-of-order

Figure 5.11: Network state during navigation

A ngul ar v el oc ity (r ad/ s) ω L ω R

Figure 5.12: Angular velocities of the left and right wheels during navigation.

Experiments

Experiments have been implemented to remotely guide the robot from the starting to the objective points via the Internet In case of obstacle appearance, the robot avoids obstacles and continues to move to the target The experimental environment is the hall of G2 building inside the campus (figure 5.13)

Figure 5.13: Experimental setup with the start position, target position, and obstacles

During the operation, the robot starts from the position A with the user following behavior At position D, the robot acknowledges a behavior switch and changes to the goal reaching behavior At positions B, E, G and I, the robot switches between the goal reaching and the obstacle avoidance behaviors due to the appearance of obstacles The robot finally reaches the goal destination at position G (figure 5.14)

Figure 5.14: Result of the behavior-based navigation

Analysis of the user following behavior: Figure 5.15 presents the network state during the user following process from position A to D The average time delay is 482ms; the out-of-order delivery is 2.5%; and the loss rate is 1.1% Figure 5.16 shows the angular velocities of the left and right wheels sent to the actuator It is able to recognize that the robot follows user commands in term of movement direction such as turn right, turn left, go backward, and go forward However, the speed of the movement is tuned to adapt to the network state Especially, at A1 and C1, a sudden long delay appears and consequently the fuzzy controller significantly reduces the velocities of the robot Those adjustments ensure the stable operation of the navigation system

Delay Loss Out-of-order

Figure 5.15: Network state during the user following process

A ngul ar v el oc ity (r ad/ s) A

Figure 5.16: Angular velocities of the left and right wheels during the user- following process

Analysis of the obstacle avoidance behavior: In experimental setup, we put three obstacles on the moving path of the robot Figure 5.17 shows the distance value measured by the front sonar of the robot It indicates that the robot reaches those obstacles at positions B, E, and G and successfully avoids them In addition, the robot is also able to avoid a moving object when a student suddenly goes toward the robot (figures 5.14 and 5.17)

Figure 5.17: Distance to objects measured by the front sonar during navigation

Analysis of the goal reaching behavior: The goal reaching behavior is activated at position D Figure 5.18 presents the velocities of the left and right wheels during the whole navigation process It starts with high velocities and steadily reduces to zero as the robot reaches destination Those results are in consistence with characteristics of the stable controller discussed in chapter 4

From the analyses, it is able to confirm the functioning operation each behavior and the whole model Figure 5.19 shows a sequence of the snapshots of the robot during the navigation

Figure 5.18: Angular velocities of the left and right wheels during navigation

Figure 5.19: A sequence of images showing the motion of robot during the navigation.

Conclusion

A behavior-based navigation approach has been proposed in which a complex navigation task is divided into individual behavior Each behavior deals with different aspect of the network and environment The localization, stabilization control, and fuzzy logic were employed to design and implement the behavior The coordinate of behaviors is handled by a supervision layer It determines the priorities of behaviors based on the context and then blending the output of each behavior to generate a crisp output

The results of this chapter are published in [6, 7, 11].

Conclusion

Though NRS is a young field of robotics, it has been actively employed in industry, medicine, education, services, and many other applications In Vietnam, NRS has gained the research interest in recent years and is expecting to yield new way of interaction to deal with urgent problems such as transportation and surveillance This thesis studies some fundamental problems of NRS including the localization, stability control, and navigation The goal is to realize new and effective algorithms con- tributing to the development of NRSs

In order to achieve the goal, the system model is first defined and formulated

The network is modeled as a module between the process and controller which delivers input signals and observation measurements with possible delay, loss, and out-of-order The delay is assumed to be random, but measurable at each sampling time; the out-of-order is considered as a long delay; and the packet loss is modeled as a binary random variable From those assumptions, a discrete time-varying state- space model of the NRS has been derived and employed as basis for the development of algorithms

A real NRS system has been developed to serve experiments It consists of a mobile robot connected to a remote controller via the Internet The robot is the type with two wheels and differential drive It contains basic components for sensing and navigation including drive motors for motion control, sonar ranging sensors for obstacle avoidance, compass and GPS sensors for heading and global positioning, and laser range finder (LRF) and vision system for mapping and navigation The remote controller contains a joystick as input device and a graphic user interface with control algorithms running at background The data communications is carried out by a multi-protocol model This model utilizes the TCP for the administrative data, UDP for the control signals, and RTP for the live video

The problem of localization is addressed from the optimal filtering approach

The Kalman filter’s theory is applied to derive a new filter called past observation- based extended Kalman filter (PO-EKF) This filter combines knowledge of the robot kinematics, network state, and feedback measurements to give an optimal estimate of the robot’s pose A relevance factor that reflects the relevance of past measurement to the present is derived to compensate the influence of network The simulation and experiment results confirms the accuracy and computational efficiency of the filter

The stabilization control is investigated based on the combination between the Lyapunov technique and the predictive filter First, the Lyapunov stability theory is employed to derive control laws that stabilizes the non-networked robot system

Those control laws are then extended to the NRS by implementing a predictive filter between the sensor and controller The filter compensates the influence of the network to acquire an accurate estimate of the system state and consequently ensures the convergence of the control laws The structure of filter includes the PO-EKF plus a time-based extrapolated phase Simulations and experiments were implemented to verify the validity of the control scheme

Finally, the results of the localization and stabilization control are integrated in- to a behavior-based model to deal with the navigation problem The model divides a complex task into individual behaviors Each behavior deals with a different aspect of the network and environment A supervision layer is used to handle the coordination between behaviors It determines the priorities of behaviors based on sensory inputs and then blends their output to give a final decision Simulations and experiments were conducted to validate the applicability of the proposed approach

The main contributions of this study are as follows:

• Development of a unified state-space representation of the NRS under the influence of network delay, message loss, and out-of-order delivery This representation has been adopted to deal with fundamental problems of NRSs A real NRS was developed as the platform for experiments and evaluations A multi- protocol model was proposed for the data communications between components of the NRS The model utilizes advantages of individual transport protocols in delivering certain types of communications data to enhance the communications performance These results were published in [1, 2, 3, 4, 5, 10]

• A new optimal filter namely the PO-EKF was proposed for the problem of state estimation and localization of NRSs The filter can deal with the mixed uncertainties of network delay, message loss, and out-of-order delivery The optimality of the filter in term of minimizing the mean square error was theoretically proven The expansion of the filter to non-linear NRSs was derived A number of simulations, comparisons, and experiments were conducted The results confirmed the accuracy, computational efficiency, and implemental capability of the filter These results were published in [12, 13]

• A control algorithm to stabilize the NRS was proposed It basically based on the approach in [88], but a new predictive filter was introduced to improve the accuracy and extend the functionality of the controller to deal with not only the network induced delay but also the message loss and out-of-order These results were published in [8, 9]

• Development of a behavior-based navigation model to navigate the networked robot in unknown environments Fuzzy logic was employed to increase the adaptation of system to the network Simulations and experiments in various environments proved the efficiency of the proposed model These results were published in [6, 7, 11]

Though this study presents significant results on NRSs, expanding those results to systems with multi networked robots would be an interesting problem In this configuration, sensory information does not assist specific robots but shares among them The localization and control algorithms need abilities to handle the heterogeneous data supplied from different sources with the synchronization and sampling mismatches The algorithms also have to be scalable to work with the partial rather than the complete set of information An initial approach for this problem could the development of a sensor fusion algorithm using multiple sampling rate

List of Publications Related to This Thesis

1 Trần Quang Vinh, Phùng Mạnh Dương, Trần Hiếu (2005), “Giám sát và điều khiển robot di động qua mạng LAN vô tuyến và Internet”, Tạp chí khoa học Tự nhiên và Công nghệ, Đại học Quốc gia Hà Nội, Tập 21, số 2, tr.85-91

2 Trần Quang Vinh, Vũ Tuấn Anh, Phùng Mạnh Dương, Trần Hiếu (2006), “Xây dựng robot di động được dẫn đường bằng các cảm biến siêu âm và cảm biến ảnh toàn phương”, Hội nghị Cơ điện tử toàn quốc lần thứ 3 (VCM), tr.153-160

3 Manh Duong Phung, Quang Vinh Tran, Kok Kiong Tan (2010), “Transport Pro- tocols for Internet-based Real-time Systems: A Comparative Analysis,” The

Third International Conference on Communications and Electronics (ICCE)

4 Phùng Mạnh Dương, Quách Công Hoàng, Vũ Xuân Quang, Trần Quang Vinh

(2010), “Điều khiển robot di động qua mạng Internet sử dụng kiến trúc truyền thông CORBA”, The International Conference on Engineering Mechanics and

5 Trần Quang Vinh, Phạm Mạnh Thắng, Phùng Mạnh Dương (2010), “Mạng thông tin điều khiển trong hệ thống tự động hóa tòa nhà”, Tạp chí Khoa học Tự nhiên và Công nghệ, Đại học Quốc gia Hà Nội, Tập 26, số 2, tr.129-140

Tiêu đề	STUDY ON SUPERVISION AND CONTROL OF ROBOT OVER COMPUTER NETWORK
Tác giả	Phung Manh Duong
Người hướng dẫn	Assoc. Prof. Dr Tran Quang Vinh
Trường học	VNU University of Engineering and Technology
Chuyên ngành	Electronics and Telecommunications Technology
Thể loại	Doctoral Thesis
Năm xuất bản	2015
Thành phố	Hanoi

Định dạng
Số trang	146
Dung lượng	5,93 MB