2012 International Conference on Control, Automation and Information Sciences (ICCAIS) Control of Internet-based Robot Systems Using Multi Transport Protocols Manh Duong Phung, Thuan Hoang Tran, Thanh Van Thi Nguyen and Quang Vinh Tran Department of Electronics and Computer Engineering University of Engineering and Technology Vietnam National University, Hanoi avoids to cope with the key issue of control over the Internet, the communication channel, by treating the data transmission between the human operator and the remote robot as a given condition and rarely touches it Abstract—This paper proposes a novel approach to implement communication channels for Internet-based robotic systems The exchange information between the operator and the robot is classified into categories with specific features and requirements Appropriate transport protocols are utilized for each set of data The TCP is for the administrative data; the UDP is for the control signals and the RTP is for the vision data Simulations and experiments show that this approach strengthens advantages of each transport protocol while maintains good performance of the system In the second approach, attempts to deal with the data communication were addressed Liu et al proposed a new transport protocol namely Trinomial [4] It is a rate-based protocol that optimizes the use of available network bandwidth and is able to adapt to the network congestion without affecting very much to the way the user teleoperates the robot An adaptive transport protocol is introduced in [5] This protocol can reconfigure itself to cater for real-time requests In [6], main transport protocols for real-time networked robots including the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), the Trinomial, the Real-Time Network Protocol (RTNP) and the Interactive Real-Time Protocol (IRTP) are simulated and compared It is concluded that each protocol has its strengths and weaknesses so that there is no single protocol can simultaneously adequate for transmitting all types of data of an online robotic system Keywords-Internet robot; transport protocols; mobile robot; robot control; telerobot I INTRODUCTION Real-time control over the Internet is attracting more interest based on its capability to open new types of interaction applications such as virtual laboratory, tele-guidance, telehomecare and disaster surveillance [1]-[6] It is wellrecognized that the most challenge and distinct difficulties with these works are associated with the inevitable Internet transmission delays, delay jitter and non-guaranteed bandwidth There are currently two approaches to deal with these issues In this paper, a multi protocol model for controlling robots over the Internet is introduced The exchange information between the robot and the operator is classified into different categories with specific features and constraints Separate transport protocols including TCP, UDP and RTP are employed to package the data and transmit them over the Internet Recovery and synchronization processes are performed at receiving ends and the data is extracted to display and execute Several simulations and experiments were carried out to evaluate the efficiency and applicability of the proposed model The first focuses on developing advanced remote control algorithms and interface techniques [1]-[3] In [1], a CORBAbased robotic system features the concept of task-level control In this system, instead of step-by-step operating over the Internet, a control command from the user is sent to the robot at a task-level such as “give me the spoon”, “grasp the blue bowl”, “coffee, please”, etc The robot then analyzes the command, makes the path planning, completes the task and returns the result to the user without requiring any additional actions In [2], a virtual environment which is proportional to the real dimension of the laboratory is constructed at the client side Before commands are sent to the server program, they are processed in the virtual environment to predict the upcoming position of the robot Based on it, the system is able to tolerate a certain amount of time delays as well as allow users to experience the interaction with the robot as in the real environment The idea of command filter is introduced in [3] In the system, control commands are stored in a command queue and outputted at each sampling time This combined with a posture estimator enables the system to recover the lost information of control commands caused by the Internet delay and therefore reduce the path error between the real robot and the virtual one Despite several advantages, this approach U.S Government work not protected by U.S copyright The paper is organized as follows Analysis of transport protocols is described in section II The multi-protocol implementation is described in Section III Section IV introduces experiments in the real Internet environment The paper concludes with an evaluation of the system, with respect to its strengths and weaknesses, and with suggestions of possible future developments II TRANSPORT PROTOCOLS FOR INTERNET-BASED ROBOT SYSTEMS Transport protocols are protocols that operate at layer four of the Internet layer model and provide end-to-end communication services for applications such as congestion avoidance, flow control, and multiplexing [12][13] Different 294 transport protocols such as TCP, TICP, ALCAP, and STCP were developed for various purposes of applications [12] Nevertheless, only protocols which are published by the Internet Engineering Task Force (IETF) are standards for the Internet and widely supported all over the world To ensure the proper functioning operation, an Internet application should only use IETF’ protocols and the most commonly ones are the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), and the Real-time Transport Protocol (RTP) These protocols cover over 90% traffic of the Internet [7]-[9] UDP is based on the idea of sending a datagram from a device to another as fast as possible without due consideration of the state of the network This protocol does not maintain a connection between the sender and the receiver, and it does not guarantee that the transmitted data packets will reach the destination as well as the chronological order of the data at the receiving end The main advantage of UDP is the relatively minimized transmission delay and delay jitter achieved under good network conditions It is therefore usually used for realtime data transmission 800 RTP UDP TCP 700 800 500 Throughput (kb/s) Thoughput (Kb/s) 600 1000 400 300 200 100 600 400 TCP UDP 200 RTP 10 15 20 Time (s) 0.3 15 20 0.3 0.25 0.25 Delay (s) 0.2 0.2 Delay (s) 0.15 0.1 RTP TCP UDP 0.05 0 200 400 600 Sequence Number 800 0.15 TCP 0.1 UDP RTP 0.05 1000 0 500 0.4 RTP UDP TCP 0.35 0.3 1000 Sequence Number 1500 2000 0.18 0.16 0.14 0.2 0.12 Jitter (s) 0.25 Jitter (s) 10 Time (s) 0.15 0.1 RTP UDP TCP 0.1 0.08 0.06 0.05 0.04 0 200 400 600 800 0.02 1000 Sequence Number 0 Figure Characteristics of RTP, TCP and UDP in case of no network congestion 500 1000 Sequence Number 1500 Figure Characteristics of RTP, TCP and UDP in case of network congestion 295 2000 and UDP flows to send the traffic to the network and is dropped from the network In addition, the similarity in behavior of UDP and RTP confirms that RTP is developed from the UDP TCP is a more sophisticated protocol which was originally designed for the reliable transmission of static data such as emails and files over low-bandwidth, high-error-rate networks In each transmission session, TCP establishes a virtual connection between the sender and the receiver, performs the acknowledgment of received data packets and implements the retransmission mechanism when necessary TCP can also adapt to the variation of network condition by applying congestion control with slow start, fast recovery, fast retransmit and window-based flow control mechanisms With these features, TCP has been effectively used in the transmission of static data, significantly contributing to the growth of the Internet Nevertheless, employing the TCP for real-time data is hardly appropriate since the timeliness of transmission outweighs the reliability concerns so that the retransmission mechanism is illsuited for real-time data delivery and the strict congestion control mechanism incurs higher delay jitter which rapidly degrades the quality of service (QoS) in a congested network (fig.2) III Hardware configuration for the Internet-based robotic systems often adopts a distributed model with three modules: the robot, the network communication, and the client controller (fig.5) In order to build an appropriate control model, the communication data need to be analyzed A Robot data analysis Various types of information need to be exchanged between the robot and the human operator over the Internet such as credential login data, control commands, synchronous messages, and image data (fig.4) Generally, they can be grouped into three classes: the administrative data, the control signals and the vision data RTP is a relatively new transport protocol designed for delivering real-time multimedia data [9][14] It bases on UDP and has facilities for jitter compensation and detection of outof-sequence arrival in data, and is usually used in conjunction with the RTP Control Protocol (RTCP) While RTP carries the media streams (audio and video), RTCP is used to monitor transmission statistics and information relating to the quality of service The administrative data includes the access control, the user validation, and the configuration data This type of data has small packet size with the bandwidth lower than 10Kbps and is usually once-forall transmission It does not require real-time delivery but critically demands the reliability The control signals, on the other hand, are periodic transmission They include but not limit to control commands, synchronous messages, and sensory measurements This data requires the real-time delivery and consumes the bandwidth from 1Kbps to 100Kbps In order to explore the behavior of protocols in an interactive network environment, simulations have been conducted The widely adopted network simulation tool ns-2, which was developed by Defense Advanced Research Projects Agency (DARPA) through the Virtual InterNetwork Testbed (VINT) project, was used for the simulations [11] Fig.1 shows the simulation results of the scenario in which three sources of traffic corresponding to RTP, UDP, and TCP are connected to a router The router forwards the traffic to a sink through a 1.5Mbps duplex link The RTP and UDP sources send data to the network at 0.5Mbps and the TCP source creates the traffic based on the state of the network (fig.3) The most important and costly information is the vision data It can be used directly as the system feedback as well as the preference for recognizing algorithms The vision data requires the large packet size, periodic transmission, significant bandwidth and real-time delivery Table.1 shows the summary of data classification and transport protocols TABLE I DATA CLASSIFICATION AND CORRESPONSIVE PROTOCOLS Data Group Administrative data (access control, user validation, configuration) Control signals (user commands, ultrasonic ranges, GPS signals, infrared distances) Live Video (camera images) Figure MULTI PROTOCOL MODEL Bandwidth (Kbps) Real-time Demand Feature Transport Protocol – 10 No One time TCP – 100 Yes Periodic UDP 100 – 2000 Yes Periodic RTP Network topology in simulations B Multi-protocol implementation According to the data and transport protocol analysis, it is insufficient for a protocol to handle all the communications of the system In addition, the variety of information means different transfer modes Real-time delivery is required for all data except for the once-for-all administrative data Consequently, in the implementation, the TCP is adopted for According to fig.1, all RTP, UDP and TCP flows share a fair bandwidth of the network and introduce a common transmission delay behavior The network jitter of the TCP flow is however quite large in comparison to RTP and UDP flows Fig.2 shows the behavior of transport protocols in case of network congestion The TCP flow cannot compete with RTP 296 the communications of administrative data: A TCP connection is opened when a teleoperation session is started and is closed once the teleoperation is geared up The RTP protocol is employed for the transmission of information on live scene images The UDP is used for sensory data and robot positions and speeds Since the human operator issues control commands based on his/her personal judgments and decisions, the timing of these control signals is random in nature and is controlled solely by the human operator The transmission frequency of control signals depends on how often the user interacts with the mobile robot Thus, the transmission rate of control signals is largely controlled by the human operator instead of by the transport protocol As a result, we still utilize UDP for control command delivery The usage of UDP is generally acceptable because control commands are small-size packets and it should not jeopardize inter-flow fairness if the human operator does not issue a burst of commands when the network is heavily congested Fig.8 describes the implementation of the multiprotocol communication Figure Hardware configuration of an Internet-based robotic system A Experimental setup The Internet-based robot platform developed for experiments is shown in fig.5 [15] It consists of three components: the mobile robot, the communication channel and the client computer The robot is a Multi-Sensor Smart Robot (MSSR) developed at our laboratory It has basic components for motor control, sensing and navigation, including battery power, drive motors and wheels, position/speed encoders, infrared sensors, integrated sonar ranging sensors, a compass sensor, a global positioning system (GPS), a laser range finder and a visual system Sensing and motor control are managed by a laptop PC placed inside the robot The 3G mobile network is utilized as the communicating bridge between the robot and the Internet A 3G USB device with an internal mobile SIM card is used The USB is plugged into the computer inside the robot and is registered to a mobile phone service provider allowing it to have access to the Internet The interaction devices at the user site simply consist of a personal computer and a joystick The computer is an ASUS notebook computer with 1.5GHz M-Centrino processor, 500Mb RAM and Windows XP operating system The joystick is a 3D Logitech Extreme series with 10 bit resolution in horizontal and vertical axes and 12 functioning buttons Figure Communications in an Internet-based robotic system and the multiprotocol transmision IV EXPERIMENTS To verify the validity and effectiveness of the proposed approach in control, an Internet-based robot system is implemented in which a mobile robot is controlled over the Internet using different transport protocols Figure Remotely navigating the mobile robot around the laboratory B Results To evaluate the performance of the system, we have carried out many experiments Fig.6 shows the setup of the environment that the MSSR moves through The robot is located at the Automatic Control and Robotics Laboratory 297 (ACRs Lab) of the Hanoi University of Engineering and Technology, Vietnam National University, and is controlled over the Internet by an operator who is 12km far away The average speed of the robot is 0.3m/s The goal of the experiments is to remotely guide the MSSR from the starting point Oo to the objective point Od In the experiments, by using the proposed transport protocol communications, the user successfully navigated the MSSR from the point Oo to the point Od via the Internet (fig.6) This result can be verified from the fig.7 and fig.8: an average delay of 43ms and an average jitter of 9ms of all protocols definitely satisfy the stable conditions of this particular setup which can tolerate the delay and jitter up to several seconds It is also noted from RTP in fig.8 that an average jitter of 9ms is much better than the 15ms jitter requirement for normal video streaming [10] Figure A sequence of images showing the motion of robot in a laboratory environment during the tele-operation Fig.9 shows a sequence of the snapshots of the MSSR when it was remotely being guided via the Internet to move from point Oo to point Od in the ACRs Lab We have conducted the experiments at different times of day: morning, noon, afternoon and night, and tried to capture the ‘‘rush hour” of the Internet traffic At all times, the user succeeded in navigating the mobile robot through the ACRs Lab We also compare between protocols by transmitting the video over the TCP (fig 10) The results show that the image quality is much lower than transmitting by RTP In addition, the dropped rate of data packets of TCP is relative high which is 10% if the available bandwidth is 500Kbps and becomes congested if the available bandwidth goes below 200Kbps In case of UDP, the quality is acceptable but the lack of flow control mechanism makes it inappropriate for the video transmission 160 UDP RTP TCP Time delay (ms) 140 120 100 80 60 40 a) b) 20 Figure 10 Images transmitted by TCP (a) and RTP (b) 0 10 20 30 40 50 60 70 Sequence number (x102) CONCLUSIONS 60 In this paper, a novel control approach for controlling Internet-based robot systems, the multi protocols, is proposed Various types of exchanged information in an online mobile robotic system are analyzed and the influence of transport protocols on the performance of system is studied Based on it, appropriate protocols are employed to transmit each class of data The result is that the operator can comfortably control a mobile robot over the Internet with good feedback and low delay 40 REFERENCES Figure Packet delay during the experiment Jitter (ms) V 100 RTP TCP 80 UDP [1] 20 [2] 0 20 40 60 80 100 Sequence Number (x102) [3] Figure Delay jitter during the experiment 298 Songmin Jia and Kunikatsu Takase, Internet-Based Robotic System Using CORBA as Communication Architecture, Journal of Intelligent and Robotic Systems 34: 121–134, 2002 Dawei Wang, Jianqiang Yi, Dongbin Zhao and Guosheng Yang, “Teleoperation System of the Internet-based Omnidirectional Mobile Robot with A Mounted Manipulator,” Proceedings of the 2007 IEEE International Conference on Mechatronics and Automation, August 8, 2007 K K Han, S Kim, Y J Kim and J H Kim, “Internet Control Architecture for Internet-Based Personal Robot”, J Autonomous Robots 10, 135–147, 2001 [4] [5] [6] [7] [8] [9] Peter X Liu, Max Q.-H Meng, Polley R Liu, and Simon X Yang, “An End-to-End Transmission Architecture for the Remote Control of Robots Over IP Networks,” IEEE/ASME transactions on mechatronics, Vol 10, No 5, October 2005 Li Ping, Lu Wenjuan, Sun Zengqi, “Transport layer protocol reconfiguration for network-based robot control system ”, Networking, Sensing and Control, 2005 Proceedings Raul Wirz, Raul Marín, José M Claver, Manuel Ferre, Rafael Aracil, Josep Fernández, “End-to-end congestion control protocols for remote programming of robots using heterogeneous networks: A comparative analysis”, J Robotics and Autonomous Systems 56, 2008 J Postel, RFC 768: “User Datagram Protocol,” 1980 J Postel, RFC 793: “Transmission Control Protocol,” DARPA Internet Program Protocol Specification, 1981 H Schulzrinne, S Casner, R Frederick, V Jacobson, RFC1889: “RTP: A Transport Protocol for Real-Time Applications,” Internet Engineering Task Force, 1996 [10] Mohamed Koubaa and Maurice Gagnaire, “A Performance Study of MPEG-4 Video Streaming in IP Networks”, European contract N° CP53 [11] The Network Simulator-ns-2 [Online]: http://www.isi.edu/nsnam/ns [12] Behrouz Forouzan, “TCP/IP Protocol Suite “, McGraw-Hill Science/Engineering/Math, 4th edition, 2009 [13] Kevin R Fall, W Richard Stevens, “TCP/IP Illustrated”, 2nd edition, Addison-Wesley Professional, 2011 [14] Colin Perkins, “RTP: Audio and Video for the Internet”, AddisonWesley Professional, 2003 [15] P M Duong, T T Hoang, N T T Van, D A Viet and T Q Vinh, “A Novel Platform for Internet-based Mobile Robot Systems”, The 7th IEEE Conference on Industrial Electronics and Applications, Singapore, July 2012 (accepted) 299 ... describes the implementation of the multiprotocol communication Figure Hardware configuration of an Internet-based robotic system A Experimental setup The Internet-based robot platform developed for... Fig.6 shows the setup of the environment that the MSSR moves through The robot is located at the Automatic Control and Robotics Laboratory 297 (ACRs Lab) of the Hanoi University of Engineering and... of transport protocols on the performance of system is studied Based on it, appropriate protocols are employed to transmit each class of data The result is that the operator can comfortably control