DSpace at VNU: Development of a tele-guidance system with fuzzy-based secondary controller

2010 11th Int Conf Control, Automation, Robotics and Vision Singapore, 7-10th December 2010 Development of a Tele-guidance System with Fuzzybased Secondary Controller Manh Duong PHUNG, Thanh Van Thi NGUYEN, Cong Hoang QUACH, Quang Vinh TRAN Department of Electronics and Computer Engineering University of Engineering and Technology Vietnam National University, Hanoi Abstract—Dealing with the uncertainties of Internet characteristics is an important issue that needs being taken into account in developing Internet-based real-time systems In this paper, we present our approach in applying fuzzy logic to develop back-up mechanisms for an Internet-based mobile robot to deal with unwanted network problems such as long delays or network interruptions A tele-guidance application involving the remote control of a mobile robot via the Internet is set up as the context to verify the effectiveness and applicability of the proposed approach Keywords—Data transmission; Internet; distributed control; fuzzy logic; robot navigation; networked robot I INTRODUCTION Since the first Internet-based telerobot appeared in 1994 [3], substantial efforts have been devoted to remotely control real-time systems over the Internet, allowing users to visit museums, tend gardens, navigate undersea, float in blimps, or handle protein crystals via the Internet [1]–[5] Whereas early researches tried to answer the question of how to control a robot through the Internet [3][6][7], recent researches focus on how to control it in real time and dealing with the inevitable Internet transmission delays, delay jitter and non-guaranteed bandwidth when controlling via the Internet becomes easier with the support of embedded Ethernet, on-chip web server, scripting languages, socket programming, etc [8]-[10] Many approaches have been proposed with their strengths and limitations A CORBA-based robotic system featured the concept of task-level control [8] In this system, instead of step-by-step operation over the Internet, a control command from the user is sent to the robot at a task-level such as “give me the spoon”, “grasp the blue bowl”, “coffee, please”, etc The robot then analyzes the command, makes the path planning, completes the task and returns the result to the user without requiring any additional actions This approach reduces the influence of time delays to the system; however, the flexibility is also reduced In [9], a virtual environment which is proportional to the real dimension of the laboratory is constructed in the client side Before commands are sent to the server program, they are processed in the virtual environment to predict the upcoming position of the robot Basing on it, the system can tolerate a certain amount of time delays as well as allowing users to 978-1-4244-7815-6/10/$26.00 ©2010 IEEE experience the interaction with the robot as in the real environment In another approach, Liu et al gave an attempt to deal with the key issue of Internet-based systems, the communication channel, by proposing a new transport protocol namely Trinomial [10] It is a rate-based protocol that optimizes the use of available bandwidth and is able to adapt to the network congestion without affecting very much to the way the user teleoperates the robot This transmission mechanism is a strong solution which uses as much bandwidth as possible, providing smoothness to a bilateral teleoperation via Internet However, it introduces extra time-delay due to the fact that it sets the router buffers to the maximum load [11] In this paper, we propose a novel approach in which a fuzzy logic controller is designed to deal with the Internet-induced uncertainties A tele-guidance application involving the remote control of a mobile robot over the Internet is set up as the basis for the evaluation of reliability and applicability of the fuzzybased approach II TELE-GUIDANCE SYSTEM ARCHITECTURE The analysis to be presented in this paper will be centered around a tele-guidance system described below A Hardware Configuration The tele-guidance system is developed following a distributed model which consists of four stand-alone modules: the mobile robot, the fuzzy-based secondary controller, the central server and the client computer The intercommunication between these modules is via a digital channel Fig.1 shows the hardware configuration of the tele-guidance system The central server, the secondary controller and the client are Intel Pentium IV computers with control software installed The central server has three interfaces: one to link it to the mobile robot to allow the communication of control and feedback data , one to communicate with the fuzzy-based controller and the other interface allows it to process requests from remote users The robot is a commercial Sputnik mobile robot [12] It has basic components for motor control, sensing and navigation, including battery power, drive motors and wheels, position/speed encoders, infrared sensors, integrated sonar ranging sensors and a visual system Sensing and motor control are managed by an on-board digital signal processor (DSP) with independent motor/power and sonar controller boards for 1826 ICARCV2010 Figure 1: System hardware configuration Client Module Fuzzy Module GUI TCP Fuzzy-based Controller Sensor Data UDP Client Requests INTERNET Control Data Sonar Data Server Module TCP UDP Data Processing Sputnik Mobile Robot Control Commands WiRobotAPI Sensor Data Figure 2: System software architecture a versatile operating environment The drive system uses highspeed, high-torque, reversible-DC motors The Sputnik mobile robot provides three sonar sensors One is mounted to the front of the robot and the others are mounted to its left and right side respectively The data of sonar sensors are the basis for fuzzybased navigating algorithms The visual system is detachable and mounted on the head of the Sputnik mobile robot It mainly consists of a MCI3908 color image module with Sharp mini color camera head LZ0P390M The image size can be up to CIF format (353 x 288 pixels) and the operation frame rate is up to 15 fps A Wi-Fi 802.11 wireless module is employed to connect the Sputnik mobile robot to the Internet B Software design The overall system software consists of two subsystems, namely, a client-server application and a backend fuzzy module The front-end involves a client side applet which presents the user with a graphical user interface (GUI) Through the GUI, the user is able to observe the remote environment, access the parameters of the mobile robot, and control it in real time to explore the laboratory over the Internet The fuzzy control software is written in MATLAB [13] This module processes ultrasonic data based on fuzzy algorithm and returns the control commands to a server program running on the central server A brief functional software structure of the platform is shown in Fig.2 1) Client-server applications: For Internet-based applications, using client-server architecture is usually considered as a judicious selection because of its flexibility, usability, scalability, and interoperability in design and development [14] In our system, we chose a client-server architecture which consists of following necessary subsystems: a) Server: A server is a computer program that provides services to other computer programs on the same computer or on another networked computer In our system, the server handles all control requests from client and the fuzzy module It then processes them in conjunction with network status, remote environment parameters, and forwards translated commands to the Sputnik robot The server also retrieves 1827 sensor information about status of the robot and transmits it to clients In short, the server acts as the bridge between the client module, the fuzzy module and the Sputnik mobile robot In the system, the server program was written in Visual Basic 6.0 [15] and the communication between the server and client was established through socket, an abstraction that represents a terminal for communications between processes across a network b) Client: On the client side, the graphical user interface (GUI) is responsible for receiving and interpreting human control commands such as mouse-click events in the control panel, and transmitting them to the server program over the Internet The GUI also displays feedback information about the remote side such as current robot position, ultrasonic ranging values, and live video of remote environment (Fig.10) c) Communication protocol: There are currently two major transport protocols for implementing remote control applications over the Internet: TCP and UDP [16][17] TCP is a reliable protocol for transmitting static data such as files, emails over low-bandwidth, high-error-rate networks UDP is merely a raw protocol that does not guarantee the destination arrival of transmitted data; however, it introduces relatively minimized transmission delay and jitter In our system, TCP is used in communicating over these Internet while UDP is the protocol for local area communication 2) Fuzzy module: : The fuzzy module connects to the server as a special client and supplies the server with following services: • Acquiring data from ultrasonic sensors and passing it to the fuzzy-based controller for creating output commands • B Defining the linguistic variables and the membership functions In this step, we convert the input data into suitable linguistic values which will be viewed as a label of fuzzy sets The distance values ui of ultrasonic sensors are divided in three subsets: Near (N), Medium (M) and Far (F) The deflection angle α has five subsets: Large Negative (LN), Negative (N), Zero (Z), Positive (P), and Large Positive (LP) Figure 3: Membership functions for three distance inputs Figure 4: Membership function for the deflection angle input Sending commands to the server to operate the mobile robot in case network congestions or unwanted obstacles appear With these functions, the fuzzy module plays the role of a secondary controller to supply the system with back-up mechanisms The implementation of the fuzzy-based controller will be described in the next section III IMPLEMENTATION OF FUZZY-BASED CONTROLLER Figure 5: Membership function for the rotation angle output The objective of the fuzzy-based controller in our system is twofold Firstly, it protects the robot by performing the obstacle avoidance during the process of tele-guidance operation; secondly, it navigates the mobile robot to a pre-defined safe point in case of a network interruption The implementation of the fuzzy logic-based controller is divided into four logical steps as follow: A Defining the problem The system has four inputs: the distance values u1, u2, u3 of three ultrasonic sensors and the deflection angle α between the robot-front direction and the safe point-to-robot direction The three sensors scan a range of -600 to +600; the deflection angle α has the range from -1800 to +1800 The outputs of the fuzzy system are the speed of robot and the rotation angle φ that the robot needs rotate Figure 6: Membership function for the speed output The rotation angle φ is divided in two ranges corresponding to the obstacle avoidance and the target finding task For the obstacle avoidance, the subsets include: Left Obstacle (LO), Straight Obstacle (SO), and Right Obstacle (RO) For the target finding, the subsets consist of Large Left (LL), Left (L), 1828 Figure 7: Remotely navigating the mobile robot around the laboratory Figure 9: Robot motion in case of a long-time delay Figure 8: Robot motion in case of a network interruption Figure 11: A sequence of images showing the motion of robot in a laboratory environment during the tele-guidance operation Figure 10: Client Graphical User Interface Straight (S), Right (R), and Large Right (LR) The speed output has Low Speed (LS), Medium Speed (MS) and Fast Speed (FS) The definition of membership functions for these fuzzy sets is described in Fig.3-Fig.6 D Defuzzification To get output values of speed and rotation angle, we used the centroid defuzzification method with min-max inference The formula is as follows: TABLE 1: Common fuzzy rules Rules 10 Sensor F F F M M N N F N F Sensor F F F M M F F F N N Sensor F F F M M F F N N F Target angle Z LN LP N P Z LP LN LP N Rotation angle Z LL LR L R RO LR LL LR L ϕ= Speed ∑ x μ(x ) ∑ μ(x ) i i i FS FS FS MS MS LS LS LS LS LS where xi is the i’th domain value, and µ(xi) is the truth membership value for that domain point IV C Defining the fuzzy rules Each rule in the fuzzy knowledge base corresponds to a fuzzy relation With four inputs and two outputs, it would be computationally expensive to produce 360 rules as well as inefficient to divert the robot from its trajectory for each fired rules In this system, we declared 29 rules for most common situations and each rule is a combination of four inputs and two outputs as the following statement: “If sensor1 is far and sensor2 is far and sensor3 is far and the deflection angle is zero then the rotation angle is zero and the speed is fast” Table shows some rules of our system EXPERIMENTS To verify the validity and effectiveness of proposed fuzzybased mechanisms for the tele-guidance system, we have carried out many experiments Fig.7 shows the setup of the environment that the Sputnik mobile robot moves through The goal of the experiments is to remotely guide the Sputnik mobile robot from the starting point Oo to the objective point Od via the Internet In case of network congestions or interruptions, the robot should be navigated to the safe point Os by the fuzzybased secondary controller In the experiments, by using the designed client GUI (Fig.10), the user successfully navigated the Sputnik mobile robot from point Oo to point Od via the Internet (Fig.7) Fig.11 shows a sequence of the snapshots of the Sputnik mobile robot when it was remotely being guided via the Internet to move from point Oo to point P3 in the Automatic Control and 1829 Robotics Laboratory (ACRs Lab) at the Hanoi University of Engineering and Technology, Vietnam To test the behavior of the system in case of a network interruption, we unplugged the network cable which connects the central server to the Internet during a tele-guidance process As shown in Fig.8, the network interruption appeared when the robot was at P2 position It continued to move to P3 before changing its direction to react to the network problem This matched our implementation which detected the network interruption as the event that the server did not receive synchronous messages from the client for 120 seconds The robot then moved through P4, P5 and finally reach the safe point Os To simulate a long-time-delay event, we forced the client and server programs not to communicate for 90 seconds starting at the time Ts of the experiment Fig.9 shows the motion path of the robot and fig.12 displays the received data from ultrasonic sensors The Ts and Te are the start and end time of the delay event The Td represents the time at which the mobile robot is very near to an obstacle and may result in a collision Due to the network delay, the operator, however, did not recognize this risk and took no action The fuzzy-based controller was then immediately set to high priority and successfully controlled the mobile robot to avoid the obstacle Lab over the Internet without worrying about the network problems such as network congestions, network interruptions and long delays ACKNOWLEDGMENT This work was supported by Vietnam's National Foundation for Science and Technology Development (NAFOSTED) REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] Figure 12: Ultrasonic data in case of a long-time-delay V CONCLUSION In this paper, we propose a novel approach to deal with the uncertainties of Internet characteristics for Internet-based realtime systems: the use of the fuzzy-based controller as a secondary control The implementation of this controller is described in details The hardware and software design of a tele-guidance system is also represented as the context to verify the proposed approach The experiments show that the user can entertainingly control the mobile robot to explore the ACRs [12] [13] [14] [15] [16] [17] 1830 K Golberg and M Mascha, “Desktop teleoperation via the World Wide Web,” Proceedings of the 1995 IEEE International Conference on Robotics and Automation, May 1995, pp.654-659 R C Luo and T M Chen, “Development of a multi-behavior based mobile robot for remote supervisory control through the Internet,” IEEE/ASME Trans Mechatronics, vol 5, no 4, pp 376–385, Dec 2000 Beyond Webcams: An Introduction to Online Robots, K Goldberg and R Siegwart, Eds Cambridge, MA: MIT Press, 2002 K Taylor and B Dalton, “Issues in Internet telerobotics,” in Proc Int Conf Field and Service Robotics, Canberra, Australia, 1997, pp 151– 157 C Sayers, Remote Control Robotics New York: Springer Verlag, 1998 http://cwis.usc.edu/dept/garden/ Eric Paulos and John Canny, “Delivering real reality to the World Wide Web via telerobotics,” Proceedings of the 1996 IEEE International Conference on Robotics and Automation, April 1996, pp.1694-1699 Songmin Jia and Kunikatsu Takase, Internet-Based Robotic System Using CORBA as Communication Architecture, Journal of Intelligent and Robotic Systems 34: 121–134, 2002 Dawei Wang, Jianqiang Yi, Dongbin Zhao and Guosheng Yang, “Teleoperation System of the Internet-based Omnidirectional Mobile Robot with A Mounted Manipulator,” Proceedings of the 2007 IEEE International Conference on Mechatronics and Automation, August 8, 2007 Peter X Liu, Max Q.-H Meng, Polley R Liu, and Simon X Yang, “An End-to-End Transmission Architecture for the Remote Control of Robots Over IP Networks,” IEEE/ASME transactions on mechatronics, Vol 10, No 5, October 2005 Raul Wirza et al., “End-to-end congestion control protocols for remote programming of robots, using heterogeneous networks: A comparative analysis,” Journal of Robotics and Autonomous Systems 56 (2008) 865– 874 http://www.drrobot.com/products/item_downloads/Sputnik_2.pdf Stormy Attaway, “Matlab: A Practical Introduction to Programming and Problem Solving,” Butterworth-Heinemann Publisher, February 2009 G Reese, “Database Programming with JDBC and Java,” Second edtion, O’Reilly & Associates, November 2000 Steve Brown, “Visual Basic Complete,” Sybex Publisher, March 1999 J Postel, RFC 768: “User Datagram Protocol,” 1980 J Postel, RFC 793: “Transmission Control Protocol,” DARPA Internet Program Protocol Specification, 1981  ... the use of the fuzzy-based controller as a secondary control The implementation of this controller is described in details The hardware and software design of a tele-guidance system is also represented... http://www.drrobot.com/products/item_downloads/Sputnik_2.pdf Stormy Attaway, “Matlab: A Practical Introduction to Programming and Problem Solving,” Butterworth-Heinemann Publisher, February 2009 G Reese, “Database Programming with JDBC and Java,” Second... Proceedings of the 1996 IEEE International Conference on Robotics and Automation, April 1996, pp.1694-1699 Songmin Jia and Kunikatsu Takase, Internet-Based Robotic System Using CORBA as Communication Architecture,