668 Industrial Robotics: Theory, Modelling and Control ture, material reflection characteristics, etc.). The behaviour of the Tripod model is controlled by Behaviour nodes, which contain user-defined control codes and state variables. Sensor data processing can be embedded into the codes for remote monitoring. Once applied to a TransformGroup node, the so- defined behaviour control affects all the descending nodes. In this example, the movable objects (X-Table, Y-Table, and Moving Platform) are controlled by us- ing three control nodes, for on-line monitoring/control and off-line simulation. As the Java 3D model is connected with its physical counterpart through the control nodes by low-volume message passing (real-time sensor signals and control commands, etc.), it becomes possible to remotely manipulate the real Tripod through its Java 3D model (see also [17]). Virtual Universe Background Lights T B BG Viewpoin t Contro l T Frame T B Y-Table X-/Y-Table Control SL-1 B GW-3 A A G A Base-1 X-Table Guide-way (GW) TransformGroup Node T B Behaviour Node A Appearance G Geometry BG BranchGroup Node User Defined Codes Moving Platform Kinematic Control B Base-1 G A Base-2 A G T T X-Table G A AG TTT GW-2GW-1 TTT SL-2 SL-3 TTT Moving Platform T End Effecter Base-2 Y-Table Moving Platform End Effecter Sliding-Leg (SL) Figure 5. Java 3D scene graph architecture for Tripod 6.2 Kinematic Modelling for Tripod Kinematics studies the geometric properties of the motion of points without regard to their masses or to the forces acting upon them. While the scene graph is the emergent standard hierarchical data structure for computer modelling of 3D worlds, kinematic models of physical devices or mechanisms that have ex- ternal constraints or constraints that span interior nodes do not fit comfortably Web-Based Remote Manipulation of Parallel Robot in Advanced Manufacturing Systems 669 into its open-branched tree topology. In the case of our Tripod monitoring and control, models of both constrained kinematics and inverse kinematics are solved separately and embedded into the behaviour control nodes in a scene graph to calculate the motions of respective components. Typically, constraints can be expressed in a number of equations or inequalities that describe the re- lationships among Tripod components. Based on sensor signals collected from the real Tripod, both constrained kinematic model and inverse kinematic model of the Tripod are needed to calculate the positions and orientations of the three sliding-legs and moving platform for 3D Tripod model rendering. For the purpose of mathematical formulation, a Tripod kinematic model is shown upside-down in Figures 6 and 7. It is a 3-dof parallel mechanism with linear motion component actuators, the type of linear motion actuated ma- chines with fixed leg lengths and base joints movable on linear guideways (e.g. HexaM, Eclipse, Hexaglide, GeorgV, Z 3 Head). This mechanism consists of three kinematic chains, including three fixed length legs with identical topol- ogy driving by ballscrews, which connects the fixed base to the moving plat- form. In this 3-dof parallel mechanism, the kinematic chains associated with the three identical legs consist, from base to platform, of an actuated linear mo- tion component (ballscrew in the case), a revolute joint (connected by a nut), a fixed length moving link, and a spherical joint attached to the moving plat- form. The arrangement of the structure would be subject to bending in the di- rection parallel to the axis of the revolute joint. The advantages of the structure are: 1) with this basic structure of parallel mechanism, it can be easily extended to 5-dof by adding two gantry type of guideways to realize the 5-dof machin- ing; 2) with the fixed length legs, one can freely choose the variety of leg forms and materials and use linear direct driver to improve the stiffness; and 3) due to reduced heat sources, it is possible to keep the precision in a high level and to maintain a stable stiffness if compared with variable legs. The kinematic equation for the position of the ith spherical joint is given as ii pRhp ′ += (1) where, = i p [ iziyix ppp ,, ] T is the vector representing the position of the ith joint in the global coordinate system O-xyz, i p ′ is the vector representing the same point but in the local coordinates zyx ′′′ -C , =h [ ccc zyx ,, ] T is the vector represent- ing the position of the moving platform, and R is the rotation matrix of the moving platform in terms of rotation angles x θ , y θ , and z θ about x, y, and z axis, respectively. 670 Industrial Robotics: Theory, Modelling and Control Revolute Joint Guide-way Base α 2 Sliding-Leg Spherical Joint Moving Platform z z ′ y ′ x ′ y x α 1 α 3 b 3 b 1 b 2 s 2 s 1 s 3 p i s i b i O l i h p 2 p 1 p 3 Rp i ′ C Figure 6. Tripod kinematic model Among the six motion components of the moving platform, it is known that c x , c y , and z θ are dependent variables. The constraint equations can be de- rived as zypc Lx θθ sincos 3 3 −= (2a) () zyxzxzypc Ly θθθθθθθ sinsinsincoscoscoscos 6 3 −+−= (2b) ¸ ¸ ¹ · ¨ ¨ © § + −= yx yx z θθ θθ θ coscos sinsin arctan (2c) While constrained kinematics of the Tripod is used for monitoring, inverse kinematics is needed for position control. Considering the ith sliding- leg/guide-way system, the kinematic equation of the position of the ith spheri- cal joint, i.e. eq. (1), can be re-written as iiii lsbp ++= (3) where i b is the vector representing the position of the lower end of the ith guide-way attached to the base, i s is the vector representing the displacement along the ith guide-way, and i l is the vector representing the ith sliding leg. Web-Based Remote Manipulation of Parallel Robot in Advanced Manufacturing Systems 671 Since s iii s us = and l iii l ul = , where s i u and l i u are the direction vectors of the ith guide-way and the ith leg, respectively, the actuator displacement i s can be solved considering that the leg length is a constant i s iiii ls =−− ubp (4) where i l is the length of the ith sliding leg. For given x θ , y θ , and c z , dependent variables c x , c y , and z θ can be determined by eqs. (2a) - (2c), then h and R are fully defined. With this, i p can be determined by eq. (1), and subsequently i s can be solved using eq. (4). The true solution of eq. (4) should be the one closer to the previous value, that is () () () ¸ ¹ · ¨ © § −−= = 1min, 2,1 jssss i k i k k i i (5) where j stands for the jth step. In practice, the initial value of i s is provided by an encoder. For the sake of brevity, interested readers are referred to [18] for further details of the Tripod kinematics. 6.3 Remote Monitoring and Control Web-based remote device monitoring and control are conducted by using the StatusMonitor and CyberController, which communicate indirectly with the de- vice controller through an application server. In the case of Tripod monitoring and control, they are further facilitated by the kinematic models, to reduce the amount of data travelling between web browsers and the Tripod controller. Figure 7. CAD model of Tripod 672 Industrial Robotics: Theory, Modelling and Control The required position and orientations of the moving platform are converted into the joint coordinates i s (i = 1, 2, 3) by the inverse kinematics for both Java 3D model rendering at client-side and device control at server-side. The three sliding-legs of the Tripod are driven by three 24V DC servomotors combined with three lead screws. Each actuator has a digital encoder (1.25 μm/count) for position feedback. The position data i s (i = 1, 2, 3) of the sliding-legs are multi- cast to the registered clients for remote monitoring, while only one user at one time is authorized to conduct remote control. A sampling rate of 1 kHz is used for the case study. Figure 8 shows how the Tripod is manipulated from one state to another within the proposed Wise-ShopFloor framework. Figure 8. Web-based remote monitoring and control Web-Based Remote Manipulation of Parallel Robot in Advanced Manufacturing Systems 673 6.4 Managerial Implications The Wise-ShopFloor is a business process that is based on new IT technology to execute business processes. It leverages the IT management tools to deliver re- liable and secured transmission of data between the end users and real shop floors. It provides not only an efficient mechanism for real-time monitoring and control in manufacturing, but it also improves a manufacturing firm's business performance. The implementation of this client-server architecture is likely to result in significant increases in productivity and revenues. 7. Conclusions This chapter presents the Wise-ShopFloor framework and describes detailed three-tier architecture. The goal of the web-based approach is to reduce net- work traffic with Java 3D models, while still providing users with intuitive en- vironments. Participating in the Wise-ShopFloor, users not only can feel reduced network traffic by real-time interactions, but also can obtain more flexible con- trol of their real shop floors. The application in modern manufacturing system is demonstrated for its feasibility and the promise of this novel approach to the growing distributed shop floor environments. As decentralization of business grows, a large application potential of this research is anticipated, in addition to remote real-time monitoring and control. 8. References Cao, J., Li, M.L., Zhang, S.S. and Den, Q. N. 2004. Composing Web Services Based on Agent and Workflow. Grid and Cooperative Computing, Part 1. Berlin: Springer-Verlag Berlin, pp. 948-955. Zeng, L. Benatallah, B., Ngu, A. H. H., Dumas, M., Kalagnanam, J., and Chang, H. 2004. QoS-Aware Middleware for Web Services Composition", IEEE Transactions on Software and Engineering, May. 30(5): 311-327. G. Pritschow, “Research and Development in the Field of Parallel Kinematic Systems in Europe”, Parallel Kinematic Machines – Theoretical Aspects and Industrial Requirements, edited by Boër, C.R., Molinari-Tosatti, L, and Smith, K.S., pp.1-16, Springer-Verlag, (1999). J. Tlusty, J. Ziegert, and S. Ridgeway, “Fundamental Comparison of the Use of Serial and Parallel Kinematics for Machine Tools”, Annals of the CIRP, Vol. 48/1, pp. 351-356, (1999). M. Honegger, A. Codourey, and E. Burdet, “Adaptive Control of the Hexaglide, a 6 DOF Parallel Manipulator”, Proceedings of the 1997 IEEE 674 Industrial Robotics: Theory, Modelling and Control International Conference on Robotics and Automation, Vol. 1, pp. 543-548, (1997). G. Pritschow, and K H. Wurst, “Systematic Design of Hexapods and Other Parallel Link Systems”, Annals of the CIRP, Vol. 46/1, pp. 291-295, (1997). M. Suzuki, K. Watanabe, T. Shibukawa, T. Tooyama, and K. Hattori, “Devel- opment of Milling Machine with Parallel Mechanism”, Toyota Technical Review, Vol. 47 No. 1, pp. 125-130, (1997). F. Pierrot, “From Hexa to HexaM”, International Parallelkinematik- Kolloquium IPK’98, ETH Zurich, pp. 75-84, (1998). H.K. Tönshoff, C. Soehner, and H. Ahlers, “A New Machine Tool Concept for Laser Machining”, Proceedings of International Seminar on Improving Machine Tool Performance, San Sebastian, pp.199-124, (1998). B.S. El-Khasawneh, and P.M. Ferreira, “The Tetrahedral Tripod”, Parallel Ki- nematic Machines – Theoretical Aspects and Industrial Requirements, ed- ited by Boër, C.R., Molinari-Tosatti, L, and Smith, K.S., pp. 419-430, Springer-Verlag, (1999). Kochan, A., “Parallel Robots Perfect Propellers”, Industrial Robot, Vol. 23, No. 4, pp. 27-30, (1996). L. Wang, B. Wong, W. Shen and S. Lang, “Java 3D Enabled Cyber Workspace”, Communications of the ACM, Vol.45, No.11, pp. 45 – 49, 2002. NCMS, “Factory-Floor Internet: Promising New Technology or Looming Secu- rity Disaster”, Manufacturing In Depth, National Center for Manufacturing Sciences, November, 2001. Dan Zhang, L. Wang and Sherman Y. T. Lang, 2005, Parallel Kinematic Ma- chines: Design, Analysis and Simulation in an Integrated Virtual Environ- ment, Transactions of the ASME Journal of Mechanical Design, Vol.127, Is- sue 7, pp. 580-588 J. Barrilleaux, 3D User Interfaces with Java 3D, Manning Publications Co., 2001. H. Sowizral, K. Rushforth and M. Deering, The Java 3D API Specification, Addi- son-Wesley, 2001. L. Wang, F. Xi, D. Zhang and M. Verner, “Design Optimization and Remote Manipulation of a Tripod”, International Journal of Computer Integrated Manufacturing, Vol.18, No.1, pp.85-95, 2005 677 24 Human-Robot Interaction Control for Industrial Robot Arm through Software Platform for Agents and Knowledge Management Tao Zhang, Vuthichai Ampornaramveth and Haruki Ueno 1. Introduction At present, industrial robot arms have been widely adopted in many areas. Unfortunately, operation of them is not easy to master for workers due to complex architectures as well as various control patterns required for each situation. Therefore, if there is a user-friendly human-robot interface and through this interface workers can operate industrial robot arms by their fa- miliar language, it will remarkably reduce the difficulty of the usage of them. The aim of this research is to develop a new human-robot interaction control approach for industrial robot arms by means of software platform for agents and knowledge management in order to construct a symbiotic human-robot system that could be adopted in industrial area (Ueno, 2002). Conventionally, industrial robot arms only can be operated by experts who should possess sufficient knowledge on features of industrial robot arms and be able to control their movement for performing a task. To improve the hu- man-robot interface, researchers have already developed some software inter- faces for the operation of industrial robots (Mizukawa et al, 2002) (Konukseven et al, 2004) (Sales et al, 2004) (Cengiz, 2003). Unfortunately, effective use of these interfaces still depends on the technical training. This paper proposes a knowledge-based human-robot interaction control approach in conjunction with a humanoid robot Robovie as a communication robot (Tao Zhang et al, 2005) for industrial robot arms. With this method, an operator can easily in- teract with the autonomous communication robot by his natural language. The communication robot transfers the sentences of the operator to a sentence parser. The key words extracted by the sentence parser are then sent to a soft- ware platform, called SPAK (Software Platform of Agents and Knowledge Management) (Ampornaramveth et al, 2004). In SPAK, it maintains sufficient knowledge on this human-robot system. According to the defined human- robot interaction control in SPAK, industrial robot arms can be correctly oper- ated according to operator’s request. With the proposed method, a person is not required to be an expert of industrial robot arms but just an ordinary op- erator of industrial robot arms. He can operate industrial robot arms like an expert. 678 Industrial Robotics: Theory, Modelling and Control Although there are many types of industrial robot arms to be operated, the knowledge on these robots can be easily defined in SPAK in a uniform manner by using frame-based knowledge representation schema. The knowledge is maintained as a key component of a communication robot, i.e. a dialog robot, for the robot arms. Therefore, a person only needs to explain his requests to the communication robot. SPAK can assist the person to select appropriate ro- bots and arrange their operations to satisfy the person’s requests to achieve tasks. In addition, SPAK can control different types of robots even they use various kinds of operation systems. From the side of operators, it is no need to possess knowledge on the operations of different types of robots. The remainder of this chapter is organized as follows. In section 2, human- robot system as well as its interaction control process is modelled by frame- based knowledge representation. In section 3, human-robot system is defined in SPAK according to its knowledge model using XML format. Through hu- man-robot interaction, industrial robot arm is controlled by use of SPAK via wireless network in section 4. Section 5 introduces an actual system comprised of human, humanoid robot (Robovie) and industrial robot arm (MELFA) and its experimental results demonstrate the effectiveness of the proposed human- robot interaction control method. 2. Modelling of Human-Robot System Human-robot interaction control for an industrial robot arm is based on the in- teraction between an operator of the robot arm and a communication robot. The operator’s request is transferred to SPAK via wireless network and con- verted into commands of the robot arm. SPAK can control the robot arm with these commands and get the feedback signals from the robot. By converting these feedback signals into the sentences of natural language and speaking out these sentences by the communication robot, the operator can understand the status of the robot arm and continue his operation successfully. From this op- eration process, the definition of human-robot system in SPAK is one of the important components. In order to implement the definition of human-robot system, the modelling of human-robot system is necessary. The modelling of human-robot system is based on the frame-based knowledge representation. It is well known that frame representation systems are cur- rently the primary technology used for large-scale knowledge representation in Artificial Intelligence (AI) (Koller & Pfeffer, 1998). A frame is a data- structure for representing a stereotyped situation (Minsky, 1974). Attached to each frame are several kinds of information, called knowledge. Collections of related frames are linked together into frame-systems. The structure of a frame is consisted of several items, such as Frame name, Frame type, A-kind-of, De- scendants, Slots, etc. (Tairyou, 1998). A frame consists of slots, each of which Human-Robot Interaction Control for Industrial Robot Arm through Software… 679 has different roles in description of knowledge. Table 1 and 2 shows the defini- tion of a frame as well as its slot. Items Meanings Frame name Identification for frame Frame type Type of frame A-kind-of Pointer to parent frame for expressing IS_A relation Descendants Pointer list to children frame Has-part Components of this frame Semantic-link-from Links from other frames according to their semantic relation Semantic-link-to Links to other frames according to their semantic relations Slots Components of the frame Table 1.Meanings of each item in a frame Items Meanings Slot name Identification for slot Role Purpose of slot From Source of slot Data type Explain the attribute of information recorded into the value Value Slot value Condition Condition of slot Argument Argument for slot If-required If slot is required, check this item. If-shared If slot can be shared with other frames, check this item. Frame-related A frame related with slot Frame-list-related Several frames related with slot Default If the slot value is not determined, the default value can be recorded. But the value and default value cannot be given at the same time. Table 2.Meanings of each item in a slot Using frames and their slots, a human-robot system can be modelled simply. This knowledge model is comprised of different frames to represent various pieces of knowledge. For instance, frames for robots include features of a communication robot and industrial robot arms as well as their operation commands. Particularly, the frames for the communication robot include the knowledge on a human-robot interface. At present, a human-robot interface can be implemented by vision recognition, robot speech, physical input, etc. While, frames for users include much information about the users. All frames for the knowledge model are organized by their ISA relations in a hierarchy. That is, a lower level frame is a subclass of its upper level frame. The bottom frames are the instances of the upper level frame. Based on these relations, a [...]...680 Industrial Robotics: Theory, Modelling and Control human-robot interaction control can be defined in frames Table 3 illustrates a part of an example of a frame about a communication robot Frame: Type: A-kind-of: Descendants: Has-part: Semantic-link-from: Semantic-link-to: … Communication robot Instance Robot Empty (mouth, motor, eyes)... knowledge management and improvement can be realized by learning functions in SPAK In the future research, more issues will be considered to solve and the control performance of human-robot system will be further improved We believe that the symbiotic human-robot system with human-robot interaction control has great potentials for future human society 692 Industrial Robotics: Theory, Modelling and Control... Platform for Agents and Knowledge Management in Symbiotic Robotics IEICE Trans Information and Systems, Vol E87-D, No 4, pp 88 6-8 95 Bruce, B (1975) Case systems for natural language Artificial Intelligence, Vol 6, pp 32 7-3 60 Cengiz, M C (2003) Software development for man-machine interface for an industrial robot Thesis of master degree Koller, D & Pfeffer, A (1998) Probabilistic frame-based systems Proc... for the network - the standard and unified network interface for industrial robot applications Proceedings of the 41st SICE Annual Conference, Vol 2, pp 92 5-9 28 Sales, J.; Fernandez, R., et al, (2004) Telecontrol of an industrial robot arm by means of a multimodal user interface: a case study Proceedings of 2004 IEEE International Conference on Systems, Man and Cybernetics, Vol 1, pp 7 6-8 1 Tairyou, G... MELFA can move strictly along a trajectory in three-dimensional space The tip of the robot arm can hold or release workpiece Figure 2 Environment of operation-Robovie system 686 Industrial Robotics: Theory, Modelling and Control In this experiment system, Robovie, MELFA, SPAK and other software components are connected via wireless TCP/IP network SPAK and other software components (such as vision recognition,... SPAK and spoken out by Robovie Figure 6 Frames for the operation of MELFA in SPAK Human-Robot Interaction Control for Industrial Robot Arm through Software… 689 6 Discussions 6.1 Flexibility and extensibility of the proposed human-robot interative control method Since the proposed human-robot interaction control method is based on the knowledge model and implemented by SPAK, it has strong flexibility and. .. for a human-robot system It is definitely needed to de- 690 Industrial Robotics: Theory, Modelling and Control velop functions to automatically fulfill these kinds of knowledge management To imitate human ability, learning new knowledge by means of SPAK for improving the knowledge model is being developed in our research At present, the learning function is implemented as a Java-based program and developed... Development of frame-based knowledge engineering environment ZERO Master Thesis, Tokyo Denki University Tao Zhang, H Ueno (2005), A Frame-Based Knowledge Model for Heterogeneous Multi-Robot System, IEEJ Trans EIS, Vol.125, No.6, pp.84 6-8 55 Ueno, H (2002) A knowledge-based information modeling for autonomous humanoid service robot IEICE Transactions on Information and systems, Vol E85-D, No 4, pp 65 7-6 65 25 Spatial... the different line points and thus complicates the equations As well, the task frame defined by the tcp, and the camera frame are different In contrast to current research (Comport et al., 2005), we assume that problems with image processing, feature detection and projections are solved, which 693 694 Industrial Robotics: Theory, Modelling and Control holds for our simple black and white scenario In addition,... the x-y-plane of the tool frame we can use r zd = 0 In this case the system of equations is limited to equations (6), (9), and (13) to determine ξ ,η , and c zs which are inserted into equation (11) Spatial Vision-Based Control of High-Speed Robot Arms 701 Strictly speaking we use a priori information as well with two lines It is the assumption that the plane of the lines is parallel to the x-y-plane . Kinematic Control B Base-1 G A Base-2 A G T T X-Table G A AG TTT GW-2GW-1 TTT SL-2 SL-3 TTT Moving Platform T End Effecter Base-2 Y-Table Moving Platform End Effecter Sliding-Leg (SL) Figure. Universe Background Lights T B BG Viewpoin t Contro l T Frame T B Y-Table X-/Y-Table Control SL-1 B GW-3 A A G A Base-1 X-Table Guide-way (GW) TransformGroup Node T B Behaviour Node A Appearance G Geometry. rotation angles x θ , y θ , and z θ about x, y, and z axis, respectively. 670 Industrial Robotics: Theory, Modelling and Control Revolute Joint Guide-way Base α 2 Sliding-Leg Spherical Joint