ROBOTIC SYSTEMS – APPLICATIONS, CONTROL AND PROGRAMMING potx

Fujie, Arianna Menciassi and Paolo Dario Chapter 2 Target Point Manipulation Inside a Deformable Object 19 Jadav Das and Nilanjan Sarkar Chapter 3 Novel Assistive Robot for Self-Feeding

ROBOTIC SYSTEMS – APPLICATIONS, CONTROL

AND PROGRAMMING

Edited by Ashish Dutta

Robotic Systems – Applications, Control and Programming

Edited by Ashish Dutta

As for readers, this license allows users to download, copy and build upon published chapters even for commercial purposes, as long as the author and publisher are properly credited, which ensures maximum dissemination and a wider impact of our publications

Notice

Statements and opinions expressed in the chapters are these of the individual contributors and not necessarily those of the editors or publisher No responsibility is accepted for the accuracy of information contained in the published chapters The publisher assumes no responsibility for any damage or injury to persons or property arising out of the use of any materials, instructions, methods or ideas contained in the book

Publishing Process Manager Anja Filipovic

Technical Editor Teodora Smiljanic

Cover Designer InTech Design Team

Image Copyright sarah5, 2011 DepositPhotos

First published January, 2012

Printed in Croatia

A free online edition of this book is available at www.intechopen.com

Additional hard copies can be obtained from orders@intechweb.org

Robotic Systems – Applications, Control and Programming,

Edited by Ashish Dutta

p cm

ISBN 978-953-307-941-7

Contents

Preface IX

Part 1 Applications 1

Chapter 1 Modular Robotic Approach in Surgical Applications –

Wireless Robotic Modules and a Reconfigurable Master Device for Endoluminal Surgery – 3

Kanako Harada, Ekawahyu Susilo, Takao Watanabe, Kazuya Kawamura, Masakatsu G Fujie,

Arianna Menciassi and Paolo Dario Chapter 2 Target Point Manipulation Inside a Deformable Object 19

Jadav Das and Nilanjan Sarkar Chapter 3 Novel Assistive Robot for Self-Feeding 43

Won-Kyung Song and Jongbae Kim Chapter 4 Robot Handling Fabrics Towards Sewing

Using Computational Intelligence Methods 61

Zacharia Paraskevi Chapter 5 Robotic Systems for Radiation Therapy 85

Ivan Buzurovic, Tarun K Podder and Yan Yu Chapter 6 Robotic Urological Surgery:

State of the Art and Future Perspectives 107

Rachid Yakoubi, Shahab Hillyer and Georges-Pascal Haber Chapter 7 Reconfigurable Automation of Carton Packaging

with Robotic Technology 125

Wei Yao and Jian S Dai Chapter 8 Autonomous Anthropomorphic Robotic System

with Low-Cost Colour Sensors

to Monitor Plant Growth in a Laboratory 139

Gourab Sen Gupta, Mark Seelye, John Seelye and Donald Bailey

Part 2 Control and Modeling 159

Chapter 9 CPG Implementations for Robot Locomotion:

Analysis and Design 161

Jose Hugo Barron-Zambrano and Cesar Torres-Huitzil Chapter 10 Tracking Control in an Upper Arm Exoskeleton

with Differential Flatness 183

E Y Veslin, M Dutra, J Slama, O Lengerke and M J M Tavera Chapter 11 Real-Time Control in Robotic Systems 209

Alex Simpkins Chapter 12 Robot Learning from Demonstration

Using Predictive Sequence Learning 235

Erik Billing, Thomas Hellström and Lars-Erik Janlert Chapter 13 Biarticular Actuation of Robotic Systems 251

Jan Babič Chapter 14 Optimization and Synthesis of a Robot Fish Motion 271

Janis Viba, Semjons Cifanskis and Vladimirs Jakushevich Chapter 15 Modeling of Elastic Robot Joints with

Nonlinear Damping and Hysteresis 293

Michael Ruderman Chapter 16 Gravity-Independent Locomotion: Dynamics and

Position-Based Control of Robots on Asteroid Surfaces 313

Marco Chacin and Edward Tunstel Chapter 17 Kinematic and Inverse Dynamic Analysis of a

C5 Joint Parallel Robot 339

Georges Fried, Karim Djouani and Amir Fijany Chapter 18 Utilizing the Functional Work Space Evaluation Tool

for Assessing a System Design and Reconfiguration Alternatives 361

A Djuric and R J Urbanic Chapter 19 Requirement Oriented Reconfiguration

of Parallel Robotic Systems 387

Jan Schmitt, David Inkermann, Carsten Stechert, Annika Raatz and Thomas Vietor

Part 3 Vision and Sensors 411

Chapter 20 Real-Time Robotic Hand Control Using Hand Gestures 413

Jagdish Lal Raheja, Radhey Shyam,

G Arun Rajsekhar and P Bhanu Prasad

Kinematics Relations with Epipolar Geometry 429

Ebrahim Mattar Chapter 22 Design and Construction of an Ultrasonic Sensor

for the Material Identification in Robotic Agents 455

Juan José González España, Jovani Alberto Jiménez Builes and Jaime Alberto Guzmán Luna

Part 4 Programming and Algorithms 471

Chapter 23 Robotic Software Systems: From Code-Driven

to Model-Driven Software Development 473

Christian Schlegel, Andreas Steck and Alex Lotz Chapter 24 Using Ontologies for Configuring Architectures

of Industrial Robotics in Logistic Processes 503

Matthias Burwinkel and Bernd Scholz-Reiter Chapter 25 Programming of Intelligent Service Robots

with the Process Model “FRIEND::Process”

and Configurable Task-Knowledge 529

Oliver Prenzel, Uwe Lange, Henning Kampe, Christian Martens and Axel Gräser

Chapter 26 Performance Evaluation of Fault-Tolerant Controllers in

Robotic Manipulators 553

Claudio Urrea, John Kernand Holman Ortiz Chapter 27 An Approach to Distributed

Component-Based Software for Robotics 571

A C Domínguez-Brito, J Cabrera-Gámez, J D Hernández-Sosa,

J Isern-González and E Fernández-Perdomo Chapter 28 Sequential and Simultaneous Algorithms to Solve

the Collision-Free Trajectory Planning Problem for Industrial Robots – Impact of Interpolation Functions and the Characteristics of the Actuators on Robot Performance 591

Francisco J Rubio, Francisco J Valero, Antonio J Besa and Ana M Pedrosa Chapter 29 Methodology for System Adaptation

Based on Characteristic Patterns 611

Eva Volná, Michal Janošek, Václav Kocian,

Martin Kotyrba and Zuzana Oplatková

Preface

Over the last few decades the focus of robotics research has moved beyond the traditional area of industrial applications to newer areas, including healthcare and domestic applications These newer applications have given a strong impetus to the development of advanced sensors, control strategies and algorithms The first section

of this book contains advanced applications of robotics in surgery, rehabilitation, modular robotics among others This is followed by sections on control and sensors, while the fourth section is devoted to robot algorithms

I would like to thank all the authors for entrusting us with their work and specially the editorial members of InTech publishing

Dr Ashish Dutta

Department of Mechanical Engineering Indian Institute of Technology, Kanpur

India

Applications

Modular Robotic Approach

in Surgical Applications – Wireless Robotic Modules and a Reconfigurable Master Device for Endoluminal Surgery –

Kanako Harada, Ekawahyu Susilo, Takao Watanabe, Kazuya Kawamura,

Masakatsu G Fujie, Arianna Menciassi and Paolo Dario

1The University of Tokyo

2Scuola Superiore Sant’Anna

3Italian Institute of Technology

4Waseda University

1,4Japan 2,3Italy

1 Introduction

The trend in surgical robots is moving from traditional master-slave robots to miniaturized devices for screening and simple surgical operations (Cuschieri, A 2005) For example, capsule endoscopy (Moglia, A 2007) has been conducted worldwide over the last five years with successful outcomes To enhance the dexterity of commercial endoscopic capsules, capsule locomotion has been investigated using legged capsules (Quirini, M 2008) and capsules driven by external magnetic fields (Sendoh, M 2003; Ciuti, G 2010; Carpi, F 2009) Endoscopic capsules with miniaturized arms have also been studied to determine their potential for use in biopsy (Park, S.-K 2008) Furthermore, new surgical procedures known

as natural orifice transluminal endoscopic surgery (NOTES) and Single Port Access surgery are accelerating the development of innovative endoscopic devices (Giday, S 2006; Bardaro, S.J 2006) These advanced surgical devices show potential for the future development of minimally invasive and endoluminal surgery However, the implementable functions in such devices are generally limited owing to space constraints Moreover, advanced capsules

or endoscopes with miniaturized arms have rather poor dexterity because the diameter of such arms must be small (i.e a few millimeters), which results in a small force being generated at the tip

A modular surgical robotic system known as the ARES (Assembling Reconfigurable Endoluminal Surgical system) system has been proposed based on the aforementioned motivations (Harada, K 2009; Harada, K 2010; Menciassi, A 2010) The ARES system is designed for screening and interventions in the gastrointestinal (GI) tracts to overcome the intrinsic limitations of single-capsules or endoscopic devices In the proposed system,

miniaturized robotic modules are ingested and assembled in the stomach cavity The assembled robot can then change its configuration according to the target location and task Modular surgical robots are interesting owing to their potential for application as self-reconfigurable modular robots and innovative surgical robots Many self-reconfigurable modular robots have been investigated worldwide (Yim, M 2007; Murata, S 2007) with the goal of developing systems that are robust and adaptive to the working environment Most

of these robots have been designed for autonomous exploration or surveillance tasks in unstructured environments; therefore, there are no strict constraints regarding the number

of modules, modular size or working space Because the ARES has specific applications and

is used in the GI tract environment, it raises many issues that have not been discussed in depth in the modular robotic field Modular miniaturization down to the ingestible size is one of the most challenging goals In addition, a new interface must be developed so that surgeons can intuitively maneuver the modular surgical robot

The purpose of this paper is to clarify the advantages of the modular approach in surgical applications, as well as to present proof of concept of the modular robotic surgical system The current paper is organized as follows: Section 2 describes the design of the ARES system Section 3 details the design and prototyping of robotic modules, including the experimental results Section 4 describes a reconfigurable master device designed for the robotic modules, and its preliminary evaluation is reported

2 Design of the modular surgical system

2.1 Clinical indications and proposed procedures

The clinical target of the ARES system is the entire GI tract, i.e., the esophagus, stomach, small intestine, and colon Among GI tract pathologies that can benefit from modular robotic features, biopsy for detection of early cancer in the upper side of the stomach (the fundus and the cardia) was selected as the surgical task to be focused on as a first step Stomach cancer is the second leading cause of cancer-related deaths worldwide (World Health Organization 2006), and stomach cancer occurring in the upper side of the stomach has the worst outcome in terms of the 5-year survival ratio (Pesic, M 2004) Thus, early diagnosis of cancer utilizing an advanced endoluminal device may lead to better prognosis The stomach has a large volume (about 1400 ml) when distended, which provides working space to assemble the ingested robotic modules and change the topology of the assembled robot inside (i.e reconfiguration) Each robotic module should be small enough to be swallowed and pass through the whole GI tract Because the size of the commercial endoscopic capsules (11 mm in diameter and 26 mm in length (Moglia, A 2007)) has already been shown to be acceptable for the majority of patients as an ingestible device, each module needs to be miniaturized to this size before being applied to clinical cases

The surgical procedures proposed for the ARES system (Harada, K 2010) are shown in Fig

1 Prior to the surgical procedure, the patient drinks a liquid to distend the stomach to a volume of about 1400 ml Next, the patient ingests 10-15 robotic modules that complete the assembly process before the liquid naturally drains away from the stomach in 10-20 minutes The number of the modules swallowed depends on the target tasks and is determined in advance based on the pre-diagnosis Magnetic self-assembly in the liquid using permanent magnets was selected for this study since its feasibility has already been demonstrated (Nagy, Z 2007) Soon after the assembly, the robot configures its topology

according to preoperative planning by repeated docking and undocking of the modules (the undocking mechanism and electrical contacts between modules are necessary for reconfiguration, but they have not been implemented in the presented design) The robotic modules are controlled via wireless bidirectional communication with a master device operated by the surgeon, while the progress in procedure is observed using intraoperative imaging devices such as fluoroscopy and cameras mounted on the modules After the surgical tasks are completed, the robot reconfigures itself to a snake-like shape to pass through the pyloric sphincter and travel to examine the small intestine and the colon, or it completely disassembles itself into individual modules so that it can be brought out without external aid One of the modules can bring a biopsy tissue sample out of the body for detailed examination after the procedure is complete

Fig 1 Proposed procedures for the ARES system

2.2 Advantages of the modular approach in surgical applications

The modular approach has great potential to provide many advantages to surgical applications These advantages are summarized below using the ARES system as shown in Fig.2 The numbering of the items in Fig.2 is correlated with the following numbering

i The topology of the modular surgical robot can be customized for each patient according to the location of the disease and the size of the body cavity in which the modular robot is deployed A set of functional modules such as cameras, needles and forceps can be selected for each patient based on the necessary diagnosis and surgical operation

ii The modular approach facilitates delivery of more components inside a body cavity that has small entrance/exit hole(s) As there are many cavities in the human body, the modular approach would benefit treatment in such difficult-to-reach places Because

several functional modules can be used simultaneously, the modular robot may perform rather complicated tasks that a single endoscopic capsule or an endoscopic device is not capable of conducting For example, if more than two camera modules are employed, the surgeon can conduct tasks while observing the site from different directions

iii Surgical tools of relatively large diameter can be brought into the body cavity Conventionally, small surgical forceps that can pass through an endoscopic channel of a few millimeters have been used for endoluminal surgery Conversely, surgical devices that have the same diameter as an endoscope can be used in the modular surgical system Consequently, the force generated at the tip of the devices would be rather large, and the performance of the functional devices would be high

iv The surgical system is more adaptive to the given environment and robust to failures Accordingly, it is not necessary for the surgical robot to equip all modules that might be necessary in the body because the surgeons can decide whether to add modules with different functionalities, even during the surgical operation After use, the modules can

be detached and discarded if they are not necessary in the following procedures Similarly, a module can be easily replaced with a new one in case of malfunction

As these advantages suggest, a modular surgical robot would be capable of achieving rather complicated tasks that have not been performed using existing endoluminal surgical devices These advantages are valid for modular robots that work in any body cavity with a small entrance and exit Moreover, this approach may be introduced to NOTES or Single Port Access surgery, in which surgical devices must reach the abdominal cavity through a small incision

In Section 3, several robotic modules are proposed, and the performance of these modules is reported to show the feasibility of the proposed surgical system

3 Robotic modules

3.1 Design and prototyping of the robotic modules

Figure 3 shows the design and prototypes of the Structural Module and the Biopsy Module (Harada, K 2009, Harada, K 2010) The Structural Module has two degrees of freedom (±90°

of bending and 360° of rotation) The Structural Module contains a Li-Po battery (20 mAh, LP2-FR, Plantraco Ltd., Canada), two brushless DC geared motors that are 4 mm in diameter and 17.4 mm in length (SBL04-0829PG337, Namiki Precision Jewel Co Ltd., Japan) and a custom-made motor control board capable of wireless control (Susilo, E 2009) The stall torque of the selected geared motor is 10.6 mNm and the speed is 112 rpm when controlled by the developed controller The bending mechanism is composed of a worm and

a spur gear (9:1 gear reduction), whereas the rotation mechanism is composed of two spur gears (no gear reduction) All gears (DIDEL SA, Switzerland) were made of nylon, and they were machined to be implemented in the small space of the capsule Two permanent magnets (Q-05-1.5-01-N, Webcraft GMbH, Switzerland) were attached at each end of the module to help with self-alignment and modular docking The module is 15.4 mm in diameter and 36.5 mm in length; it requires further miniaturization before clinical application The casing of the prototype was made of acrylic plastic and fabricated by 3D rapid prototyping (Invison XT 3-D Modeler, 3D systems, Inc., USA) The total weight is 5.6 g Assuming that the module would weigh 10 g with the metal chassis and gears, the maximum torque required for lifting two connected modules is 5.4 mNm for both the bending DOF and rotation DOF Assuming that the gear transmission efficiency for the bending mechanism is 30%, the stall torque for the bending DOF is 28.6 mNm On the other hand, the stall torque for the rotation DOF is 8.5 mNm when the transmission efficiency for the rotation mechanism is 80% The torque was designed to have sufficient force for surgical operation, but the transmission efficiency of the miniaturized plastic gears was much smaller than the theoretical value as explained in the next subsection

• Controller

The aforementioned brushless DC motor came with a dedicated motor driving board (SSD04, Namiki Precision Jewel Co., Ltd., 19.6 mm × 34.4 mm × 3 mm) This board only allows driving of one motor; hence, two boards are required for a robotic module with 2 DOFs Because there was not sufficient space for the boards in the robotic module, a custom made high density control board was designed and developed in-house This control board consisted of one CC2430 microcontroller (Texas Instrument, USA) as the main wireless controller and three sets of A3901 dual bridge motor drivers (Allegro MicroSystem, Inc., USA) The fabricated board is 9.6 mm in diameter, 2.5 mm in thickness and 0.37 g in weight, which is compatible with swallowing The A3901 motor driver chip was originally intended for a brushed DC motor, but a software commutation algorithm was implemented to control

a brushless DC motor as well An IEEE 802.15.4 wireless personal area network (WPAN) was introduced as an embedded feature (radio peripheral) of the microcontroller The implemented algorithm enables control of the selected brushless DC motor in Back Electro-Motive Force (BEMF) feedback mode or slow speed stepping mode When the stepping mode is selected, the motor can be driven with a resolution of 0.178º

For the modular approach, each control board shall be equipped with a wired locating system for intra-modular communication in addition to the wireless communication Aside from wireless networking, the wired locating system, which is not implemented in the presented design, would be useful for identification of the sequence of the docked modules

in real time The wired locating system is composed of three lines, one for serial multidrop communication, one for a peripheral locator and one as a ground reference When the modules are firmly connected, the intra-modular communication can be switched from wireless to wired to save power while maintaining the predefined network addresses When one module is detached intentionally or by mistake, it will switch back to wireless mode

• Battery

The battery capacity carried by each module may differ from one to another (e.g from 10 mAh to 50 mAh) depending on the available space inside the module For the current design, a 20 mAh Li-Po battery was selected Continuous driving of the selected motor on its maximum speed using a 20 mAh Li-Po battery was found to last up to 17 minutes A module does not withdraw power continuously because the actuation mechanisms can maintain their position when there is no current to the motor owing to its high gear reduction (337:1) A module consumes power during actuation, but its power use is very low in stand-by mode

• Biopsy Module

The Biopsy Module is a Functional Module that can be used to conduct diagnosis The grasping mechanism has a worm and two spur gears, which allows wide opening of the grasping parts The grasping parts can be hidden in the casing at the maximum opening to prevent tissue damage during ingestion The motor and other components used for the Biopsy Module are the same as for the Structural Module The brushless DC geared motors (SBL04-0829PG337, Namiki Precision Jewel Co Ltd., Japan), the control board, a worm gear and two spur gears (9:1 gear reduction) were implemented in the Biopsy Module A permanent magnet (Q-05-1.5-01-N, Webcraft GMbH, Switzerland) was placed at one side to

be connected to another Structural Module

Fig 3 Design and prototypes of the structural module (left) and the biopsy module (right)

The Biopsy Module can generate a force of 7.1 N at its tip, and can also open the grasping parts to a width of 19 mm with an opening angle of 90 degrees These values are much larger than those of conventional endoscopic forceps, which are 2-4 mm in diameter As a demonstration, Figure 3 shows the Biopsy Module holding a coin weighing 7.5 g

In conventional endoscopy, forceps are inserted through endoscopic channels that are parallel to the direction of the endoscopic view, which often results in the forceps hiding the target Conversely, the Biopsy Module can be positioned at any angle relative to the endoscopic view owing to the modular approach, thereby allowing adequate approach to the target

3.2 Performance of the Structural Module

The mechanical performance of the bending and rotation DOFs of the Structural Module was measured in preliminary tests (Menciassi, A 2010), and the results are summarized in Fig.4 The bending angle was varied by up to ± 90° in steps of 10° three times in succession The measured range of the bending angle was -86.0° to +76.3°, and the maximum error was 15.8° The rotation angle was increased from 0° to 180° in steps of 45° three times in succession, and the measured range of the rotational angle was between 0° and 166.7° with a maximum error of 13.3° The difference between the driven angle and the measured angle was due to backlash of the gears and the lack of precision and stiffness of the casing made

by 3D rapid prototyping Regardless of the errors and the hysteresis, the repeatability was sufficient for the intended application for both DOFs These results indicate that the precision of each motion can be improved by changing the materials of the gears and the casing Since the motor can be controlled with a resolution of 0.178°, very precise surgical tasks can be achieved using different manufacturing processes

180 135 90 45 0

to the aforementioned fabrication problems The thin walls of the casing made of acrylic plastic were easily deformed, which caused friction between the parts The casing made of metal or PEEK and tailor-made metal gears with high precision will improve the mechanism rigidity and performance, thus producing the optimal stability

3.3 Possible designs of robotic modules

Figure 5 shows various designs of robotic modules that can be implemented in the modular surgical robot The modules can be categorized into three types: structural modules, functional modules, and other modules Structural modules are used to configure a robotic topology Functional modules are used for diagnosis or intervention, while other modules can be added to enhance the performance and robustness of the robotic system Obviously,

an assembled robot made of different types of modules (i.e a robot with high heterogeneity) may provide high dexterity, but the self-assembly in the stomach and control of the modules would become more difficult To optimize the level of heterogeneity, self-assembly of the robotic modules must be developed so that the reconfiguration of the robotic topology following the self-assembly can be planned in advance Employing pre-assembled modular arms or tethered modules can be another option to facilitate assembly in a body cavity; however, this would require further anesthesia, and it would hinder the promotion of massive screening

Fig 5 Various designs of the robotic modules

4 Reconfigurable master device

4.1 Design and prototyping of the reconfigurable master device

One main advantage of using a modular approach in surgical applications is the adaptivity

to the given environment as mentioned in Section 2.2 Wherever the robotic platform is deployed in the GI tract, the robotic topology can be changed based on preoperative plans

or the in-situ situation to fit in any particular environment This dynamic changing and reshaping of the robotic topology should be reflected on the user interface Since it is possible for a robotic topology to have redundant DOFs, the master device for the modular surgical system needs to be able to handle the redundancy that is inherent to modular robots Based on these considerations, we propose a reconfigurable master device that resembles the robotic platform (Fig.6) When the assembled robot changes its topology, the master device follows the same configuration The robotic module shown in Fig 6 has a diameter of 15.4 mm, while a module of the reconfigurable master device has a diameter of

30 mm The master modules can be easily assembled or disassembled using set screws, and

it takes only a few seconds to connect one module to another

Each robotic module is equipped with two motors as described in the previous section; thus, each master module is equipped with two potentiometers (TW1103KA, Tyco Electronics) that are used as angular position sensors Calculating the angular position of each joint of the reconfigurable master device is quite straightforward A common reference voltage is sent from a data acquisition card to all potentiometers, after which the angular position can

be calculated from the feedback readings Owing to the identical configuration, the angle of each joint of the robotic modules can be easily determined, even if the topology has redundancy

Fig 6 Robotic modules (top line) and the reconfigurable master device (bottom line): one module (left), assembled modules (middle) and prototypes (right)

The advantages of the proposed master device include intuitive manipulation For example, the rotational movement of a structural module used to twist the arm is limited to ± 180°, and the master module also has this limitation This helps surgeons intuitively understand the range of the motion and the reachable working space of the modules Using a conventional master manipulator or an external console, it is possible that the slave manipulator cannot move owing to its mechanical constraints, while the master manipulator can still move However, using the proposed master device, the surgeon can intuitively understand the mechanical constraints by manipulating the master device during practice/training Furthermore, the position of the master arm can indicate where the robotic modules are, even if they are outside of the camera module's view These characteristics increase the safety of the operation This feature is important because the entire robotic system is placed inside the body In other surgical robotic systems, the position or shape of the robotic arms is not important as they are placed outside of the body and can be seen during operation Unlike other master devices, it is also possible for two or more surgeons to move the reconfigurable master device together at the same time using multi arms with redundant DOFs

4.2 Evaluation

A simulation-based evaluation setup was selected to simplify the preliminary evaluation of the feasibility of the reconfigurable master device The authors previously developed the Slave Simulator to evaluate workloads for a master-slave surgical robot (Kawamura, K 2006) The Slave Simulator can show the motion of the slave robot in CG (Computer Graphics), while the master input device is controlled by an operator Because the simulator can virtually change the parameters of the slave robot or its control, it is easy to evaluate the parameters as well as the operability of the master device This Slave Simulator was appropriately modified for the ARES system The modified Slave Simulator presents the CG models of the robotic modules to the operator The dimension and DOFs of each module in

CG were determined based on the design of the robotic modules The angle of each joint is given by the signal from the potentiometers of the reconfigurable master device, and the slave modules in CG move in real time to reproduce the configuration of the master device This Slave Simulator is capable of altering joint positions and the number of joints of the slave arms in CG so that the workspace of the reconfigurable master device can be reproduced in a virtual environment for several types of topologies The simulator is composed of a 3D viewer that uses OpenGL and a physical calculation function This function was implemented to detect a collision between the CG modules and an object placed in the workspace

To simplify the experiments to evaluate the feasibility of the proposed master device and usefulness of the developed simulator, only one arm of the reconfigurable master device was used Three topologies that consist of one Biopsy Module and one or two Structural Module(s) were selected as illustrated in Fig.7 Topology I consists of a Structural Module and a Biopsy Module, and the base is fixed so that the arm appears with an angle of 45 degrees One Structural Module is added to Topology I to configure Topology II, and Topology III is identical to Topology II, but placed at 0 degrees Both Topology II and Topology III have redundant DOFs The projection of the workspace of each arm and the shared workspace are depicted in Fig.8 A target object on which the arm works in the experiments must be placed in this shared area, which makes it easy to compare topologies

A bar was selected as the target object instead of a sphere because the height of the collision point is different for each topology when the object appears in the same position in the 2D plane

The experiment was designed so that a bar appears at random in the shared workspace The bar represents a target area at which the Biopsy Module needs to collect tissue samples, and this experiment is a simple example to select one topology among three choices given that the arm can reach the target We assumed that this choice may vary depending on the user, and this experiment was designed to determine if the reconfigurability of the master device, i.e customization of the robot, provides advantages and improves performance

Combination

Master

device

Simulator

Fig 7 Three topologies used in the experiments

During the experiment, the operator of the reconfigurable master device could hear a beeping sound when the distal end of the arm (i.e the grasping part of the biopsy module) touched the bar The task designed for the experiments was to move the arm of the reconfigurable master device as quickly as possible, touch the bar in CG, and then maintain its position for three seconds The plane in which the bar stands is shown in grids (Fig.9), and the operator obtains 3D perception by observing these grids The plane with the grids is the same for all topologies The angle of the view was set so that the view is similar to that from the camera module in Fig.6

Five subjects (a-e) participated in the experiments, none of whom were surgeons Each subject was asked to freely move the master device to learn how to operate it; however, this practice was allowed for one minute before starting the experiments Each subject started from Topology I, then tested Topology II and finally Topology III The time needed to touch the bar and maintain it for three seconds was measured This procedure was repeated ten times for each topology with a randomized position of the bar During the procedure, the bar appeared at random; however, it always appeared in the shared workspace to ensure

that the arm could reach it After finishing the experiment, the subjects were asked to fill in a questionnaire (described below) for each topology The subjects were also asked which topology they preferred

Fig 8 Workspace of each topology and the shared workspace

Fig 9 Simulator and the master device during one test

A NASA TLX questionnaire (NASA TLX (website)) was used to objectively and quantitatively evaluate the workload that the subjects felt during the experiments This method has versatile uses, and we selected this method also because it was used to evaluate the workload in a tele-surgery environment (Kawamura, K 2006) This method evaluates Metal Demand, Physical Demand, Temporal Demand, Performance, Effort and Frustration, and gives a score that represents the overall workload that the subject felt during the task

4.3 Results

The time spent conducting the given task, the workload score evaluated using the NASA TLX questionnaire and the preference of the topology determined by the subjects are

summarized in Table 1 For each item, a smaller value indicates a more favorable evaluation

by the subject

Considering the results of the time and workload score, Topology II was most difficult The difference between Topology I and III was interesting Two of the subjects ((b) and (c)) preferred Topology I, which did not have a redundant DOF Conversely, three of the subjects ((a), (d) and (e)) preferred Topology III because they could select the path to reach the target owing to the redundant DOF The average scores of the NASA TLX parameters shown in Fig.10 suggest that the Physical Demand workload was high for Topology I, while the Effort workload was high for Topology III

The two subjects who preferred Topology I rather than Topology III claimed that it was not easy to determine where the bar was located when Topology III was used owing to the lack

of 3D perception In addition, they reported that the bar seemed to be placed far from the base However, the bar appeared randomly, but in the same area; therefore, the bar that appeared in the experiment that employed Topology III was not placed farther from the base when compared to the experiments that used Topology I or Topology II Accordingly, these two subjects may have had difficulty obtaining 3D perception from the gridded plane

In Topology III, the arm was partially out of view in the initial position; thus, the operator needed to obtain 3D perception by seeing the grids It is often said that most surgeons can obtain 3D perception even if they use a 2D camera, and our preliminary experimental results imply that this ability might differ by individual Some people appear to obtain 3D perception primarily by seeing the relative positions between the target and the tool they move Redundant DOFs may also be preferred by operators with better 3D perception capability Although the experiments were preliminary, there must be other factors that accounted for the preference of the user Indeed, it is likely that the preferable topology varies depending

on the user, and the developed simulator would be useful to evaluate these variations The proposed reconfigurable master device will enable individual surgeons to customize the robot and interface as they prefer

Mental Demand

Physical Demand

Temporal Demand

PerformanceEffort

Frustration

Topology ITopology IITopology III

Fig 10 NASA TLX parameters for three topologies

5 Conclusion

A modular robot was proposed for endoluminal surgery The design, prototyping and evaluation of the modules were reported Although there are some issues related to the fabrication problems, the results of the performance tests show the feasibility of the modular surgical system A reconfigurable master device has also been proposed, and its feasibility was evaluated by simulation-based experiments The preliminary results showed that the preferred topology may vary depending on the user Moreover, the reconfigurable master device would enable each surgeon to customize the surgical system according to his/her own preferences Development of the robotic modules and the reconfigurable master device provided proof of concept of the modular robotic system for endoluminal surgery, suggesting that the modular approach has great potential for surgical applications

6 Acknowledgments

This study was supported in part by the European Commission in the framework of the ARES Project (Assembling Reconfigurable Endoluminal Surgical System, NEST-2003-1-ADVENTURE/15653), by the European Union Institute in Japan at Waseda University (EUIJ Waseda, http://www.euij-waseda.jp/eng/) within the framework of its Research Scholarship Programme, and by the Global COE Program "Global Robot Academia" from the Ministry of Education, Culture, Sports, Science and Technology of Japan The authors are grateful to Mr Nicodemo Funaro for manufacturing the prototypes and Ms Sara Condino for her invaluable technical support

7 References

Bardaro, S J & Swanström, L (2006) Development of advanced endoscopes for Natural

Orifice Transluminal Endoscopic Surgery (NOTES) In: Minim Invasive Ther Allied Technol., 15(6), pp 378–383

Carpi, F & Pappone, C (2009) Magnetic maneuvering of endoscopic capsules by means of a

robotic navigation system In: IEEE Trans Biomed Eng, 56(5), pp 1482-90

Ciuti, G.; Valdastri, P., Menciassi, A & Dario, P (2010) Robotic magnetic steering and

locomotion of capsule endoscope for diagnostic and surgical endoluminal

procedures In: Robotica, 28, pp 199-207

Cuschieri, A (2005) Laparoscopic surgery: current status, issues and future developments

In: Surgeon, 3(3), pp 125–138

Giday, S.; Kantsevoy, S & Kalloo, A (2006) Principle and history of natural orifice

translumenal endoscopic surgery (notes) In: Minim Invasive Ther Allied Technol.,

15(6), pp 373–377

Harada, K.; Susilo, E., Menciassi, A & Dario, P (2009) Wireless reconfigurable modules for

robotic endoluminal surgery In: Proc IEEE Int Conf on Robotics and Automation, pp

2699-2704

Harada, K.; Oetomo, D., Susilo, E., Menciassi, A., Daney, D., Merlet, J.-P & Dario, P (2010)

A Reconfigurable Modular Robotic Endoluminal Surgical System: Vision and

preliminary results, In: Robotica, 28, pp 171-183

Kawamura, K.; Kobayashi, Y & Fujie, M G (2006) Development of Real-Time Simulation

for Workload Quantization in Robotic Tele-surgery In: Proc IEEE Int Conf on Robotics and Biomimetics, pp.1420-25

Menciassi, A.; Valdastri, P., Harada, K & Dario, P (2010) Single and Multiple Robotic

Capsules for Endoluminal Diagnosis and Surgery In: Surgical Robotics - System Applications and Visions, Rosen, J., Hannaford, B & Satava, M., pp 313-354.,

Springer-Verlag, 978-1441911254

Moglia, A.; Menciassi, A Schurr, M & Dario, P (2007) Wireless capsule endoscopy: From

diagnostic devices to multipurpose robotic systems In: Biomed Microdevices, 9, pp

235–243

Murata, S & Kurokawa, H (2007) Self-reconfigurable robots In: IEEE Rob Autom Mag.,

14(1), pp 71–78

Nagy, Z.; Abbott, J J & Nelson, B J (2007) The magnetic self-aligning hermaphroditic

connector: A scalable approach for modular microrobotics In: Proc IEEE/ASME Int Conf Advanced Intelligent Mechatronics pp 1–6

NASA TLX available from http://humansystems.arc.nasa.gov/groups/TLX/

Park, S K.; Koo, K I., Bang, S M., Park J Y., Song, S Y., & Cho, D G (2008) A novel

microactuator for microbiopsy in capsular endoscopes In: J Micromech Microeng.,

18 (2), 025032

Pesic, M.; Karanikolic, A., Djordjevic, N., Katic, V., Rancic, Z., Radojkovic, M., Ignjatovic N

& Pesic, I (2004) The importance of primary gastric cancer location in 5-year

survival rate In: Arch Oncol, 12, pp 51–53

Quirini, M.; Menciassi, A., Scapellato, S., Stefanini, C., and Dario, P (2008) Design and

fabrication of a motor legged capsule for the active exploration of the

gastrointestinal tract In: Proc IEEE/ASME Trans Mechatronics, 13, pp 169–179

Sendoh, M.; Ishiyama, K & Arai, K (2003) Fabrication of magnetic actuator for use in a

capsule endoscope In: IEEE Trans Magnetics, 39(5), pp 3232–34

Susilo, E.; Valdastri, P., Menciassi, A & Dario, P (2009) A miniaturized wireless control

platform for robotic capsular endoscopy using advanced pseudokernel approach

In: Sensors and Actuators A, 156(1), pp.49-58

World Health Organisation, “Fact sheet n.297,” Available from: http://www.who.int/

mediacen-ter/factsheets/ fs297, 2006

Yim, M.; Shen, W., Salemi, B., Rus, D., Moll, M., Lipson, H., Klavins, E & Chirikjian, G

(2007) Modular self-reconfigurable robot systems [Grand Challenges of Robotics]

In: IEEE Rob Autom Mag., 14(1), pp 865–872

Target Point Manipulation Inside a Deformable Object

Jadav Das and Nilanjan Sarkar

Vanderbilt University, Nashville, TN

USA

1 Introduction

Target point manipulation inside a deformable object by a robotic system is necessary in many medical and industrial applications such as breast biopsy, drug injection, suturing, precise machining of deformable objects etc However, this is a challenging problem because

of the difficulty of imposing the motion of the internal target point by a finite number of actuation points located at the boundary of the deformable object In addition, there exist several other important manipulative operations that deal with deformable objects such as whole body manipulation [1], shape changing [2], biomanipulation [3] and tumor manipulation [4] that have practical applications The main focus of this chapter is the target point manipulation inside a deformable object For instance, a positioning operation called linking in the manufacturing of seamless garments [5] requires manipulation of internal points of deformable objects Mating of a flexible part in electric industry also results in the positioning of mated points on the object In many cases these points cannot be manipulated directly since the points of interest in a mating part is inaccessible because of contact with a mated part Additionally, in medical field, many diagnostic and therapeutic procedures require accurate needle targeting In case of needle breast biopsy [4] and prostate cancer brachytherapy [6], needles are used to access a designated area to remove a small amount of tissue or to implant radio-active seed at the targeted area The deformation causes the target

to move away from its original location To clarify the situation we present a schematic of needle insertion for breast biopsy procedure as shown in Figure 1 When tip of the needle reaches the interface between two different types of tissue, its further insertion will push the tissue, instead of piercing it, causing unwanted deformations These deformations move the target away from its original location as shown in Figure 1(b) In this case, we cannot manipulate the targeted area directly because it is internal to the organ It must be manipulated by controlling some other points where forces can be applied as shown in Figure 1(c) Therefore, in some cases one would need to move the positioned points to the desired locations of these deformable objects (e.g., mating two deformable parts for sewing seamlessly) while in other cases one may need to preserve the original target location (e.g., guiding the tumor to fall into the path of needle insertion) In either of these situations, the ability of a robotic system to control the target of the deformable object becomes important, which is the focus of this chapter

To control the position of the internal target point inside a deformable object requires appropriate contact locations on the surface of the object Therefore, we address the issue of

determining the optimal contact locations for manipulating a deformable object such that the internal target point can be positioned to the desired location by three robotic fingers using minimum applied forces A position-based PI controller is developed to control the motion of the robotic fingers such that the robotic fingers apply minimum force on the surface of the object to position the internal target point to the desired location However, the controller for target position control is non-collocated since the internal target point is not directly actuated by the robotic fingers It is known in the literature that non-collocated control of a deformable object is not passive, which may lead to instability [7] In order to protect the object and the robotic fingers from physical damage and also in order to diminish the deterioration of performance caused by unwanted oscillation, it is indispensable to build stable interaction between the robotic fingers and the object Here we consider that the plant (i.e., the deformable object) is passive and does not generate any energy So, in order to have stable interaction, it is essential that the controller for the robotic fingers must be stable Thus, we present a new passivity-based non-collocated controller for the robotic fingers to ensure safe and accurate position control of the internal target point The passivity theory states that a system is passive if the energy flowing in exceeds the energy flowing out Creating a passive interface adds the required damping force to make the output energy lower than the input energy To this end we develop a passivity observer (PO) and a passivity controller (PC) based on [8] for individual robotic finger where PO monitors the net energy flow out of the system and PC will supply the necessary damping force to make PO positive Our approach extends the concept of PO and PC in [8] to multi-point contacts with the deformable object

Fig 1 Schematics of needle breast biopsy procedure: (a) needle insertion, (b) target

movement, and (c) target manipulation

The remainder of this chapter is organized as follows: we discuss various issues and prior research in Section 2 The problem description is stated in Section 3 Section 4 outlines the mathematical modelling of the deformable object A framework for optimal contact locations is presented in Section 5 The control methods are discussed in Section 6 The effectiveness of the derived control law is demonstrated by simulation in Section 7 Finally, the contributions of this work and the future directions are discussed in Section 8

2 Issues and prior research

A considerable amount of work on multiple robotic systems has been performed during the last few decades [9-11, 12-15] Mostly, the position and/or force control of multiple manipulators handling a rigid object were studied in [9-11] However, there were some works on handling deformable object by multiple manipulators presented in [12-15] Saha

and Isto [12] presented a motion planner for manipulating deformable linear objects using two cooperating robotic arms to tie self-knots and knots around simple static objects Zhang

et al [13] presented a microrobotic system that is capable of picking up and releasing operation of microobjects Sun et al [14] presented a cooperation task of controlling the reference motion and the deformation when handling a deformable object by two manipulators In [15], Tavasoli et al presented two-time scale control design for trajectory tracking of two cooperating planar rigid robots moving a flexible beam However, to the best of our knowledge the works on manipulating an internal target point inside a deformable object are rare [4, 5] Mallapragada et al [4] developed an external robotic system to position the tumor in image-guided breast biopsy procedures In their work, three linear actuators manipulate the tissue phantom externally to position an embedded target inline with the needle during insertion In [5] Hirai et al developed a robust control law for manipulation of 2D deformable parts using tactile and vision feedback to control the motion

of the deformable object with respect to the position of selected reference points These works are very important to ours present application, but they did not address the optimal locations of the contact points for effecting the desired motion

A wide variety of modeling approaches have been presented in the literature dealing with computer simulation of deformable objects [16] These are mainly derived from physically-based models to produce physically valid behaviors Mass-spring models are one of the most common forms of deformable objects A general mass-spring model consists of a set of point masses connected to its neighbors by massless springs Mass-spring models have been used extensively in facial animation [17], cloth motion [18] and surgical simulation [19] Howard and Bekey [20] developed a generalized method to model an elastic object with the connections of springs and dampers Finite element models have been used in the computer simulation to model facial tissue and predict surgical outcomes [21, 22] However, the works

on controlling an internal point in a deformable object are not attempted

In order to manipulate the target point to the desired location, we must know the appropriate contact locations for effecting the desired motion There can be an infinite number of possible ways of choosing the contact location based on the object shapes and task to be performed Appropriate selection of the contact points is an important issue for performing certain tasks The determination of optimal contact points for rigid object was extensively studied by many researchers with various stability criteria Salisbury [23] and Kerr [24] discussed that a stable grasp was achieved if and only if the grasp matrix is full row rank Abel et al [25] modelled the contact interaction by point contact with Coulomb friction and they stated that optimal grasp has minimum dependency on frictional forces Cutkosky [26] discussed that the size and shape of the object has less effect on the choice of grasp than by the tasks to be performed after examining a variety of human grasps Ferrari

et al [27] defined grasp quality to minimize either the maximum value or sum of the finger forces as optimality criteria Garg and Dutta [28] shown that the internal forces required for grasping deformable objects vary with size of object and finger contact angle In [29], Watanabe and Yoshikawa investigated optimal contact points on an arbitrary shaped object

in 3D using the concept of required external force set Ding et al proposed an algorithm for computing form closure grasp on a 3D polyhedral object using local search strategy in [30]

In [31, 32], various concepts and methodologies of robot grasping of rigid objects were reviewed Cornella et al [33] presented a mathematical approach to obtain optimal solution

of contact points using the dual theorem of nonlinear programming Saut et al [34]

presented a method for solving the grasping force optimization problem of multi-fingered dexterous hand by minimizing a cost function All these works are based on grasp of rigid objects

There are also a few works based on deformable object grasping Like Gopalakrishnan and Goldberg [35] proposed a framework for grasping deformable parts in assembly lines based

on form closure properties for grasping rigid parts Foresti and Pellegrino [36] described an automatic way of handling deformable objects using vision technique The vision system worked along with a hierarchical self-organizing neural network to select proper grasping points in 2D Wakamatsu et al [37] analyzed grasping of deformable objects and introduced bounded force closure However, position control of an internal target point in a deformable object by multi-fingered gripper has not been attempted In our work, we address the issue

of determining the optimal contact locations for manipulating a deformable object such that the internal target point can be positioned to the desired location by three robotic fingers using minimum applied forces

The idea of passivity can be used to guarantee the stable interaction without exact knowledge of model information Anderson and Spong [38] published the first solid result

by passivation of the system using scattering theory A passivity based impedance control strategy for robotic grasping and manipulation was presented by Stramigioli et al [39] Recently, Hannaford and Ryu [40] proposed a time-domain passivity control based on the energy consumption principle The proposed algorithm did not require any knowledge about the dynamics of the system They presented a PO and a PC to ensure stability under a wide variety of operating conditions The PO can measure energy flow in and out of one or more subsystems in real-time by confining their analysis to system with very fast sampling rate Meanwhile the PC, which is an adaptive dissipation element, absorbs exactly net energy output measured by the PO at each time sample In [41], a model independent passivity-based approach to guarantee stability of a flexible manipulator with a non-collocated sensor-actuator pair is presented This technique uses an active damping element

to dissipate energy when the system becomes active In our work we use the similar concept

of PO and PC to ensure stable interaction between the robotic fingers and the deformable object Our work also extends the concept of PO and PC for multi-point contact with the deformable object

Manipulation points: are defined as the points that can be manipulated directly by robotic fingers In our case, the manipulation points are the points where the external robotic fingers apply forces on the deformable object

Positioned points: are defined as the points that should be positioned indirectly by controlling manipulation points appropriately In our case, the target is the position point The control law to be designed is non-collocated since the internal target point is not directly actuated by the robotic fingers The following result is useful in determining the number of actuation points required to accurately position the target at the desired location

Result [42]: The number of manipulated points must be greater than or equal to that of the

positioned points in order to realize any arbitrary displacement

In our present case, we assume that the number of positioned points is one, since we are

trying to control the position of the target Hence, ideally the number of contact points

would also be one But practically, we assume that there are two constraints: (1) we do not

want to apply shear force on the deformable object to avoid the damage to the surface, and

(2) we can only apply control force directed into the deformable object We cannot pull the

surface since the robotic fingers are not attached to the surface Thus we need to control the

position of the target by applying only unidirectional compressive force

In this context, there exists a theorem on the force direction closure in mechanics that helps

us determining the equivalent number of compressive forces that can replace one

unconstrained force in a 2D plane

Theorem [43]: A set of wrenches w can generate force in any direction if and only if there

exists a three-tuple of wrenches {w w w1, 2, 3} whose respective force directions f1, f2, f3

satisfy:

i Two of the three directions f1, f2, f3 are independent

ii A strictly positive combination of the three directions is zero

1+ 2+ 3=0

where α, β, and γ are constants The ramification of this theorem for our problem is that

we need three control forces distributed around the object such that the end points of their

direction vectors draw a non-zero triangle that includes their common origin point With

such an arrangement we can realize any arbitrary displacement of the target point Thus the

problem can be stated as:

Problem statement: Given the number of actuation points, the initial target and its desired

locations, find appropriate contact locations and control action such that the target point is

positioned to its desired location by controlling the boundary points of the object with

minimum force

4 Deformable object modelling

Consider a schematic in Figure 2 where three robotic fingers are positioning an internal

target (point A) in a deformable object to the desired location (point B) We assume that all

the end-effectors of the robotic fingers are in contact with the deformable object such that

they can apply only push on the object as needed

The coordinate systems are defined as follows: w is the task coordinate system, o is the

object coordinate system, fixed on the object and i is the i-th robotic finger coordinate

system, fixed on the i-th end-effectors located at the grasping point In order to formulate

the optimal contact locations, we model the deformable object using mass-spring-damper

systems The point masses are located at the nodal points and a Voigt element [20] is

inserted between them Figure 3 shows a single layer of the deformable object Each element

is labeled as E for j j=1,2, , NE where NE is total number of elements in a single layer

Position vector of the i-th mesh point is defined as p =[ ]T

i x i y i , i=1,2,3, ,N where, N

is total number of point masses k and c are the spring stiffness and the damping

coefficient, respectively Assume that no moment exerts on each mesh point Then, the

resultant force exerted on the mesh point, pi, can be calculated as

= −

∂

w p

i i

U

(2)

where, U denotes the total potential energy of the object

Fig 2 Schematic of the robotic fingers manipulating a deformable object

Fig 3 Model of a deformable object with interconnected mass-spring-damper

5 Framework for optimal contact locations

We develop an optimization technique that satisfies the force closure condition for three

fingers planar grasp The resultant wrench for the contacts of three robotic fingers is given by

where, ( )n ri i is the unit inner normal of i-th contact and f i denotes the i-th finger’s force

We assume that the contact forces should exist in the friction cone to manipulate objects

without slip of the fingertip Now we need to find three distinct points, r1( )θ1 , r2( )θ2 , and

3( )3

r θ , on the boundary of the object such that Equation (3) is satisfied Here, θ1, θ2, and θ3

are the three contact point locations measured anti-clockwise with respect to the x axis as

shown in Figure 4 In addition, we assume that the normal forces have to be non-negative to

avoid separation and slippage at the contact points, i.e.,

0

≥

i

Fig 4 Three fingers grasp of a planar object

A physically realizable grasping configuration can be achieved if the surface normals at

three contact points positively span the plane so that they do not all lie in the same

half-plane [44] Therefore, a realizable grasp can be achieved if the pair-wise angles satisfy the

following constraints

min≤| i− j|≤ max

θ θ θ θ ,θlow≤ ≤θ θi high, ,i j=1,2,3, i j≠ (5)

A unique solution to realizable grasping may not always exist Therefore, we develop an

optimization technique that minimizes the total force applied on to the object to obtain a

particular solution The optimal locations of the contact points would be the solution of the

following optimization problem

Once we get the optimal contact locations, all three robotic fingers can be located at their

respective positions to effect the desired motion at those contact points

6 Design of the controller

In this section, a control law for the robotic fingers is developed to guide a target from any

point A to an arbitrary point B within the deformable object as shown in Figure 2

6.1 Target position control

At any given time-step, point A is the actual location of the target and point B is the desired

location of the target n1, n2 and n3 are unit vectors which determine the direction of force

application of the actuation points with respect to the global reference frame w Let

assume, pd is the position vector of point B and p is the position vector of point A

Referring to Figure 2, the position vector of point A is given by

=

where, x and y are the position coordinates of point A in the global reference frame  w The

desired target position is represented by point B whose position vector is given by

Once the optimal contact locations are determined from Equation (6), the planner generates

the desired reference locations for these contact points by projecting the error vector

between the desired and the actual target locations in the direction of the applied forces,

All robotic fingers are controlled by their individual controllers using the following

proportional-integral (PI) control law

= e + e

where, K Pi, and K Ii are the proportional and integral gains Note that in the control law

(12), mechanical properties of the deformable object are not required Forces applied by the

fingers on the surface of the deformable object are calculated by projecting the error vector

in the direction of the applied forces But the Equation (12) does not guarantee that the

system will be stable Thus a passivity-based control approach based on energy monitoring

is developed to guarantee the stability of the system

6.2 Passivity-based control

A passivity-based control approach based on energy monitoring is developed for

deformable object manipulation to guarantee passivity (and consequently stability) of the

system The main reason to use passivity-based control is to ensure stability without the

need of having an accurate model of the deformable object It is not possible to develop a

precise dynamic model of a deformable object due to complex nonlinear internal dynamics

as well as variation in geometry and mechanical properties Thus passivity based control is

an ideal candidate to ensure stability since it is a model independent technique The basic

idea is to use a PO to monitor the energy generated by the controller and to dissipate the

excess energy using a PC when the controller becomes active [41], without the need for

modeling the dynamics of the plant (deformable object)

Passivity Observer (PO)

We develop a network model with PO and PC similar to [41] as shown in Figure 5 The PO

monitors the net energy flow of the individual finger’s controller When the energy becomes

negative, PC dissipates excess energy from the individual controller Similar to [41] energy

is defined as the integral of the inner product between conjugate input and output, which

may or may not correspond to physical energy Definition of passivity [41] states that the

energy applied to a passive network must be positive for all time Figure 5 shows a network

representation of the energetic behavior of this control system The block diagram in Figure

5 is partitioned into three elements: the trajectory generator, the controller and the plant

Each controller corresponds to one finger Since three robotic fingers are used for planar

manipulation, three individual controller transfer energy to the plant

The connection between the controller and the plant is a physical interface at which

conjugate variables (f i, v i; where f i is the force applied by i-th finger and v i is the velocity

of i-th finger) define physical energy flow between controller and plant The forces and

velocities are given by

where, x d and y d are the desired target velocities, respectively The desired target velocity

along the direction of actuation of the i-th robotic finger is given by

The trajectory generator essentially computes the desired target velocity along the direction

of actuation of the robotic fingers If the direction of actuation of the robotic fingers, ni, and

desired target velocity, pd, are known with respect to a global reference frame then the

trajectory generator computes the desired target velocity along the direction of actuation of

the fingers using Equation (16)

The connections between the trajectory generator and the controller, which traditionally consist of a one-way command information flow, are modified by the addition of a virtual feedback of the conjugate variable [41] For the system shown in Figure 5, output of the trajectory generator is the desired target velocity, v di , along direction of i-th finger and

output of the controller is calculated from Equation (12)

Fig 5 Network representation of the control system α1iandα2iare the adjustable damping

elements at each port, i=1,2,3

For both connections, virtual feedback is the force applied by the robotic fingers Integral of the inner product between trajectory generator output (v di) and its conjugate variable (f i) defines “virtual input energy.” The virtual input energy is generated to give a command to the controller, which transmits the input energy to the plant through the controller in the form of “real output energy.” Real output energy is the physical energy that enters to the plant (deformable object) at the point where the robotic finger is in contact with the object Therefore, the plant is a three-port system since three fingers manipulate the object The conjugate pair that represents the power flow is f i, v i (the force and the velocity of i-th

finger, respectively) The reason for defining virtual input energy is to transfer the source of energy from the controllers to the trajectory generator Thus the controllers can be represented as two-ports which characterize energy exchange between the trajectory generator and the plant Note that the conjugate variables that define power flow are discrete time values and so the analysis is confined to systems having a sampling rate substantially faster than the system dynamics

For regulating the target position during manipulation, v di=0 Hence the trajectory generator is passive since it does not generate energy However, for target tracking, v di≠0

and f i≠0 Therefore the trajectory generator is not passive because it has a velocity source

as a power source It is shown that even if the system has an active term, stability is

guaranteed as long as the active term is not dependent on the system states [45] Therefore,

passivity of the plant and controllers is sufficient to ensure system stability

Here, we consider that the plant is passive Now we design a PO for sufficiently small

where, ΔT is the sampling period and t j= × Δj T In normal passive operation, ( ) E t i j

should always be positive In case when ( ) 0E t i j < , the PO indicates that the i-th controller is

generating energy and going to be active The sufficient condition to make the whole system

passive can be written as

where k means the k-th step sampling time

The active and passive port can be recognized by monitoring the conjugate signal pair of

each port in real time A port is active if fv<0that means energy is flowing out of the

network system and it is passive iffv≥0, that means energy is flowing into the network

system The input and output energy can be computed as [46]

1 1

1

( 1) ( ) ( ) if ( ) ( ) 0( )

2

( 1) ( ) ( ) if ( ) ( ) 0( )

1

( 1) ( ) ( ) if ( ) ( ) 0( )

2

( 1) ( ) ( ) if ( ) ( ) 0( )

E k are the energy flowing in and out at the trajectory side of the

controller port, respectively, whereas 1P( )

i

E k and 2P( )

i

E k are the energy flowing in and out at

the plant side of the controller port, respectively So the time domain passivity condition is

In order to dissipate excess energy of the controlled system, a damping force should be

applied to its moving parts depending on the causality of the port As it is well known, such

a force is a function of the system's velocities giving the physical damping action on the

system Mathematically, the damping force is given by

=

d

where α is the adjustable damping factor and v is the velocity From this simple

observation, it seems necessary to measure and use the velocities of the robotic fingers in the

control algorithm in order to enhance the performance by means of controlling the damping

forces acting on the systems On the other hand, velocities measurements are not always

available and in these cases position measurements can be used to estimate velocities and

therefore to inject damping

When the observed energy becomes negative, the damping coefficient is computed using

the following relation (which obeys the constitutive Equation (25)) Therefore, the algorithm

used for a 2-port network with impedance causality (i.e., velocity input, force output) at

each port is given by the following steps:

1 Two series PCs are designed for several cases as given below:

Case 1: If ( ) 0E k i ≥ , i.e., if the output energy is less than the input energy, there is no

need to activate any PCs

Case 2: If ( ) 0E k i < , i.e., if the output energy is more than the input energy, i.e.,

( ) 0( ) ( ) / ( )

2

( ) ( ) / ( )( ) 0