RESEA R C H Open Access A robotic wheelchair trainer: design overview and a feasibility study Laura Marchal-Crespo 1* , Jan Furumasu 2 , David J Reinkensmeyer 1 Abstract Background: Experiencing independent mobility is important for children with a severe movement disabil ity, but learning to drive a powered wheelchair can be labor intensive, requi ring hand-over-hand assistance from a skilled therapist. Methods: To improve accessibility to training, we developed a robotic wheelchair trainer that steers itself along a course marked by a line on the floor using computer vision, haptically guiding the driver’s hand in appropriate steering motions using a force feedback joystick, as the driver tries to catch a mobile robot in a game of “robot tag”. This paper provides a detailed design description of the computer vision and control system. In addition, we present data from a pilot study in which we used the chair to teach children without motor impairment aged 4-9 (n = 22) to drive the wheelchair in a single training session, in order to verify that the wheelchair could enable learning by the non-impaired motor system, and to establish normative values of learning rates. Results and Discussion: Training with haptic guidance from the robotic wheelchair trainer impr oved the steering ability of children without motor impairment significantly more than training without guidance. We also report the results of a case study with one 8-year-old child with a severe motor impairment due to cerebral palsy, who replicated the single-session training protocol that the non-disabled children participated in. This child also improved steering ability after training with guidance from the joystick by an amount even greater than the children without motor impairment. Conclusions: The system not only provided a safe, fun context for automating driver’s training, but also enhanced motor learning by the non-impaired motor system, presumably by demonstrating through intuitive movement and force of the joystick itself exemplary control to follow the course. The case study indicates that a child with a motor system impaired by CP can also gain a short-term benefit from driver’s training with haptic guidance. Introduction Independent mobility is crucial for children’ cognitive, emotional, and psychosocial development [1-5]. Provid- ing a child with self-controlled, powered mobility pro- vides motivation for learning since the chair becomes a tool for exploration, locomotion, and play. However, many children with disabilities do not achieve indepen- dent mobility, especially at a young age, when thi s sti- mulus of mobility particularly influences development. It seems likely that this situation is caused in part by lim- ited training time: children with severe disabilities can and do learn new motor skills, but often more slowly than children without developmental disorders. Because the c onventional approach for powered wheelch air dri- ver’s training is expensive and labor-intense, typically requiring the hand-over-hand assistance of a skilled therapist to facilitate learning and ensure safety during training sessions, children who do not learn quickly may experience limited training time, preventing them from achieving independent driving ability. To lower the cost and improve accessibility to train- ing, we have developed a robotic powered wheelchair system on which young children with a disability can safely develop driving skills at their own pace with mini- mum a ssistance from a therapist. We equipped a pow- ered wheelchair with a web-cam that identifies and tracks a line on the floor to achieve a self steering func- tion along a training course. We added a force-feedback * Correspondence: laura.marchal@mavt.ethz.ch 1 Mechanical and Aerospace Engineering Department, University of California, Irvine, CA, USA Full list of author information is available at the end of the article Marchal-Crespo et al. Journal of NeuroEngineering and Rehabilitation 2010, 7:40 http://www.jneuroengrehab.com/content/7/1/40 JNER JOURNAL OF NEUROENGINEERING AND REHABILITATION © 2010 Marchal-Crespo et al; licensee BioMed Central Ltd. This is an Open Ac cess article distributed under t he terms of the Crea tive Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. joystick to implement an algorithm [6] that can demon- strate (through movement and force of the joystick itself) exemplary control to follow the course, while sys- tematically modulating the streng th and sensitivity of such haptic demonstration, making the joystick stiffer (and more damped) when more assistance is needed. This method gradually exposes the child to the dynamics of a normal powered wheelchair, in an analo- gous fashion to bicycle training wheels. The idea is to let the individual learn from the experience of making errors repeatedly and safely in a structured environment, while reducing demands on the supervising caregiver. The smart powered wheelchair described here is intended to work as a tool targeted specifically at dri- ver’s training, in contrast to most other pediatric smart wheelchairs developed to the date (e.g. [1,7,8]), which aim to help children with disabilities to steer a power wheelchair during activities of daily living by relieving some of the contr ol burden. The pediatric smart wheel- chair de veloped at the CALL Center of the University of Edinburgh, Scotland [8] is a relevant example t o our work. T his institution has developed a pediatric smart wheelchair trainer with bump sensors, sonar sensors, and the ability to follow tape lines on the floor to train disabled children drivers to improve their mobility using different levels of autonomy. However, the CALL Center smart wheelchair does not provide haptic feedback whilefollowingthelineonthefloor.Aprimarydesign goal of the system described here was to have it gradu- ally and automatically give more control to the child as learning progresses, rather than “take over” control. Our working hypothesis is that by appropriately challenging the child, the development of steering skill will be facili- tated, a hypothesis consistent with the Challenge Point Theory from motor learning research [9]. To intelligently challenge the user, the chair uses fad- ing, ha ptic guidance. Haptic guidance is a motor-train- ing strategy in which a trainer physically interacts with the participant’s limbs du ring movement training, steer- ing them along desired movements [10-13]. Haptic gui- dance is co mmonly used by rehabilitation therapists in wheelchair driver’s training, as well as in many other rehabilitation and sports training applications. Besides providing a safety benefit, a common concept is that physically demonstrating a movement may help people learn how to perform it. However, there i s little evi- dence that robotic guidance is beneficial for human motor learning beyond enhancing safety, compared to unassisted practice. The long-standing “ Guidance Hypothesis” in fact asserts that providing too much phy- sical or cognitive assistance during training will impair learning, because it obviates t he nervous system from learning the error-correction strategies required to suc- cessfully perform the target tas k [14,15]. A number of studies have confirmed this hypothesis, finding that phy- sically guiding movements does not aid motor learning and may in fact hamper it [10-13,16-21]. Thus, a concern we had at the onset with the approach presented here is that, while providing haptic guidance could make training safer and help automate training, it may impair learning of driving skill. To address this concern, we performed preliminary studies with a virtual reality wheelchair driving simulator and non-impaired, adult subjects [6,22]. We developed a control algorithm to provide haptic guidance with a force feedback steering wheel as a person steers a simu- lated power wheelchair. We incorporated a novel gui- dance-as-needed strategy, which adjusts levels of guidance based on the ongoing performance of the dri- ver. Preliminary studies from our lab showed that train- ing with guidance-as-needed improved the drivers’ steering ability more than training without guidance, apparently because it helped learn when to begin turns [6]. Furthermore, training with haptic guidance was more beneficial for initially less skilled people [22]. These previous studies were done with a virtual wheelchair that moved at a constant speed, with a force feedback steering wheel, and with adult participants. As described in this paper, we have now implemented the steering algorithm using a force feedback joystick on a pediatric wheelchair. This necessitated development of a computer vision system, as we ll as an extension of the haptic guidance algorithm to take into account changes in wheelchair velocity. To determine if the resulting robotic wheelchair trainer could assist effectively in training, we performed an experiment with 22 non-dis- abled children (aged 3-9, mean 6.6 ± .5 SD) randomly assigned into “ Guidance” and “No Guidance” groups. We compared the resulting performance after training with guidance and training without assistance in a single training session in order to determine if robotic gui- dance pro motes learning compared to training w ithout guidance for the non-injured, developing human motor system. We also report the results of a case study with one 8-year-old child with a severe motor impairment due to cerebral palsy, who re plicated the Guidance sin- gle-session training protocol. We compared her increase of steering ability with the “Guidance” and “ No Gui- dance” gro ups to determine if a impaired motor system can also benefit from haptic guidance during driver’ s training. Methods The smart power wheelchair system We developed a prototype pediatric smar t wheelchair (ROLY -RObot-assisted Learning for Young drivers) that incorporates a webcam to a chieve a self steering func- tion along a training course (defined by a black line on Marchal-Crespo et al. Journal of NeuroEngineering and Rehabilitation 2010, 7:40 http://www.jneuroengrehab.com/content/7/1/40 Page 2 of 12 the floor), and a force-feedback joystick to implement an algorithm that can demonstrate (through movement and force o f the joystick itself) exemplary control to follow the course, while systematically modulati ng the strength and sensitivity of such haptic demonstration (Figure 1). We installed the camera, joystick and a laptop on a commercial pediatric powered wheelchair (Quickie Z-500). The force-feedback joystick (Figure 1, Immer- sion Impulse Stick) uses electric motors that can be pro- grammed t o produce forces up to 14.5 N (3.5 lbf), and can move to a desire d position with a resolution of 0.01 degrees. The joystick can physically demonstrate the control m otion required for successful driving along the test course, applying forces to the participants’ hands only when s/he makes steering errors, and thus correct the joystick motion to bring the power wheelchair back to the desired circuit. The stiffness and damping effects of the force-feedback joystick can be modified, thus making the joystick stiffer (and more damped) when more assistance is needed. The guidance provided by the joystick was designed to anticipate tu rns, as is described in previous work [6]. As a wheelchair is a non-holomonic vehicle, in order to minimize the tracking error when turning, the driver has to start the movement before the track changes direction. The driving action is then dependent on what the driver s ees in front of him or her. We translated this look-ahead idea to the guidance controller, similarly to Sheridan’s work in constrained preview control [23], using the distance and direction error with respect to a point situated a determined distance d ahead of the vehicle. We also incorporated previous findings [6,13,22] in motor learning through the implementation of a fadedcontrolalgorithmthatchangesthe“firmness” of the guidance as the participants perform the task, limit- ing large errors, while being constantly presented with a higher degree of challenge. The guidance controller had the following form: Jx Kd e Ka e Ba de ang dt des dis ang =⋅ +⋅ +⋅ () . (1) The desired joystick x-axis position (Jx des ) d epends on the look-ahead distance error (e dis ), the look-ahead direction error (e ang ), and its time derivative (d(e ang )/dt). Theguidancewasdefinedasaforcethatthejoystick applies on the child’s hands. Note that only the steering command is controlled (x-axis), while the wheelchair speed (y-axis) was freely selected by the driver during the experiment. The guidance force (F assist ) was calcu- lated as follows: FKjJxJxBj dJx dt ssist desa =⋅ − +⋅() () (2) Where K j and B j are the joystick’s stiffness and damp- ing coefficients, which can be modulated through the DirectX force feedback (FFB) libraries, and Jx is the cur- rent x-axis joystick position. It is clear that as the wheelchair’s position and direction errors become lar- ger, the desired joystick x-axis position (Jx des )andthe joystick position error (Jx - Jx des ) increase, and thus the guidance force (F assist ) becomes larger. Note however, that at equal errors, when the stiffness and damping coefficients (K j and B j ) are larger, the guidance force will be larger. We faded the firmness of the force feedback allowing more freedom (more error) around the line as training progressed, but a lways limiting large errors. In other words, as the participan t drove, the joystick applied less Figure 1 ROLY -RObot-assisted Learning for Young drivers.We developed a robotic wheelchair trainer that steers itself along a course marked by a line on the floor using computer vision, haptically guiding the child’s hand in appropriate steering motions using a force feedback joystick. The child is instructed to follow the line with a spot of light from a laser pointer mounted on the chair, creating the smallest amount of error possible. To motivate the children during training, we programmed a small mobile robot to follow the same black line, and requested the child to try to catch it in a game of “robot tag”. Marchal-Crespo et al. Journal of NeuroEngineering and Rehabilitation 2010, 7:40 http://www.jneuroengrehab.com/content/7/1/40 Page 3 of 12 force for the same error values by updating the stiffness and damping control gains (K j and B j ). GfG iRi+ =⋅ 1 (3) where G represents the value of the control gains, f R is the “forgetting factor” ( f R = 0.9976), and the subscript i indicates the i-thiteration. Note that the forgetting factor f R must be less than 1 in order to decrease the value of the guidance as training proceeds; the particular value chosen was selected to decrease guidance expo- nentially with a time constant of 4.63 minutes. We observed in preliminary experiments with experi- enced drivers that their look-ahead distance was linearly dependent on the spee d: as the wheelchair moves faster, they needed a larger look-ahead distance to correctly react to the sudden changes of line direction, and steer accurately with minimal tracking e rror. We ran several trials with the chair at different speeds and found a lin- ear correlation between the optimal look-ahead distance and the power wheelchair speed that al lows the wheel- chair to steer accurately at different speeds, where the optimal look-ahead distance is defined as the look-ahead distance that minimizes the overall tracking error in a trial, when the wheelchair steers autonomously: dJy=− ⋅ +80 160 (4) where d is the optimal look-ahead distance in image coordinates, and Jy is the y-axis position of the joystick handler (ranging from 0 to 1). Thus, as the wheelchair moves faster, the computer v ision system calculates t he look-ahead errors using a greater look-ahead distance. The maximum wheelchair speed ( y - ax is =1)inlongi- tudinal direction is 1.28 m/s, and the maximum mean speed through the circuit is 0.38 m/s. Sensor systems To calculate the appropriate steering assistance forces, the smart wheelchair has to kno w the l ook-ahead error (e dis ). We developed an on-board vision system that uses a low-cost webcam (Figure 1, QuickCam Pro 9000) mounted at the front of the wheelchair and implemented a line-following algorithm on the laptop using Simulink. The vision system algorithm identifies the black line in the video stream using color classification, edge detection algorithms and the Hough transformation, tracks the lin e using a Kalman filter, and calculates the look-ahead dis- tance of the wheelchair to the line (e dis ), and the direction of the wheelchair with respect to the line (e ang )using inverse perspective mapping, as a continu ous variable no matter what the wheelchair’s position. The vision system algorithm is fed with 240 × 320 greyscale frames. However, to reduce computational time, we further reduced the size of the regio n of interest (ROI) to 40 pixels above and below the look- ahead position (represented as a horizontal white line on Figure 2). The greyscale ROI was then converted into a black and white image (BW), such that pixels in the ROI with an intensity value below a threshold (I = 0.3) were considered as candidate points to be part of the line (candidate points = black). We defined two 2 D FIR filters to detect vertical left and right edges in the new BW frame and applied the Hough transform to the filtered images (one per each left and right edges) to seek potential lines’ edges. In order to overcome noise problems created by the wheelchair’s continuous move- ment, we designed a robust tracking system that uses two Kalman filters (one per each left and right line edges), and a parameter classification algorithm, able to determine if the two edges of a candidate line are indeed the edges of the course line, based on the dis- tance between edges. The desired ROI is then further reduced to 40 pixels to the sides of the detected line (depicted as a square in Figure 2). When the candidate edges are classified as “no line”, the ROI is increased by 5 pixels to the sides at each sample time until a correct tracking line is detected. The camera w as mounted in front of the wheelcha ir and tilted with respect to vertical, and thus images from the webcam were perturbed by perspective: parallel lines in the real world appeared as converging lines in the image plane. We restored t he image to i ts original undi storted 3-D coordinates, which required the knowl- edge of the camera parameters, such as height, tilt angle, a nd focal length, which w ere calculated through the camera calibration [24]. Figure 2 Image from the camera. The image from the camera is 240 × 320, however we define a region of interest (ROI) of 80 × 320 (area between dashed blue lines) around the look-ahead distance (depicted as a horizontal white line). The final ROI is calculated through tracking algorithms and depicted as a square around the line. Marchal-Crespo et al. Journal of NeuroEngineering and Rehabilitation 2010, 7:40 http://www.jneuroengrehab.com/content/7/1/40 Page 4 of 12 In order to control the wheelchair movement, the power wheelchair has a low level controller (Pilot+, Penny & Giles) that translates the signals sent by the default commercial jo ystick into the two independent electrical motors/brakes. In order for our central com- puter to communicate with the Pilot+ controller, we added an OMNI+ interface which accepts signals from many different types of input devices (such as ana log joysticks, and 5 switch input devices) and translates them into commands compatible with the Pilot+ con- troller. The analog signals required to be translated through the OMNI+ special interface are computer gen- era ted, and are generally proportiona l to the position of the joystick handler. The pseudo analog joystick signals are converted to analog signals through a low-cost A/D card (Labjack U12) at up to 50 Hz per channel. For additional safety, we incorporated five low cost infrared proximity detectors (Sharp GP2D120), on the front of the wheelchair. These sensors take continuous distance readings and send the m to an Arduino Dieci- mila minicontroller which sends a digital signal to the OMNI+ interface when an obstacle is detected in order to safely stop the wheelchair. The Driving Task: Robot Tag To motivate the children during training, we pro- grammed a small mo bile robot to follow the same black line on the floor, and requested the child to try to catch itinagameof“robot tag” . If a child s teered off t he black line, trying to take a shortcut, the smar t wheel- chair halved its speed, whereas the speed of the small robot was kept constant (controllable through a remote). We also vibrated the joystick, to reinforce the acquisi- tion of the cause-effect relationship between the drive cutting the corners and the wheelchair slowin g down, and the joystick vibrating. However, we note that the joystick vibration is a kind of haptic assistance input. Thus, when practicing without assistance, both haptic guidance, and haptic vibration sensory inputs were disabled. The small robot is caught when the wheelchair vision system detects the red tag on the small robot (Figure 1) through Y’ CbCr color segmentation. When the robot is caught, the wheelchair stops for 10 seconds, plays an amusing sound on the laptop, and sends a signal to the small robot through a wireless transmitter, which makes the small robot stop and perform a funny “dance” while beeping. Ergonomic modifications to account for a child with CP We slightly modified the smart wheelchair system to account for the child with special needs due to cerebral palsy. Specifically, we located the camera overhead in order to facilitate transferring the child to the chair. The camera height change relative to the floor increase d the field of vision (FOV) by 80 % and a came ra recalibration was required. The c hild who volunteered in the case study reported here had a severe limitation in her hand range of motion, and thus we moved the joystick from the side to the front of her body to facilitate the joystick handle grasping. Furthermore, me reduc ed the handle height by 50%, and the side to side range of motion of thejoystickby40%.ThechangeofFOVandthe reduced range of motion of the joystick required a change in the controller g ains, meaning that the c hild did not e xperience the same control law as the childr en in the “Guidance” group, although it was quite similar. The increase in FOV facilitated more freedom around the line, and thus the child with impairment was able to experience larger errors than the non-disabled children. Experimental Protocol To determine if the robotic wheelchair trainer could help children learn to steer the wheelchair while limiting errors, we p erformed an experim ent with 22 non-dis- abled children (aged 3-9, mean 6.6 ± 1.5 SD), and a child with a severe motor impairment due to cerebral palsy. All experiments were approved by the Institu- tional Review Board of the University of California at Irvine, and subjects provided informed consent. Non- disabled ch ildren were randomly assigned into two, age- matched groups of 11 members each. Children in the “No Guidance” group (average age 6.96 ± 1.33 SD) were instructed to drive without any guidance from the roboticjoystickduring10minutes,tryingtokeepa laser pointer (pointing to the ground just below the child’s feet) on the black line that defined the 19 m long driving circuit (Figure 3, down). Children in the “Gui- dance ” group (average age 6.43 ± 1.47 SD) drove during the first 50 seconds without robotic guidance, followed by 9 trials (450 seconds) with a form of guidance that was systematically decreased by reducing the joystick’ s mechanical impedance (Figure 3, Top), and two l ast trials of 50 seconds without guidance. The child with a severe motor impair ment who per- formed the experiment is a bright but severely physically impaired 8-year-old girl as a result of C erebral Palsy at birth. She had low tone in her trunk and could not use her upper extremities well. She had not self-initiated mobility when very young, and she did not pass the cut off points on the Powered Mobility Readiness test [25] until she was 4 1/2. Initially she used switches to learn to drive her po wer wheelchair for the first few months to learn control of direction’ as using a proportional joy- stick was too demanding and overwhelming with her processing impairments. At the time of the study she used a center mount proportional joystick to drive her power wheelchair at home. The child with the motor Marchal-Crespo et al. Journal of NeuroEngineering and Rehabilitation 2010, 7:40 http://www.jneuroengrehab.com/content/7/1/40 Page 5 of 12 impairment replicated the single-session training proto- col that the non-disabled children in the “ Guidance” group participated in. Data and statistical analysis One participant in the Guidance group did not finish the experiment because she felt afraid, so data was ana- lyzed for 10 subjects only. At each time sample, the tracking error, speed of the chair, and the value of the guidance control gains K j and B j , were measured. To determine whether the guidance reduced the track- ing error and increased speed when it was first introduced, we performed a paired t-test in the Guidance group com- paring the mean errors and speed in the first experiment’s trial with the error created during the second trial (when guidance was first applied). We performed an independent samples t- test to compare the error created during trials between groups. To test the training effectiveness of the guidance strategy, we compared the tracking error and speed between the first trial and the last trial, which were both without guidance, through a paired t-test. To deter- mine whether guidance improved learning compared to no guidance, we used an independent samples t-test to compare the final mean distance error between the two groups. We tested with an independent sample t-test if either of the two strategies was more effective at reducing errors from trial 1 to the last trial without guidance. We also tested with an independent sample t-test if the child with the motor impairment reduced errors from trial 1 to the last trial without guidance by an amount similar than the children without motor impairment, in any of the two guidance strategies. The significance level was set to 0.05 for all tests. Results Guidance significantly reduced tracking error and increased speed of non-disabled children when applied during training Twenty-two non-disabled children (aged 3-9) attempted to drive the smart powered wheelchair trainer around a 19 m circuit defined by a blac k line, in order to catch a small mobile robot moving ahead of them along the line, in a game of “robot tag”. The chair slowed if they moved too far away from the black line. Half of the chil- dren trained without any haptic g uidance, while half experienced faded haptic guidance throughout the train- ing laps. At the end of the training session, we measured improvements in unassisted line tracking error, com- pared to at the beginning of the training session. The robotic assistance provided by the smar t wheel- chair’s robotic joy stick was effective in reducing steering errors while it was applied, as evidenced by the fact that faded guidance reduced the tracking error on the first trial when guidance was applied, compared to the initial trial w ithout guidance (Figure 4A, t-test, p <0.001).It also resulted in better steering performance across the trials it was applied when compared to the no guidance group (individual trials 2-6, p < 0.01, and individual trials 7-10 not significant, p < 0.14). Similarly, the gui- dance increased the driving speed on the first trial when guidance was applied, compared to the initial trial with- out guidance (Figure 4B, t-test, p < 0.001), and resulted in faster driving across the initial trials it was applied when compared to the no guidance group (trials 2-4, p < 0.01). Because the guidance was faded gradually, when the robotic guidance was removed in trial 11, there was not a significant increment in error or decrement of speed when compared to the last trial with guidance. Training with guidance improved unassisted steering performance of non-disabled children Non-disabled participants in the guidance group improved their unassisted steering performance follow- ing training with faded guidance. The guidance group Figure 3 Control gain and driving course.Top:ControlgainK j used for each subject. Subjects drove through the circuit during 50 seconds without robotic guidance from the robotic joystick followed by 450 seconds of robotically guided training, and 100 seconds without guidance. Down: Picture of the 19 m long driving course. Marchal-Crespo et al. Journal of NeuroEngineering and Rehabilitation 2010, 7:40 http://www.jneuroengrehab.com/content/7/1/40 Page 6 of 12 showed better performance characterized by a signifi- cant reduction of the tracking error from trial 1 to last trial unassisted (trial 12) (Figure 5A, t-test, p = 0 .05), and a significant increase of the driving speed (Figure 5B, t-test, p = 0.003). In the no guidance group, bot h the t racking error and driving speed remained without significant changes from trial 1 to last trial 12 (Figure 5A, B). Training non-disabled children with haptic guidance produced better performance at the end of the training session than non-guided training Non-disabled participants who trained with physical gui- dance improved their steering performance more than subjects who trained without guidance. The faded gui- dance group showed a larger performance improvement characterized by a greater reduction of the tracking 2 4 6 8 10 12 8 10 12 14 16 18 20 22 24 A: Tracking Error (cm) Trial Guidance Group No Guidance Group * Best achievable error Guidance on during 8 minutes 2 4 6 8 10 12 0.2 0.25 0.3 0.35 0.4 B: Speed (m/s) Trial Guidance Group Guidance on during 8 minutes No Guidance Group Figure 4 Average tracking errors and mean speed during training of 22 non-disabled children aged 3-9. Children in the Guidance group did not receive assistance on trials 1, 11 and 12, and received faded guidance during trials 2-10. Children in the No Guidance group did not receive assistance during training. A: Tracking error during each 50 s trial. Note that the tracking error was significantly reduced when guidance was applied at trial 2. When guidance was removed during the last 2 trials, children who trained with guidance followed the line better than children who never received guidance. B: Mean speed during each trial. Note the increase of speed in the guidance group when guidance was applied at trial 2. Error bars in all plots show ± 1 SD. *p < 0.05, t-test. No GuidanceGuidance Speed increase (m/s) 0.08 0.06 0.04 0.02 0.00 B:Mean Speed * * No GuidanceGuidance Tracking error reduction (cm) 3.00 2.00 1.00 0.00 -1.00 -2.00 A: Tracking Error * * Figure 5 Tracking error and speed increase from initial trial to last trial. A: Non-disabled subjects in the Guidance group significantly reduced more the tracking error than subjects who trained without guidance. B: Non-disabled subjects in the Guidance group significantly increased the speed after training, and there was a non-significant tendency of a greater speed increase in the guidance group (p = 0.1413). Error bars in all plots show +/- 1 SD. *p < 0.05. Marchal-Crespo et al. Journal of NeuroEngineering and Rehabilitation 2010, 7:40 http://www.jneuroengrehab.com/content/7/1/40 Page 7 of 12 error from trial 1 to the last unassisted trial (trial 12) (Figure 5A, t-test, p = 0.031) compared to the non-gui- dance group, and a significant tendency of driving faster after training (Figure 5B, 1 tailed t-test, p = 0.05). The final tracking error (on trial 12) for the guidance group, was significantly less than the final tracking error for the groupthatlearnedwithoutguidance(Figure4A,t-test, p = 0.05). The guidance-trained group showed a faster speed after training, but the difference was not signifi- cant (Figure 4B, t-test, p = 0.1413). Effect of age on initial performance We found a significant linear relationship between initial steering skill level and age. Very young ch ildren system- atically performed worse than older children when steer- ing the power wheelchair through the circuit, creating large errors and systematically losing the black line. Very young children especially had problems command- ing the direction and speed of the wheelchair simulta- neously, resulting in large tracking errors (Figure 6 top, Pearson’ s coefficient, r =0.795,p <0.001),andslower speed (Figure 6 bottom, Pearson’s coefficient, r =0.702, p < 0.001). A child with a severe motor impairment due to CP can benefit in the short-term from haptic guidance during driver’s training One 8-year-old child with aseveremotorimpairment due to cerebral palsy (CP) replicated the single-session training protocol performed by the non-disabled chil- dren in the “Guidanc e” group with small ergonomic changes of the system (see Methods). At the end of the training session, we measured the impro vement in the non-assisted line tracking error, and compared it to the relative improvements of the non-disabled children in the “Guidance” and “No Guidance” groups. The tracking errors created by the child with CP dur- ing the training protocol follow a similar patter as those created by the non-disabled children in the “Guidance” group (Figure 7A). The error was reduced when the assistance was introduced in the second trial, and it increased systematically as the guidance was faded. Whentheguidancewasremoved,thetrackingerror remained smaller than the tracking error in first trial. As described in the Methods section, we moved the webcam to an overhead location to facilitate the child with special needs sitting transfer. This change on the camera height increased the FOV, and thus allowed the child with cerebral palsy to experience larger errors around the line. Hence, it was not possible to compare the initial and final tracking errors be tween the child with CP and the non-disabled children. However, we found that the child with CP improved her steering abil- ity after training with guidance from the joystick by a percentage greater than the children without motor impairment both in the “Guidance” group (Figure 7B, 1 sided t-test p =0.05)andinthe“ No Guidance” group (Figure 7B, t-test, p = 0.02). There were no significant differences in the driving speed change from trial 1 to 12 between the child w ith CP and non-disable children in any guidance groups. Discussion We developed a smart wheelchair on which young chil- dren can safely learn and develop driving skills at their own pace with minimum assistance from a therapist. We implemented a vision system able to detect a line on the floor, track it and calculate the position of the wheelchair with respect to the line. We also developed an al gorithm that can demonstrate (through mo vements from a force feedback joystick) exemplary control to fol- low the course, while systematically modulating the strength and sensitivity of the haptic guidance. We 9876543 .40 .35 .30 .25 .20 .15 .10 Mean Tracking Error at trial 1(cm) 26 24 22 20 18 16 14 12 10 8 AGE 9876543 AGE Mean Velocity at trial 1(m/s) Figure 6 Initial performance improves with age. Top: There is a linear correlation between age and tracking error during trial 1. Down: Linear relationship between age and speed at trial 1. Marchal-Crespo et al. Journal of NeuroEngineering and Rehabilitation 2010, 7:40 http://www.jneuroengrehab.com/content/7/1/40 Page 8 of 12 designed an engaging training game using this technol- ogy, in which the driver tries to catch a small mobile robot moving ahead of him or her on the course. In a pilot study with non-disabled children, we found that learning to drive a power wheelchair with faded guidance did not hamper learning, but indeed promoted leaning of the steering task, within a sing le traini ng ses- sion. Furthermore, training with guidance was more effective than training without guidance. Final tracking errors in the guidance group were significantly lower thaninthenoguidancegroup.Theguidancegroup showed a greater increase of speed than the no guidance group. We also reported the results of a case study with one 8- year-old child with a severe motor impairment due to cerebral palsy trained with faded guidance. This child also improved steering ability after training with guidance from the joystick by an amount even greater than the children without motor impairment. We first discuss the implications of these results for wheelchair technology, motor learning research, and robot rehabilitation and then describe important directions for future research. Implications for wheelchair technology A powered wheelchair offers a means of independent mobility to individuals with disabilities [26]. However, some individuals with severe disabilities lack the neces- sary motor control, or cognitive skills to easily learn to drive a wheelchair, and therefore have no other practical option for independent mobility [27]. Examples of such populations include children with cerebral palsy (CP), our first target population, but also people with high- level spinal cord injury (SCI), multiple sclerosis (MS), brain injury (BI), and stroke. To accommodate these individuals’ mobility needs, there have been multiple attempts to develop “ Smart Wheelchairs” (e.g. [1,8,26,28]). These technologies usually aim at providing fully or semi-autonomous navigation. However, provi- sion of such a semi autonomous wheelchair could unin- tentionally prevent the development of new driving skills. Development of such skills could in turn simplify the technological requirements of the prescribed smart wheelchair, for example, by allowing chairs with obstacle avoidance but not advanced navigation computation and control, to be useful for more people. The approach described in this paper is designed to lower the cost and improve accessibility to training for individuals with severe sensory motor impairments who require intensive/long duration practice to become com- petent in powered mobility. The technology we devel- opedinthisstudycouldserveasanaffordablewayto allow individuals by themselves to attain some of the 2 4 6 8 10 12 10 15 20 25 30 35 A: Tracking Error (cm) Trial Guidance on during 8 minutes Guidance Group No Guidance Group CP Child CP ChildNo GuidanceGuidance 0.40 0.30 0.20 0.10 0.00 -0.10 B: Percentage error reduction * * Figure 7 Tracking errors during training of all subjects, 22 non-disabled children aged 3-9, and one 8-year-old child with a severe motor impairment due to cerebral palsy. Children in the Guidance group and the child with CP did not receive assistance on trials 1, 11 and 12, and received faded guidance during trials 2-10. Children in the No Guidance group did not receive assistance during training. A: Tracking error during each 50 s trial. Note that the tracking error was significantly reduced when guidance was applied at trial 2 in both, non-disable children and child with CP. When guidance was removed during the last 2 trials, children who trained with guidance followed the line better than at the beginning of the training session. B: Percentage of tracking error reduction from trial 1 to last trial. The child with CP significantly reduced more the tracking error than children without a motor impairment who trained without guidance, and showed a tendency of larger reduction than children without a motor impairment trained with guidance (p = 0.104). Error bars in all plots show +/- 1 SD. *p < 0.05. Marchal-Crespo et al. Journal of NeuroEngineering and Rehabilitation 2010, 7:40 http://www.jneuroengrehab.com/content/7/1/40 Page 9 of 12 skills necessary to safely drive a standard powered wheel- chair. We hypothesize that many people who are c ur- rently unable to drive a wheelchair can learn to drive in a structured environment given proper intensive training. The joystick used in this study cost $4K. While this may be an acceptable cost for a training device that gets many hours of use by multiple users, it would b e even more desirable to use a lower cost joystick. We have done preliminary evaluations on less expensive joysticks including the Microsoft Sidewinder Force Feedback and the The Novint Falcon. The Microsoft joystick proved to be too weak for the application, and the Falcon joy- stick’ s sensitivity was low and the communication speed two slow for fine control of the desired position of the handle. More research on how to adapt very low cost joysticks is needed. We highlight that the focus of our work so far is learning to drive in a structured environment, which is an important first step f or three reasons. First, learning to drive in a structured environment allows experience of dynamic self-initiated movement, a critical aspect of advancement of cognitive, perceptual, and motor abil- ities [1-4]. This technology has the potential to make the experience of dynamic self-initiated movement more widely accessible. Second, learning to drive in a struc- tured environment in the clinic could enhance the use of smart wheelchair technology outside the clinic [1,8,26,28]. With simple modifications added to the home or school (e.g. a line on the floor between play areas), smart wheelcha ir technology would allow new driving skills to be used outside the clinic. Third, suc- cess at learning to drive in a structured environment is a necessary precursor for learning to drive indepen- dently in an unstructured environment. Implications for motor learning research These results extend our previous findings [6,22] about the benefits of physical guidance for enhanci ng learning of a steering task. Previo us work was performed using a virtual environment with adult subjects steering a robotic steering wheel. This work shows that haptic gui- dance provided by a joystick helps children develop a real-world steering skill. As explained in [22] a possible interpretation of these resultsisthatsubjectsperformingcomplextaskssuch as skiing [29], learning a complex spatiotemporal trajec- tory [10] and driving a vehicle learn t o anticipate the timin g of their movements better with cues provided by haptic guidance, such as the moment to begin a turn when encountering a sharp curve or the moment to rec- tify after a curve [6]. T he concept that guidance can improve the learning of anticipatory timing is also con- sistent with t he results of a recent experiment we per- formed [30], which showed a benefit of haptic guidance from a robot on less skilled participants in learning to play a time-critical task (pinball game). In the same line, recent work [10,31,32] found a benefit of haptic gui- dance from a robot in learning to reproduce the tem- poral, but not spatial, characteristics of a complex spatiotemporal curve. Thus, there is emerging eviden ce that haptic guidance may be specifically useful for learn- ing anticipatory timing of for ces in dynamic tasks. These results also have implications for the long-stand- ing Guidance Hypothesis from motor learning research, which states that providing too much guidance will inhi- bit motor learning because i t obviat es the motor system from learning the necessary motor control strategies to perform the desired task. Since guidance was provided on all training laps during the steering train ing, the question arises why this continuously-provided guidance was not “too much” , and thu s did not inhibit learning. The possible negative effects of guidance may have been reduced because we used a compliant, faded form of guidance (cf. [13]). The amount of guidance decreased as training progressed, offering the driver the ability to overpower the joystick, which perhaps encouraged the user to pay attention and t o develop appropriate motor control strategies. Alternately, the driving task itself may be peculiarly amenable to guidance-based training, and thereby forming an exception to the Guidance hypoth- esis, because it requires the learning of timing of forces. Fundamentally, these results show that a simplistic interpretation of the Guidance Hypothesis - that gui- dance categorically impairs learning - misses an impor- tant aspect of human motor behavior: training with appropriately designed physical assistance can enhance the ability o f the brain to learn some motor skills. The mechanisms of this benefit are still unclear. One possibi- lity is that guidance may demonstrate better movement strategies (such as the need to initiate turns earlier). Alternately, guidance may make difficult tasks more optimally challenging, and thus improve motor learning, as suggeste d by the Challenge Point Theory [9]. Haptic guidance may make the inp ut-output r elationship between joystick motion and wheelchair motion more intelligible to the youngest children, preventing the chair from wandering too far from the path, in which case complex joystick motions are needed to return to the path. These h ypotheses may help to explain the surprising finding that the nondisabled children did not become better at driving the wheelchair after training without guidance. Of course, given a longer training time, we believe it is likely that they would have improved their performance. However, even with the short training duration, the group that received guidance improved their performance. This indeed suggests that the non- guided group was perhaps stuck in a “local minimum”, Marchal-Crespo et al. Journal of NeuroEngineering and Rehabilitation 2010, 7:40 http://www.jneuroengrehab.com/content/7/1/40 Page 10 of 12 [...]... disabilities: A clinical survey Journal of Rehabilitation Research and Development 2000, 37(3):353-360 28 Simpson R, LoPresti E, Hayashi S, Guo S, Ding D, Ammer W, Sharma V, Cooper R: A prototype power assist wheelchair that provides for obstacle detection and avoidance for those with visual impairments Journal of NeuroEngineering and Rehabilitation 2005, 2:30 29 Wulf G, Shea CH, Whitacre C: Physical-guidance... task that was more appropriately challenging because of the haptic guidance, allowing them to learn more quickly Another interesting aspect of the results described here is that the tracking error increased (and speed was reduced) during the last few trials with guidance (Figure 4, trials 6-8), when the guidance has already been faded by more than 70% of its initial value (Figure 3) The faded guidance... on real-time measurement of tracking performance, may have better limited the amount of tracking errors during all trials where guidance was applied Such a “Guidance-as-needed” algorithm would slowly decrease the assistance on the drivers hands when tracking error is small, but would increase the assistance in response to larger tracking errors Implications for robot rehabilitation research We reported... learning a complex motor skill Journal of Motor Behavior 1998, 30(4):367 30 Milot M, Marchal-Crespo L, Cramer S, Reinkensmeyer D: Comparison of error amplification and haptic guidance training techniques for learning of a timing-based motor task by healthy individuals Experimental Brain Research 2010, 201(2):119-31 31 Bluteau J, Coquillart S, Payan Y, Gentaz E: Haptic guidance improves the visuo-manual... and edited and revised the manuscript All authors read and approved the manuscript Competing interests The authors declare that they have no competing interests Received: 3 March 2010 Accepted: 13 August 2010 Published: 13 August 2010 References 1 Galloway J, Ryu J, Agrawal S: Babies driving robots: self-generated mobility in very young infants Intel Serv Robotics 2008, 1:123-134 2 Bertenthal B, Campos... critical reappraisal Psychol Bull 1984, 95(3):355-86 16 Armstrong TR: Training for the production of memorized movement patterns Human Performance Center Tech Rep 26, University of Michigan, Ann Arbor 1970 17 Tsutsui S, Imanaka K: Effect of manual guidance on acquiring a new bimanual coordination pattern Research Quarterly for Exercise and Sport 2003, 74:104-109 18 Gillespie RB, O’Modhrain MS, Tang P, Zaretzky... consent was obtained from the patient for publication of this case report and accompanying images A copy of the written consent is available for review by the Editor-in-Chief of this journal Acknowledgements Support for this project was provided by Field-Initiated Grant H133G09011 from the National Institute on Disability and Rehabilitation Research, Department of Education The authors would like to thank... Rehabilitation Center, Downey, CA, USA Authors’ contributions LMC designed and developed the smart wheelchair system, run the study, performed the statistical analysis and draft the manuscript JF participated in the design of the experimental setup with the child with disability, helped in the recruitment of subjects, and helped to improve the system to accommodate children with special needs DJR and. .. McNeal for helpful discussions, Andrea Reinkensmeyer for her help recruiting young subjects for the study, and Sunrise Medical for donating the OMNI+ interface to us Author details 1 Mechanical and Aerospace Engineering Department, University of California, Irvine, CA, USA 2Rehabilitation Engineering Research Center on Technology for Children With Orthopedic Disabilities, Rancho Los Amigos National... results of a case study with one 8-yearold child with a severe motor impairment due to cerebral palsy trained with faded guidance This child also improved steering ability after training with guidance from the joystick by an amount even greater than the children without motor impairment This case study indicates that a child with a impaired motor system can also benefit from haptic guidance during . this article as: Marchal-Crespo et al.: A robotic wheelchair trainer: design overview and a feasibility study. Journal of NeuroEngineering and Rehabilitation 2010 7:40. Marchal-Crespo et al. Journal. RESEA R C H Open Access A robotic wheelchair trainer: design overview and a feasibility study Laura Marchal-Crespo 1* , Jan Furumasu 2 , David J Reinkensmeyer 1 Abstract Background: Experiencing. detection algorithms and the Hough transformation, tracks the lin e using a Kalman filter, and calculates the look-ahead dis- tance of the wheelchair to the line (e dis ), and the direction of the wheelchair