Sensing Intelligence Motion - How Robots & Humans Move - Vladimir J. Lumelsky Part 13 docx

336 HUMAN PERFORMANCE IN MOTION PLANNING computer mouse. Every time the cursor approaches a labyrinth wall within some small distance—that is your “radius of vision”—the part of the wall within this radius becomes visible, and so you can decide where to turn to continue the motion. Once you step back from the wall, that piece of the wall disappears from the screen. Your performance in this new setting will of course deteriorate compared to the case with complete information above. You will likely wander around, hitting dead ends and passing some segments of the path more than once. Because you cannot now see the whole labyrinth, there will be no hope of producing a near- optimal solution; you will struggle just to get somehow to point T .Thisis demonstrated in two examples of tests with human subjects shown in Figure 7.3. Among the many such samples with human subjects that were obtained in the course of this study (see the following sections), these two are closest to the best and worst performance, respectively. Most subjects fell somewhere in between. While this performance is far from what we saw in the test with complete information, it is nothing to be ashamed of—the test is far from trivial. Those who had a chance to participate in youth wilderness training know how hard one has to work to find a specific spot in the forest, with or without a map. And many of us know the frustration of looking for a specific room in a large unfamiliar building, in spite of its well-structured design. Human Versus Computer Performance in a Labyrinth. How about com- paring the human performance we just observed with the performance of a decent motion planning algorithm? The computer clearly wins. For example, the Bug2 algorithm developed in Section 3.3.2, operating under the same conditions as for the human subjects, in the version with incomplete information produces elegant solutions shown in Figure 7.4: In case (a) the “robot” uses tactile information, and in case (b) it uses vision, with a limited radius of vision r v , as shown. Notice the remarkable performance of the algorithm in Figure 7.4b: The path produced by algorithm Bug2, using very limited input information—in fact, a fraction of complete information—almost matches the nearly optimal solution in Figure 7.2a that was obtained with complete information. We can only speculate about the nature of the inferior performance of humans in motion planning with incomplete information. The examples above suggest that humans tend to be inconsistent (one might say, lacking discipline): Some new idea catches the eye of the subject, and he or she proceeds to try it, without thinking much about what this change will mean for the overall outcome. The good news is that it is quite easy to teach human subjects how to use a good algorithm, and hence acquire consistency and discipline. With a little practice with the Bug2 algorithm, for example, the subjects started producing paths very similar to those shown in Figure 7.4. This last point—that humans can easily master motion planning algorithms for moving in a labyrinth—is particularly important. As we will see in the next section, the situation changes dramatically when human subjects attempt motion planning for arm manipulators. We will want to return to this comparison when PRELIMINARY OBSERVATIONS 337 (a) (b) Figure 7.3 Two examples of human performance when operating in the labyrinth of Figure 7.1 with incomplete information about the scene. Sample (a) is closer to the best performance, while sample (b) is closer to the worst performance observed in this study. 338 HUMAN PERFORMANCE IN MOTION PLANNING (a) (b) S T S T r u Figure 7.4 Performance of algorithm Bug2 (Chapter 3) in the labyrinth of Figure 7.1. (a) With tactile sensing and (b) with vision that is limited to radius r v . PRELIMINARY OBSERVATIONS 339 discussing the corresponding tests, so let us repeat the conclusion from the above discussion: When operating in a labyrinth, humans have no difficulty learning and using motion planning algorithms with incomplete information. 7.2.2 Moving an Arm Manipulator Operating with Complete Information. We are now approaching the main point of this discussion. There was nothing surprising about the human perfor- mance in a labyrinth; by and large, the examples of maze exploration above agree with our intuition. We expected that humans would be good at moving in a labyrinth when seeing all of it (moving with complete information), not so good when moving in a labyrinth “in the dark” (moving with incomplete information), and quite good at mastering a motion planning algorithm, and this is what hap- pened. We can use these examples as a kind of a benchmark for assessing human performance in motion planning. We now turn to testing human performance in moving a simple two-link revolute–revolute arm, shown in Figure 7.5. As before, the subject is sitting in O 2 J 1 S T P Θ 2 O 3 O 4 O 1 J 0 Θ 1 l 1 l 2 Figure 7.5 This simple planar two-link revolute–revolute arm manipulator was used to test human performance in motion planning for a kinematic structure: l 1 and l 2 are two links; J 0 and J 1 are two revolute joints; θ 1 and θ 2 are joint angles; S and T are start and target positions in the test; P is the arm endpoint in its current position; O 1 , O 2 , O 3 ,and O 4 are obstacles. 340 HUMAN PERFORMANCE IN MOTION PLANNING front of the computer screen, and controls the arm motion using the computer mouse. The first link, l 1 , of the arm rotates about its joint J 0 located at the fixed base of the arm. The joint of the second link, J 1 , is attached to the first link, and the link rotates about point J 1 , which moves together with link l 1 . Overall, the arm looks like a human arm, except that the second link, l 2 , has a piece that extends outside the “elbow” J 1 . (This kinematics is quite common in industrial and other manipulators.) And, of course, the arm moves only in the plane of the screen. How does one control the arm motion in this setup? By positioning the cursor on link l 1 and holding down the mouse button, the subject will make the link rotate about joint J 0 and follow the cursor. At this time link l 2 will be “frozen” relative to link l 1 and hence move with it. Similarly, positioning the cursor on link l 2 and holding down the mouse button will make the second link rotate about joint J 1 , with link l 1 being “frozen” (and hence not moving at all). Each such motion causes the appropriate link endpoint to rotate on a circular arc. Or—this is another way to control the arm motion—one can position the cursor at the endpoint P of link l 2 and drag it to whatever position in the arm workspace one desires, instantaneously or in a smooth motion. The arm endpoint will follow the cursor motion, with both links moving accordingly. During this motion the corresponding positions of both links are computed automatically in real time, using the inverse kinematics equations. (Subjects are not told about this mechanism, they just see that the arm moves as they expect.) This second option allows one to control both links motion simultaneously. It is as if someone moves your hand on the table —your arm will follow the motion. We will assume that, unlike in the human arm, there are no limits to the motion of each joint in Figure 7.5. That is, each link can in principle rotate clockwise or counterclockwise indefinitely. Of course, after every 2π each link returns to its initial position, so one may or may not want to use this capability. [Looking ahead, sometimes this property comes in handy. When struggling with moving around an obstacle, a subject may produce more than one rotation of a link. Whether or not the same motion could be done without the more-than-2π link rotation, not having to deal with a constraint on joint angle limits makes the test psychologically easier for the subject.] The difficulty of the test is, of course, that the arm workspace contains obsta- cles. When attempting to move the arm to a specified target position, the subjects will need to maneuver around those obstacles. In Figure 7.5 there are four obsta- cles. One can safely guess, for example, that obstacle O 1 may interfere with the motion of link l 1 and that the other three obstacles may interfere with the motion of link l 2 . Similar to the test with a labyrinth, in the arm manipulator test with complete information the subject is given the equivalent of the bird’s-eye view: One has a complete view of the arm and the obstacles, as shown in Figure 7.5. Imagine you are that subject. You are asked to move the arm, collision-free, from its starting position S to the target position T . The arm may touch an obstacle, but the system PRELIMINARY OBSERVATIONS 341 will not let you move the arm “through” an obstacle. Take your time—time is not a consideration in this test. Three examples of performance by human subjects in controlled experiments are shown in Figure 7.6. 3 Shown are the arm’s starting and target positions S and T , along with the trajectory (dotted line) of the arm endpoint on its way from S to T . The examples represent what one might call an “average” performance by human subjects. 4 The reader will likely be surprised by these samples. Why is human perfor- mance so unimpressive? After all, the subjects had complete information about the scene, and the problem was formally of the same (rather low) complexity as in the labyrinth test. The difference between the two sets of tests is indeed dramatic: Under similar conditions the human subjects produced almost optimal paths in the labyrinth (Figure 7.2) but produced rather mediocre results in the test with the arm (Figure 7.6). Why, in spite of seeing the whole scene with the arm and obstacles (Figure 7.5), the subjects exhibited such low skills and such little understanding of the task. Is there perhaps something wrong with the test protocol, or with control means of the human interface—or is it indeed real human skills that are represented here? Would the subjects improve with practice? Given enough time, would they perhaps be able to work out a consistent strategy? Can they learn an existing algo- rithm if offered this opportunity? Finally, subjects themselves might comment that whereas the arm’s work space seemed relatively uncluttered with obstacles, in the test they had a sense that the space was very crowded and “left no room for maneuvering.” The situation becomes clearer in the arm’s configuration space (C-space, Figure 7.7). As explained in Section 5.2.1, the C-space of this revolute–revolute arm is a common torus (see Figure 5.5). Figure 7.7 is obtained by flattening the torus by cutting it at point T along the axes θ 1 and θ 2 . This produces four points T in the resulting square, all identified, and two pairs of identified C-space boundaries, each pair corresponding to the opposite sides of the C-space square. For reference, four “shortest” paths (M-lines) between points S and T are shown (they also appear in Figure 5.5; see the discussion on this in Section 5.2.1). The dark areas in Figure 7.7 are C-space obstacles that correspond to the four obstacles in Figure 7.5. Note that the C-space is quite crowded, much more than one would think when looking at Figure 7.5. By mentally following in Figure 7.7 obstacle outlines across the C-space square boundaries, one will note that all four workspace obstacles actually form a single obstacle in C-space. This simply means that when touching one obstacle in work space, the arm may also touch some other 3 The experimental setup used in Figure 7.6c slightly differs from the other two; this played no visible role in the test outcomes. 4 The term “average” here has no formal meaning: It signifies only that some subjects did better and some did worse. A more formal analysis of human performance in this task will be given in the next section. A few subjects did not finish the test and gave up, citing tiredness or hopelessness (“There is no solution here”, “You cannot move from S to T here” ). 342 HUMAN PERFORMANCE IN MOTION PLANNING S T S T S T (a) (b) (c) Figure 7.6 Paths produced by three human subjects with the arm shown in Figure 7.5, given complete information about the scene. PRELIMINARY OBSERVATIONS 343 TT TT S Figure 7.7 C-space of the arm and obstacles shown in Figure 7.5. obstacle, and this is true sequentially, for pairs (O 1 ,O 2 ), (O 2 ,O 3 ), (O 3 ,O 4 ), (O 4 ,O 1 ). No wonder the subjects found the task difficult. In real-world tasks, such interaction happens all the time; and the difficulties only increase with more complex multilink arms and in three-dimensional space. Operating the Arm with Incomplete Information. Similar to the test with incomplete information in the labyrinth, here a subject would at all times see points S and T , along with the arm in its current positions. Obstacles would be hidden. Thus the subject moves the arm “in the dark”: When during its motion the arm comes in contact with an obstacle—or, in the second version of the test, some parts of the obstacle come within a given “radius of vision” r v from some arm’s points—those obstacle parts become temporarily visible. Once the contact is lost—or, in the second version, once the arm-to-obstacle distance increases beyond r v —the obstacle is again invisible. The puzzling observation in such tests is that, unlike in the tests with the labyrinth, the subjects’ performance in moving the arm “in the dark” is on aver- age indistinguishable from the test with complete information. In fact, some subjects performed better when operating with complete information, while others 344 HUMAN PERFORMANCE IN MOTION PLANNING performed better when operating “in the dark.” One subject did quite well “in the dark,” then was not even able to finish the task when operating with a completely visible scene, and refused to accept that in both cases he had dealt with the same scene: “This one [with complete information] is much harder; I think it has no solution.” It seems that extra information doesn’t help. What’s going on? Human Versus Computer Performance with the Arm. As we did above with the labyrinth, we can attempt a comparison between the human and computer performance when moving the arm manipulator, under the same conditions. Since in previous examples human performance was similar in tests with complete and incomplete information, it is not important which to consider: For example, the performance shown in Figure 7.6 is representative enough for our informal comparison. On the algorithm side, however, the input information factor makes a tremendous difference—as it should. The comparison becomes interesting when the computer algorithm operates with incomplete (“sensing”) information. Shown in Figure 7.8 is the path generated in the same work space of Figure 7.5 by the motion planning algorithm developed in Section 5.2.2. The algorithm operates under the model with incomplete information. To repeat, its sole input information comes from the arm sensing; known at all times are only the arm S T M1 r u Figure 7.8 Path produced in the work space of Figure 7.5 by the motion planning algo- rithm from Section 5.2.2; M1 is the shortest (in C-space) path that would be produced if there were no obstacles in the workspace. PRELIMINARY OBSERVATIONS 345 positions S and T and its current position. The arm’s sensing is assumed to allow the arm to sense surrounding objects at every point of its body, within some modest distance r v from that point. In Figure 7.8, radius r v is equal to about half of the link l 1 thickness; such sensing is readily achievable today in practice (see Chapter 8). Similar to Figure 7.6, the resulting path in Figure 7.8 (dotted line) is the path traversed by the arm endpoint when moving from position S to position T . Recall that the algorithm takes as its base path (called M-line) one of the four possible “shortest” straight lines in the arm’s C-space (see lines M 1 ,M 2 ,M 3 ,M 4 in Figure 5.5); distances and path lengths are measured in C-space in radians. In the example in Figure 7.8, the shortest of these four is chosen (it is shown as line M1, a dashed line). In other words, if no obstacles were present, under the algorithm the arm endpoint would have moved along the curve M1; given the obstacles, it went along the dotted line path. The elegant algorithm-generated path in Figure 7.8 is not only shorter than those generated by human subjects (Figure 7.6). Notice the dramatic differ- ence between the corresponding (human versus computer) arm test and the labyrinth test. While a path produced in the labyrinth by the computer algorithm (Figure 7.4) presents no conceptual difficulty for an average human subject, they find the path in Figure 7.8 incomprehensible. What is the logic behind those sweeping curves? Is this a good way to move the arm from S to T ? The best way? Consequently, while human subjects can easily master the algorithm in the labyrinth case, they find it hard—in fact, seemingly impossible—to make use of the algorithm for the arm manipulator. 7.2.3 Conclusions and Plan for Experiment Design We will now summarize the observations made in the previous section, and will pose a few questions that will help us design a more comprehensive study of human cognitive skills in space reasoning and motion planning: 1. The labyrinth test is a good easy-case benchmark for testing one’s general space reasoning abilities, and it should be included in the battery of tests. There are a few reasons for this: (a) If a person finds it difficult to move in the labyrinth—which happens rarely—he or she will be unlikely to handle the arm manipulator test. (b) The labyrinth test prepares a subject for the test with an easier task, making the switch to the arm test more gradual. (c) A subject’s successful operation in the labyrinth test suggests that whatever difficulty the subject may have with the arm test, it likely relates to the subject’s cognitive difficulties rather than to the test design or test protocol. 2. When moving the arm, subjects exhibit different tastes for control means: Some subjects, for example, prefer to change both joint angles simulta- neously, “pulling” the arm endpoint in the direction they desire, whereas other subjects prefer to move one joint at the time, thus producing circular [...]... Task Task Task Task Task Task Task 1: 2: 3: 4: 5: 6: 7: 8: Virtual, visible, left-to-right Virtual, visible, right-to-left Virtual, invisible, left-to-right Virtual, invisible, right-to-left Physical, visible, left-to-right Physical, visible, right-to-left Physical, invisible, left-to-right Physical, invisible, right-to-left In addition to these tasks, a smaller study was carried out to measure the... subjects’ performance with the rightto-left direction of motion was significantly worse than their performance with the left-to-right direction of motion: Depending on the task, the mean length of path for the right-to-left direction is about two to five times longer than that for the left-to-right direction.10 10 This alone would make a smart robot conclude that we humans are terribly inconsistent: What... (right-to-left) task This calls for more refined statistical tests, with two separate direction-of -motion data sets These are summarized next 4 Here the Mann–Whitney test measures the effect of the interface factor using only the left-to-right (LtoR) data sets The results are shown in Table 7.6 Given the significance level p < 0.01, we reject the null hypothesis and conclude that in the left-to-right... for the (easier) left-to-right direction of motion Once the task became a bit harder, the difference disappeared: When moving the physical arm in the right-to-left direction, more often than not the subjects’ performance was significantly worse than when moving the virtual arm in the left-to-right direction, and more or less comparable to moving the virtual arm in the same right-to-left direction (see... point robot) or a part of its body (in case of the arm) is close enough to an obstacle, in which case a small part of the obstacle near the contact point becomes visible for the duration of contact The arm is visible at all times D Direction factor (for the arm manipulator test only), with two levels: 1 “Left-to-right” motion (denoted below LtoR), as in Figure 7.8 2 “Right-to-left” motion (denoted RtoL);... object-to -move factor, which distinguishes between moving a point robot in a labyrinth versus moving a two-link arm manipulator among obstacles The arm test is the primary focus of this study; the labyrinth test is used only as a benchmark, to introduce the human subjects to the tests’ objectives The complete list of factors, each with two levels (settings), is therefore as follows9 : A Object-to -move. .. Variable Path length Vis Invis U p-Level Vis Invis 8881.000 9264.000 4321.000 0. 6133 76 95 95 361 RESULTS—EXPERIMENT ONE agrees with our intuition—that an opposite is true in the point-in-the-labyrinth test: One performs significantly better with a bird’s-eye view of the labyrinth than when seeing at each moment only a small part of the labyrinth Apparently, something changes dramatically when one switches... Variable Path length Virt Phys U p-Level virt phys 2505.000 2055.000 927.0000 0 .134 619 48 47 the virtual group data and the physical group data This result agrees with the results above obtained for the combined LtoR and RtoL data 5 Here the Mann–Whitney U-test measures the effect of the interface factor using only the right-to-left (RtoL) task data sets The results are shown in Table 7.7 Given the significance... subjects move the arm on the computer screen) and “physical” tests (tests where subjects move the physical arm) In other words, the interface factor has a statistically significant effect on the length of paths produced by the subjects Furthermore, this effect is present whether or not the task is implemented in a visible or invisible environment, and whether or not the direction of motion is left-to-right... environment, and whether or not the direction of motion is left-to-right or right-to-left While the Mann–Whitney statistical test isolates the single factor we are interested in, the interface factor, its results do not reconcile easily with the observations summarized in Table 7.1 Namely, Table 7.1 shows that while in the easier (left-to-right) task the subjects performed better with the physical arm than with . each subject can be subjected to: Task 1: Virtual, visible, left-to-right Task 2: Virtual, visible, right-to-left Task 3: Virtual, invisible, left-to-right Task 4: Virtual, invisible, right-to-left Task. right-to-left Task 5: Physical, visible, left-to-right Task 6: Physical, visible, right-to-left Task 7: Physical, invisible, left-to-right Task 8: Physical, invisible, right-to-left In addition to these tasks,. only), with two levels: 1. “Left-to-right” motion (denoted below LtoR), as in Figure 7.8. 2. “Right-to-left” motion (denoted RtoL); in Figure 7.8 this would corre- spond to moving the arm from