Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera
3.3. Obstacle and Freespace Detection
3.3.4. Obstacle and Freespace Detection
Based on the left and right images, a 2D distance map uses a geometric projection for converting from a 3D point map. The 2D distance map represents the obstacles and freespace in populated environments. Examples of the obstacle and freespace detection are used to illustrate the effectiveness of this approach.
3.3.4.1. Experiment 1: Obstacle Detection
In the first experiment, many obstacles and freespace are detected as shown in Figures 3.27a and 3.27b. The left and right images show one freespace and various obstacles, including the box o2, the chair o3, the bar of freespace, the boards o1, o4 and the wall o5. From the left and right images, the SAD algorithm is used for producing a disparity map as shown in Figure 3.28.
o2
o5
o4
o3
Bar o1
o1
o2
o3
Bar
o4
o5
Figure 3.27a: Left image with many obstacles o1-o5 and a bar.
Figure 3.27b: Right image with many obstacles o1-o5 and a bar.
Figure 3.28: Disparity map with many obstacles at different positions corresponding
to different colours.
Figure 3.29a: 3D point map viewed from the front side including five obstacles and one
freespace.
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
Figure 3.29b: 3D point map viewed from the right side.
Figure 3.29c: 3D point map viewed from the left side.
The 3D point map includes many kinds of obstacles and a freespace, which can be viewed from various sides (refer Figures 3.29a, 3.29b and 3.29c).
0 1 2 3 4 5 6
-1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7
) (m Z
) (m X w
o1
o2
A B
o3
o4 o2
Figure 3.30: 2D distance map includes five obstacles and one freespace.
As shown in Figure 3.30, in this 2D distance map, the shapes and values of the obstacles and freespaces are approximately determined. The main advantage of the approach is that the 3D information is represented with depth (Z-axis), and the vertical (Y-axis) and horizontal (X-axis) coordinates. Due to its largest disparity, the box o2 is closest to the camera position. The figure also represents two points A and B at two positions of the freespace w. Given the two points A and B in the 2D distance map, the width w of the freespace is then determined.
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
3.3.4.2. Experiment 2: Estimation of the height of freespace
An experiment is performed to detect and estimate the height hf of the freespace, in which the height hf is greater than the height h of the wheelchair as shown in Figure 3.31. As such, the freespace in the 2D distance map is considered as a free space. In this case, the wheelchair can move through the freespace, as shown in Figures 3.32 to 3.35.
h
Figure 3.31: Wheelchair of height h.
Figures 3.32a and 3.32b show one freespace with the top bar. The computation of the freespace is applied so that if the height of the freespace is greater than that of the wheelchair, it can pass through it. In this case, the height of the freespace is greater than that of the wheelchair, so the bar of the freespace has disappeared (refer Figure 3.35).
h Bar
h Bar
Figure 3.32a: Left image with a freespace. Figure 3.32b: Right image with a freespace.
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
Figure 3.33: Disparity map. Figure 3.34: 3D point map.
The 2D map has a freespace w and two obstacles o1 and o2 shown in Figure 3.36.
0 1 2 3 4 5 6
-1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7
) (m Z
) (m X w
o1
o2
Figure 3.35: 2D distance map with one freespace.
Another experiment shows that the height hf of the detected freespace is less than that of the wheelchair as shown in Figures 3.36 to 3.39.
h
Bar Bar
h
Figure 3.36a: Left image with a freespace. Figure 3.36b: Right image with a freespace.
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
Figure 3.37: Disparity map. Figure 3.38: 3D point map.
The height hf of the freespace is less than that of the wheelchair, so the 2D distance map has no freespace, this being considered as an obstacle as shown in Figure 3.39. In this circumstance, the wheelchair is unable to pass through the freespace.
0 1 2 3 4 5 6
-1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7
) (m Z
) (m X w
o1
o2
Figure 3.39: 2D distance map with one freespace as the obstacle.
3.3.4.3. Experiment 3: Freespace Detection
Using a stereoscopic camera system mounted on a power wheelchair is a great advantage for recognising obstacles and freespace. Compared to the freespace, the wheelchair approximately estimates the same freespace at different positions of the wheelchair. The wheelchair is at the left position of the freespace, meaning that the stereoscopic camera can provide information about freespaces using the constant value, with different view positions. In this experiment, it is assumed there is a freespace at the constant position. A wheelchair fitted with the stereoscopic camera will measure this freespace at the left, central and right positions.
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
In particular, as shown in Figure 3.40, the wheelchair is at the left position of freespace.
From the left and right images, the disparity map and the 3D point map, as shown in Figures 3.41 to 3.44, converts the 2D distance map. At this position, the wheelchair measures the freespace width w, based on pixels (two points) A and B as shown in Figure 3.44.
Figure 3.40: Wheelchair at left position of the freespace.
Figure 3.41a: Left image with one freespace and bar.
Figure 3.41b: Right image with one freespace and one bar.
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
Figure 3.42: Disparity map. Figure 3.43: 3D point map.
0 1 2 3 4 5 6
-1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7
) (m Z
) (m X w
o1 o2
A B
Figure 3.44: 2D distance map with one freespace w and two obstacles o1 and o2. The wheelchair, as shown in Figure 3.45, is at the centre position of the freespace.
Figure 3.45: Wheelchair at the central position of the freespace.
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
As shown in Figures 3.46-3.48, based on the left and right images, the disparity map and the 3D point map, a 2D distance map is created as shown in Figure 3.47. The wheelchair at this position provides the width w of the freespace based on two point A and B as shown in Figure 3.49.
Figure 3.46a: Left image with one freespace and one bar.
Figure 3.46b: Right image with one freespace and one bar.
Figure 3.47: Stereo disparity map. Figure 3.48: 3D point map one freespace.
0 1 2 3 4 5 6
-1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7
) (m Z
) (m X w
A B
o1
o2
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
The wheelchair, as shown in Figure 3.50, is situated at the right of the freespace.
Figure 3.50: Wheelchair at the right position of the freespace.
From the left and right images, the disparity map is computed for producing a 3D point map, this shown in Figures 3.51 to 3.53. A 2D distance map is converted from the 3D point map, and the width w of the freespace, based on two points A and B. It is then measured as per Figure 3.54.
Figure 3.51a: Left image with one freespace and one bar.
Figure 3.51b: Right image with one freespace and one bar.
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
Figure 3.52: Disparity map with a dark blue freespace.
Figure 3.53: 3D point map with a freespace.
0 1 2 3 4 5 6
-1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7
) (m Z
) (m X w
o1 o2
A B
Figure 3.54: 2D distance map with one freespace w and two obstacles o1 and o2.
The wheelchair starts at three different positions, namely the left side, the central side and the right side, for obtaining the width w of the freespace. The experiment results in estimating the same values of the width of the freespace. In particular, the camera system captures the left and right images, with points A and B of the freespaces. These are illustrated in Figures 3.44, 3.49 and 3.54. Points A and B are pixels at the edge of the freespace these being closest to the camera position. The computation estimates the approximate width of the freespace in the 2D distance map, which corresponds to the different positions of the wheelchair. In conclusion, at some different positions of the wheelchair, the measured widths of the same freespace have the nearly same values.
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System