Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera
3.2.3. Image Pair Rectification and Edge Detection
Rectification of stereo images is an important part of determining disparity. In order to more efficiently perform stereo matching for the disparity, images planes need to be rectified.
Distorted left and right images
Rectified left and right images
Edge left and
right images Depth image
Figure 3.8: Block diagram of processing distorted images to produce a depth image.
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
In order to collect a complete stereo disparity map, some issues related to image processing follow the stages shown in Figure 3.8.
• Stage 1: From the lens distorted left and right images, a rectified algorithm is proposed to correct lens distortion and to align images.
• Stage 2: Given the rectified images, an approach of Laplacian of Gaussian is applied to filter and to remove intensity bias to create the image edges.
• Stage 3: Based on the block matching method, the SAD) correlation criteria is employed for estimating disparity through correlating pixels from the left and right image edges.
3.2.3.1. Image Pair Rectification
There are many reasons as to why the pair of two images captured from a stereoscopic camera system are distorted, however the primary reason lies is the internal parameters and lens. Distorted lens images influence the disparity of two images. For this reason, image pairs are transformed, allowing epipolar lines to be horizontally aligned. In this case, the stereo matching algorithm can take easy advantage of the epipolar constraint, reducing the search space to one dimension. This means that the rows of the image pairs are rectified. The key initiative of the image rectification method consists of reparameterizing around the image epipoles.
B
P
'
pL '
pR
line Epipolar
CL ' CR
IL '
IR
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
As shown in Figure 3.7, given a unique geometry, assume if the corresponding image point pL of any image point pR on the image planes may be found along horizontal scan- lines corresponding to the image points p’L and p’R on the rectified image planes I’L and I’R as shown in Figure 3.9.
The first step consists of determining the common region for both images. Then, starting from one of the extreme epipolar lines, the rectified image is built up line by line. If the epipole is in the image an arbitrary epipolar line can be chosen as a starting point. In this case, boundary effects are avoided by adding a size overlap of the matching window of the stereo algorithm. The distance between consecutive epipolar lines is determined independently for every half epipolar line, so that no pixel compression occurs. This non-linear warping allows minimal achievable image size without losing image information.
The rectified images are built row by row. Each row corresponds to a certain angular sector. The length along the epipolar line is then preserved. The coordinates of every epipolar line are saved in a list for later reference. This follows that the transformation will be back to original images. The distance of the first and the last pixels are recalled for every epipolar line, this information allowing a simple inverse transformation through a constructed look-up table.
Distorted edge
Distorted edge
Figure 3.10a: Distorted left image. Figure 3.10b: Distorted right image.
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
Often, lenses will cause distortion to raw images. For example, in the raw images the straight lines appear curved. This effect is particularly evident in the image corners. For example, the left and right images in Figures 3.10a and 3.10b have distorted images showing edges as curved lines.
Rectified edge
Rectified edge
Figure 3.11a: Rectified left image with white and black color.
Figure 3.11b: Rectified right image with white and black color.
Rectification is the process of correcting input images for the lensed distortion. Further, rectified images are corrected so that the image rows are horizontally aligned. For similarity, the image columns will produce the vertical rectified results. Figures 3.11a and 3.11b show the image rectified as black and white.
Rectified edge
Rectified edge
Figure 3.12a: Left image with color rectified. Figure 3.12b: Right image with color
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
As shown in Figures 3.12a and 3.12b, the left and right colour images have been corrected. In these images, edges are rectified as straight lines. The rectified images provide accurate information for computing a 3D point map, as well as a 2D distance map. These maps can be applied to compute autonomous wheelchair control strategies.
3.2.3.2. Edge Detection
TV edge Board edge Chair edge TV edge Board edge Chair edge
Figure 3.13a: Edge left image. Figure 3.13b: Edge right image.
Edge detection is an optional feature allowing matching for the brightness changes rather than the absolute values of the image pixels. This is a useful feature, as the camera has auto gain control. If the auto gains in the camera do not change identically, absolute brightness between images may differ. Even though absolute brightness may not be the same, the intensity change remains constant. Therefore, using edge detection can assist in environments where the lighting conditions significantly change. To demonstrate the effectiveness of these approaches, Figures 3.13a and 3.13b represent the edges of obstacles and freespace such as chairs, boxes, boards and television edges.
This section presents then the rectification of two images, and edge detection. In particular, the epipolar lines in two image planes are transformed as horizontal alignment. This is very important for computing the disparity between two images. To this end, matching the algorithms to determine disparity is simpler if processing pixel columns without raws in the images. Edge detection also plays an important role for
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
matching the brightness change in the environment. Figures from 3.10 to 3.13 show the results from experiments with rectified distorted images, as well as edge detection.