Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera
3.2.2. Epipolar Geometry and the Fundamental Matrix
Epipolar geometry refers to the intrinsic projective geometry between two views of a scene, while using a stereoscopic camera system. The internal parameters and pose of the system are important parts of the epipolar geometry. The fundamental matrix plays a major role in the corresponding problem of assessing the epipolar geometry of producing the first image point in one image plane, and having it corresponding to the second plane, from the scene point in a 3D space.
3.2.2.1. Stereoscopic camera system
As shown in Figure 3.5, the stereoscopic camera system consists of two cameras, this producing absolute symmetry. The stereoscopic camera is mounted on a power wheelchair for providing the left and right images in a given environment.
Figure 3.5: The “Bumblebee” stereoscopic camera system with “two eyes”
The baseline of the camera is parallel with the horizontal ground, the system being connected between the left and right cameras. The geometric relationship between any parts of the camera system can be kept constant during rotation and translation tasks (refer Figure 3.6).
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
Figure 3.6: Configuration of the dimensions of the “Bumblebee” stereoscopic camera system
3.2.2.2. Epipolar Geometry
As shown in Figure 3.7, the epipolar geometry from two views is the intrinsic projective geometry of image planes containing image points and epipoles. The baseline B contains the epipoles eR and eL and is connected by the centres of two cameras, CR and CL. In this model, assume that an obstacle point P lies in the world space, and the image points pR and pL lie in two image planes IR and IL respectively.
B
P
pL pR
eL
CL eR CR
IL IR
lL lR
Figure 3.7: Epipolar geometry of a pair of left and right images IL and IR.
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
From two images IL and IR, a left image point pL has a corresponding epipolar line lR in the other image. Similarly, an image point pR in the second image which matches pL produces a right epipolar line lR. It is clear that through the camera left centre CL of the first camera, the right epipolar line lR represents the projection of the ray in the right image from pL. A map therefore exists from an image point in one image to the corresponding epipolar line of the other image. This means that the mapping correlates projective mapping from image points and epipolar lines, using the fundamental matrix F.
3.2.2.3. The Fundamental Matrix
The epipolar geometry problem is concerned with the internal parameters of the camera system and its relative poses (Hartley and Zisserman 2003). In particular, the fundamental matrix F of the epipolar geometry relates to intrinsic geometry in a 3x3 matrix of rank 2. If a scene point P in the world space is imaged as the left image point pL in the first view, and the right image point pR in the second one, the image points therefore satisfy the relation pRT
FpL. The properties of the matrix are specified from the fundamental matrix, which is derived from the mapping between an image point and its epipolar line.
• If the image points pL and pR are corresponding image points, then L =0
TFp pR
for all corresponding points pL↔ pR.
• If F is the fundamental matrix of the pair of cameras (Q,Q’), then FT represents the fundamental matrix of two cameras (Q,Q’) in opposite order.
• The left image point pL in the first image has a corresponding epipolar line lR=FpL. Similarly, lL=FpR represents the epipolar line which corresponds to the right image point pR of the second image.
• Epipoles: The left image point pL corresponding to the epipolar line lR = FpL contains the epipole eR. Thus, the epipole eR satisfies (eTRF)pL=(eTLF)pR=0 for all pL. It follows that (eTRF)=0, if the epipole eR is the left null-vector of the
Chapter 3. Obstacle and Freespace Detection using a Stereoscopic Camera System
fundamental matrix F. In similarity, (eTLF)=0 if the epipole eL is the left null- vector of the fundamental matrix F.
• A 3x3 homogeneous matrix has eight independent ratios, however, the fundamental matrix F also satisfies the constraint deft=0 this removing one degree of freedom. Thus the fundamental matrix F has seven degrees of random.
• The fundamental matrix F is a correlation. This means that the left image point pL in the first image defines the epipolar line lR=FpL in the second one. In similarity, the right image point pR in the second image defines the epipolar line lL=FpR in the first one. Clearly, there is no inverse mapping, and the fundamental matrix F is not of full rank. For this reason, the fundamental matrix F is not a proper correlation.
Epipolar geometry and the fundamental matrix between a pair of images, these relating to intrinsic projective geometry, are applied for this thesis study. The geometry is used in stereo matching for searching for corresponding points. In particular, a point pL in the left image is used through the camera system to find corresponding point pR in the right image. The fundamental matrix forms a component of the correspondence between two images. From a point in one image to the corresponding epipolar line of the other image, a projective mapping of the images relates to fundamental matrix F. It is clear that epipoles and epipolar lines are determined via the fundamental matrix F.