Xử lý ảnh trong cơ điện tử: Machine Vision. Chapter 9. Camera Calibration and 3D Reconstruction83

TRƯỜNG ĐẠI HỌC BÁCH KHOA XỬ LÝ ẢNH TRONG CƠ ĐIỆN Machine Vision Giảng viên: TS Nguyễn Thành Hùn Đơn vị: Bộ môn Cơ điện tử, Viện Cơ Hà Nội, 2021 Chapter Camera Calibration and 3D R Camera calibration Robot Camera Calibration Pose estimation Stereo vision Chapter Camera Calibration and 3D R Camera calibration Robot Camera Calibration Pose estimation Stereo vision Camera calibration ❖ Geometric camera calibration, also referred to as camera parameters of a lens and image sensor of an image or video ❖ You can use these parameters to correct for lens distortio object in world units, or determine the location of the came ❖ These tasks are used in applications such as machine vis objects They are also used in robotics, for navigation reconstruction Camera calibration Camera calibration ❖ Camera parameters include intrinsics, extrinsics, and distor ❖ To estimate the camera parameters, you need to have 3corresponding 2-D image points ❖ You can get these correspondences using multiple image such as a checkerboard Using the correspondences, you parameters Camera calibration ❖ After you calibrate a camera, to evaluate the accuracy of you can: ➢ Plot the relative locations of the camera and the calibration ➢ Calculate the reprojection errors ➢ Calculate the parameter estimation errors Camera calibration ❖ Pinhole Camera Model ➢ A pinhole camera is a simple camera without a lens and with a s Camera calibration ❖ Pinhole Camera Model Camera calibration ❖ Pinhole Camera Model ▪ The calibration algorithm calculates the camera matrix usin ➢ The intrinsic parameters ➢ The extrinsic parameters Scale factor Image points Intrinsic matrix Extrinsics Rotation and translation Camera Pose Estimatio ❖ Algorithm ➢ The algorithm for determining pose estimation is based on algorithm The main idea is to determine the correspondences be points on the 3D model curve (a) Reconstruct projection rays from the image points (b) Estimate the nearest point of each projection ray to a point on (c) Estimate the pose of the contour with the use of this correspo (d) goto (b) Camera Pose Estimation with Chapter Camera Calibration and 3D R Camera calibration Robot Camera Calibration Pose estimation Stereo vision Stereo Vision ❖ What is stereo correspondence? ➢ only have 2D information when capture images ➢ the depth information is lost ➢ our brain takes two images and builds a 3D map using stereo vis ➢ capture two photos of the same scene using different view corresponding points to obtain the depth map of the scene → ste Stereo Vision ❖ What is stereo correspondence? Left images Stereo Vision ❖ What is stereo correspondence? ➢ The absolute difference between d1 and d2 is greater than the ab and d4 ➢ The camera moved by the same amount, there is a big diff distances between the initial and fnal positions → This happe object closer to the camera; the apparent movement decreases w from different angles ➢ This is the concept behind stereo correspondence: we captu knowledge to extract the depth information from a given scene Stereo Vision ❖ What is epipolar geometry? Stereo Vision ❖ What is epipolar geometry? Epipolar lines Epipolar lines Stereo Vision ❖ What is epipolar geometry? ➢ Our goal is to match the keypoints in these two images to extrac ➢ The way we this is by extracting a matrix that can assoc between two stereo images → the fundamental matrix ➢ The point at which the epipolar lines converge is called epipole ➢ If you match the keypoints using SIFT, and draw the lines tow left and right images, they will look like this: Stereo Vision ❖ What is epipolar geometry? Stereo Vision ❖ What is epipolar geometry? ➢ If two frames are positioned in 3D, then each epipolar line b intersect the corresponding feature in each frame and each of the ➢ This can be used to estimate the pose of the cameras with respec ➢ We will use this information later on, to extract 3D information Stereo Vision ❖ Building the 3D map Stereo Vision ❖ Building the 3D map ➢ The frst step is to extract the disparity map between the two ima ➢ Once we fnd the matching points between the two images, we epipolar lines to impose epipolar constraints Stereo Vision ❖ Building the 3D map ➢ x and x′ are the distance between points in image plane corresp and their camera center ➢ B is the distance between two cameras (which we know) and f (already known) ➢ So in short, above equation says that the depth of a point in a s to the difference in distance of corresponding image points and t ➢ So with this information, we can derive the depth of all pixels in Stereo Vision ❖ Building the 3D map .. .Chapter Camera Calibration and 3D R Camera calibration Robot Camera Calibration Pose estimation Stereo vision Chapter Camera Calibration and 3D R Camera calibration Robot Camera Calibration. .. that makes the camera optical axes p Chapter Camera Calibration and 3D R Camera calibration Robot Camera Calibration Pose estimation Stereo vision Robot Camera Calibrati Eye To Hand Camera fixed... errors Camera calibration ❖ Pinhole Camera Model ➢ A pinhole camera is a simple camera without a lens and with a s Camera calibration ❖ Pinhole Camera Model Camera calibration ❖ Pinhole Camera