C06.03 - 2D-2D Geometry 3D Reconstruction

• recover the 3D structure by computing the intersection of corresponding rays

• -> i.e. we need 2D point correspondances…

• we know the relative pose between cameras

• output 1 from previous methods (i.e. relative poses between cameas / absolute pose between first and following cameras)

• output 2: reconstructed 3D points in world frame (camera frame of first camera)

• general case for sparse reconstruction

  • triangulation

• simplified case for dense reconstruction

  • depth from disparity

general case:

• non identical cameras and not aligned

• => have to actually backproject and intersect

simplified case

• identical (at least virtual identical) cameras and aligned

• aligning -> creating “virtual” camera by warping so that

• image planes are coplanar

• epipolar lines are colinear

• => similar to human eyes…

• extrinsic parameters (relative rotaiton and translation)

  • obtaioned by epipolar constraint or other methods (i.e. PnP, ICP)

• intrinsic parameters:

  • focal length, principal point of each camera (u0,v0)

  • obtained by using a calibration method (e.g. Tsai, Zhang)

• determining the 2D position of a point

• given a set of corresponding 2D points

• and known camera poses

• no

• -> due to numerical errors they wond meet exactly

• => we can only compute an approximation

• we set left camera to world frame

• -> this yields following constraints:

• this yields a linear system of equations

• where we have two linear independent constraints for each camera

• => then we solve P with SVD…

• as projection rays do not exactly intersect

• -> least square approximation yields midpoint of the shortest 3D line segment connecting the two projection rays

• non-linear optimization

• -> finding the intersecting point by minimizing the squared reprojection errors from both sides (sum of squared reprojection error in both camera frames)

• displacement vector of a point

• -> brain allows us to percieve disparity by using left and right “images” from eye…

• => depth inversively proportional to disparity

• => enables depth perception…

• diestance between optimcal centers of two cameras

• essentially:

• Z coordinates are:

• baseline * focallength / (point left - point right)

• => here, only the u coords are given as we assume aligned cameras (i.e. epipolar lines are coplanar… -> only need “x” axis)

large:

• advantage: small depth error

• disadvantage: difficult search problem for close objects (projection may also be outside the right image)

small:

• advantage: easier search problem for close objects

• disadvantage: large depth error

• adjusting the image pairs

• -> so that they agree to the assumptions of our stereo camera (i.e. we can use detph from disparity….)

• yes!

• -> never perfectly aligned

• epipolar lines are aligned to horizontal scanlines

• -> makes correspondance search very efficient!

raw:

aligned:

• warp the original image planes onto (virtual) planes

• -> paralell to the baseline

=> one transformation for each image!

=> resulting in epiplolar lines alignet to horizontal scanlines

• the 2D projection points coordinates are changed accordingly

• define two new matrices to rotate old image planes respectively around their optical centers

• -> so that new image planes become coplanar and are both paralell to baseline

  • => ensures that epipolar lines are paralell

• also ensure that baseline is paralell to new x-axis

  • ensures that epiplolar lines are horizonatl and not only paralell

• in addition: corresponding points must have same y-coordinate (row ID)

  • requirement: new virtual cameras have same intrinsic parameters

• displacement of a 2D point in the image

• is only caused by extrinsic parameters (and not intrinisc… -> as intrinsic is the same…)

• as we assume y-coords to be the same

• -> only x-coords differ (i.e. disparity…)

• -> create a map where we assign each point (x,y) the disparity value x’-x -> i.e. right image x-ccord - left image x-coord of corresponding point

• -> close objects have bigger disparity -> are shown in brighter color in disparity map