C06.01 - 2D-2D Geometry Overview

• Estimate the Rotation and Translation from 2D potin correspondances (between two images -> relative rotation, translation)

• reconstruct the 3D points

• no!

• -> need 2D point correspondances

1. estimate the R,T from 2D point correspondances

2. Use this estimation to triangulate the 3D points for reconstruction (i.e. least square plus reprojection error minimization)

• n point correspondances (2D)

• calibration matrix

• we have two cameras

• usually emply left-camera frame as reference coordinate frame

• => i.e. 3D points are assumed to be provided in left camera frame coordinates

  • -> R,T for this is [I|0]

• We cannot tell the difference between rescaled scenes

• based on the image coordinates

• => pinhole: size inverse proportional to distance

  • -> ambiguity when we both scale size and distance in the same way…

• not possible to recover absolute scale of scnee

• -> can only solve up to scale

• => thus have only 5 degrees of freedom

  • three for rotation

  • two for translation (up to scale)

    • direction but not norm(distance)

• each correspondance yields 4 knowns

  • -> u,v for left, u,v, for right camera

• we have 5+3n unknowns

  • 5 for motion up to scale (i.e. 3 rotation, 2 translation)

  • 3 for each point (3D coordinates)

• 4n >= 5n+3

• => n >= 5

• => we need at least 5 point correspondances

• we have two camera centers and the respective corresponding poitns

• projection ray through camera center and their respective points determine the epipolar plane

  
  

• intersection of epipolar plane with image planes

• back projected ray of p onto the other camera image

  • -> line from camera center through point

  • each point on that line projected onto the other image plane

  • -> dots form a line…

• projection of the camera center of the other camera onto the image plane

projection:

• from 3D poitn onto image plane

backprojection:

• from camera center through point on image frame back into 3D space

• assume we have normalized image coordinate points p2 right camera and p1 for left camera

• they are orthogonal to the normal of the epipolar plane

• -> can derive set of equations

normal of epipolar plane is the same as the crossproduct between translation and the left camera point p1

As we want to use our left camera frame, we have to rotate the left camera point p1

We can then refomulate it using skew symmetric matrix to get

=> where R and T can be computed from E…

essential:

• makes use of normalized image coordinates

fundamental:

• makes use of unnormalized image coordinates

• can directly work on ordinary image plane instead of normalized image plane

• eight point method

  • essential and fundamental matrix

• five-point mehtod

  • essential matrix

• each pair of correspondances provides linear equation

• we can rewrite this to:

-> i.e. simply write out above formula…

=> rewrite this to Q*E (where left hand side are stacked uv parts, and E is the e-vector

• solve minimal solution or over-determined solution

• decompose E into R and T

• all entries of E are independent

• -> not true as for calibrated case, depend on only 5 parameters (R, T up to scale)

• => 5-point algo takes this into consideration…

• Our Q matrix should have rank 8 (up to scale to solve 9dim Q matrix)

  • differently scaled E matrices lead to same result up to scale

• -> i.e. 8 points as each provides 1 equation

• n> 8 points

• minimize ||QE||^2 subject to constraint ||E||^2 = 1

• => solution is eigenvector correspoinding to smallest eigenvalue of matrix QTQ

• same as essential matrix…

• orders of magnitude difference between column of data matrix (as we wor directly on image coords…)

• -> least squares method yields poor results

=> poor numerical conditioning makes results very sensitive to noise

• use normalized 8-point method

• => i.e. rescaling the data

• rescale so that

• centroid is 0

• standard deviation is sqrt(2)

• calculate mean and variance

• rescale each point with

as matrix:

1. normalize the correspondances

2. estimate normalized fundamental matrix with 8-point mehtod and normalized correspnodances

3. unnormalize the fundamental matrix to get the regular fundamental matrix