Formulas

• virtual image plane

• -> with focal length = 1 unit

• origin of pixel coords at principal point

• unnormalized -> we simply have (u,v)

• -> normalized -> we can calc direction ray as our optical center now aligns to the camera center …

• -> [u,v,1]…

• => easier to calc backprojections…

=> 3D points and normalized image points are colinear

• -> as the vector from O -> [u,v,1] is the same direction as from O -> P (but with scale…)

• also helpful for expressing certain geometric constraints (i.e. coplanarity constrain)

• vanishing point as projection of point at infinity

• vanishing direction as image normalization of our vanishing point

• colinearity between 3D vectors in camera frame

• (lecture C03 part 1 slide 37 schauen)

• vanishing directoin -> line through camera center and vanishing point

• -> can be computed by image normalization

  
  

• assume we have vanishing point that lies on the line from P0 to v

• we extend v in the vanishing direction (unit direction of the 3D line the vanishing point lies on) P=P0+tD

• reformulated (division by t) this yields

• if we let t->infinity

• provided we have two point clouds

• assumption: we know the 3D-3D correspondances

1. compute the center of mass for both

2. point set normalization by normalizing both 3D point sets with their respective center of mass

3. compute the W matrix

4. perform SVD on the W matrix

5. compute rotation and translation from the SVD

• move the center of the point cloud

• to the origion of our coordinate system

• -> thus, we try to find the rotation and translation to align the origins in such a way

• that the point correspondances match in 3D…

assume: same cardinality for bot point sets

-> for each correspondance

-> calculate x’p’T

=> yielding a 3x3 matrix

and then elementwise sum over it…

• transformation form p->x

• conduct SVD decomposition of W

• homogeneous coordinates -> still 2d point, but extended to allow certain calculations

• normalized coordinates -> 3D direction in camera center colinear to the 3D point that projected it…

• Is encodes the geometric relationship between calibrated cameras w.r.t. Rotation and translation

• it also encodes epipolar geometry

  • describes relationship between points in one camera view and corresponding epipolar lines on other camera view

• used in e.g. 8-point method to compute relative pose

• we have to work with normalized image coordinates

• we define n as the normal of the epiploar plane

• then, we can express the orthogonality constraings in the right frame as

• we can express the normal in the right camera frame n as cross product between the translation vector and the direction p1’ which is the rotated version of the point in the left camera frame (-> thus in the direction in the right camera frame)

• we can then replace the p’_1 from the right camera frame with the rotaion of the left camera frame direction p_1 to bring both into consideration… (before only considered right camera frame…)

• this puts the left and right camera frame in correlation (p2 right, p1 left)

• they are coplanar w.r.t. the epipolar plane

• use conclusion from before (coplanarity constraint relating two point correspondances in two camera frames)

• here, we replace cross product with skew symmetric matrix

• combining T_X and R -> we get to our essentrial matrix that encopdes rotation and translation as well as the coplanarity constraint (epipolar geometry) between two calibrated cameras provided a point correspondance

• i.e. used in 5/8 point method

• -> use point correspondnaces and previous conclusions to derive a matrix

• decompose with SVD

• in the fundamental matrix

• -> we do not have normalized image coordinates

• -> and we do not know the intrinsic parameters

• it relates with the essential matrix by:

• from left camera (c1) to right camera (c2)

• we can work directly in ordinary image plane

• instead of normalized image plane

• -> i.e. can be used if we do not know intrinsic parameters K and thus cannot normalize…

• eight poitn:

  • fundamental

  • essential

• five point

  • essential

• transformation of (typically 2D-2D) point correspondences

  • can also be used for 3D-2D i.e. zhang…

• based on perspective projection (more general than affine transforamtion)

• encodes co-planarity information

=> encodes transformation information of points that lie on the same plane

=> between two camera view from different perspective

• provided we have sevaral point in 3D on a plane and their corresponding points in both camera frames

• we can expres the 3D plane w.r.t. one of the points as follows:

• we then can write an equation to express the point p2 in the right camera frame depending on the point P in 3D

• i.e. apply extrinsic and intrinsic parameters…

• then, due we can introduce our 3D plane expression by modifying it to be equal to 1

• here, all still happens in the right cmaera frame (c2)

• we can then use the disritbutive law to “extract” the point P

• we then can reqrite the point P w.r.t. the its projected point p1 in the left camera frame (normalized image coordinates -> thus k^-1 is for camera 1…) and the first K is for camera 2…

• here, p1, and p2 by itself are homogenous coordinates!

• => here, we assume same calibration as we want to use it for SLAM (i.e. used same camera…)

• we can then extract this block infront of p1 to get our homography

• which encodes the relative camera pose information

• normal n

• a poitn P lying on the plane

• and the distance d from the origint to the plane

• we iterate over all pixels in left patch (H) and right patch(F)

• -> sum up the square of their distance…

• Zp -> depth inforamtion

• b -> baseline length

• f -> focal length

• ul -> x-coords in left image

• ur -> x-coords in right image

• where e is the pixel wise error

• I1,I2 are the images (i.e. 2D matrix with intensity values)

• p1, p2 are the pixel coordinates

• minimize both left and right reprojection error

• left part of the reprojection error

• here, reprojection error means that after triangulation, we project the triangulated point back and compare it to the initial point we used to triangulate it

• p1 is the initial point we used for tirangulation

• pi(P,K1,I,0) is the projection of the triangulated point

• -> i.e. K1|[I|0]P

  • -> here I|0 as we assign the left camera frame as world frame

• the right part is

• where again p2 is the point in camera frame, and we backproject the 3D point using K2, R and T (assumption we know the pose…)