C04 - Camera Calibration

Determine the

• extrinsic parameters (R,T) of a camera

• intrinsic parameters (K plus lens distortion) of a camers

• w.r.t. the world frame

• -> absolute position

• Tsai’s method

• Zhang’s method

• measure n 3D points with n >= 6

• these 3D points lie on a calibration target (e.g. cube with chessboard pattern)

• the points are measured in the world frame

• for the 3D points, we also need their 2D projections in the image plane

• in general, we use a single image

• use direct linear transfor (DLT)

• -> we create the system of linear equations for each 3D points and their respective projection

• -> We stack these equations in a large linear system

• -> we solve it using least squares using SVD, as it is overdetermined

• ! solution is up to scale, but does not really matter for our purposes

• 2D points = K[R|T] * 3D points

• -> multiply K[R|T] to get single matrix -> M

• Rewrite homogenous coordinates to pixel coordinates

• i.e. lambda u / lambda

• substitute both parts of the fraction w.r.t. our M matrix and the 3D point

• -> i.e. for u: m1*P / m3*P; i.e. for v -> m2*P / m3*P

• bring both on same side to have homogenous equation (right hand side is 0…)

• -> (m1 - ui m3) * P = 0

• -> (m2 - vi m3) * P = 0

• => for each point…

-> when rearranging the terms so that on left side is matrix with all except m and right is only m we get

-> [[P1 0 -uP1][0 P1 -vP1]] [m1 m2 m3] = [0 0]

=> write P out with P = [X1w Y1w Z1w] -> and stack for all points yields our matrix Q…

• 2n * 12

• -> where n is the number of points

• -> as each point provides.2 independent equations

• 12 dimensions

• 11 to have unique (up to scale) solution

• we have overdetermined LGS

• least squared minimizes the sum of squared residuals (i.e. QM = 0 -> residuals are ||QM|^2 -> as not prefectly 0…)

• 
  

• ||M||^2 = 1 -> to exclude the trivial solution of M = 0

• SVD

• arg min M ||QM|||

• subhect to ||M||_2.= 1

• => Q = UEV where optimal solution is column of V corresponding to the smallest singular value

• decompose it into the intrinsic (K) and extrinsic (R,T) part

• as our K is an upper triangular and RT is orthogonal

• -> we can apply the (generalized -> non square matrix) QR decomposition

• decomposing M into an orthogonal matrix R, T and an upper triangular matrix K

• => orthogonality inherently satisfied…

• must be non coplanar!!!

• use many more than 6 points (ideally > 20)

  
  

• corners can be detected with accuracy < 0.1 pixels

• multiple images

• 3D coplanar points

• 2D projected points

tsai:

• one image

• more complex calibration target

• non-coplanarity

zhang:

• more images

• less complex caliration target

• coplanar points

• allows us to set Z_w to 0… (as lie on same plane…)

• no, initially neglect it

• similar to tsais

• -> but leave z-row simply out (as it is 0…)

• => combine K and R,T matrices just like in Tsais

• -> but now H instead of M

• -> H is called Homography

• the same way as in tsais

• Q -> 2nx9

• H -> 9

  
  

• 9

• => each point introduces 2 contstraints

• -> 4 non-colinear points are required

• similar to tsai

• -> use SVD to calculate homography

• but: decomposition of homography differs…

• as we removed the z-row

• -> R,T not an orthogonal matrix anymore

• -> QR is not applicable…

• Idea: Intrinsic matrix always stays the same, regardless of camera perspective (considering we have same camera)

• => each homography is different but intrinsic part is the same

• -> use multiple images to first estimate the intrinsic part

• -> then simply solve for the extrinsic part…

• => thus zhangs method requires multiple views

• 20-50

• spanning entire field of view of the camera for best calibration results

• use prior constraints of rotation to derive formulas w.r.t. only unknown intrinsic part

• i.e. write homography as matrix multiplication of intrinsic and extrinsic (rotation) part

• -> reformulate formulas based on

  • norm of rotation = 1

  • individual rotation parts are orthogonal to each other => r1Tr2 = 0

• again use SVD to solve for the intrinsic part in the above creates LGS

1. get multiple images with measured 3D poitns and 2D projections

2. For each, compute the Homography

3. Based on rotation constraints, use the homographies to formulate a LGS we can solve for the intrinsic part

   1. as the intrinsic part is the same for each homography

4. After obtaining the intrinsic part, use the LGS to compute the extrinsic part

5. Project the result (extrinsic) from the matrix space to the SO(3) manifold to fulfill the rotation matrix constraints

• compute initial intrinsic parameters

  • initially: distortion set to 0

• estimate initial extrinsic parameters as if the intrinsic have been already known

• run gradient descent to minimize reprojection error

  • to jointly optimize/estimate intrinsic, extrinsic and distortion parameters

• euclidean distance in pixels

• between observed image point and corresponding 3D point reprojected onto camera frame

• with the sum of squared distances

• between the real imate point (measired initially) and the reprojected one with current parameters (rotation,translation, radial distortion, intrinsic) using pinhole model

• => using i.e. zhang for the initial guess for these parameters…