C09 - Single-View Geometry

• extract geometric information from single image

• no longer need point correspondences

• -> make use of “human knowledge”

• -> i.e. structural regularity such as orthogonality and paralellism

• high-level geometric features such as vanishing points

• “structured” lines and planes

  • -> i.e. projection of 3D lines that are paralell/orthogonal/coplanar…

• single-view 3D reconstruction

• camera pose estimation and optimization

  • need man made structure regularity… as it uses vanishing points

• camera calibration to estimate intrinsic parameters

• introduce three city “types” that show different structural regularity

• -> general: man made structures show such regularities and can be exploited to introduce geometrical constraints

cities:

1. mahnattan

   • vertical dominant direction(DD)

   • two horizontal dominant directions(DD)

   • -> I.e. vertical: facade of buildings

   • horizontal: flucht des hauses; verbindung haus-haus (e.g. brücke…)

1. atlanta

• one vertical DD (facade)

• multiuple horizonral DD (multiple house angles…)

1. hong-kong

   • one vertical DD

   • multiple horizontal DD

   • multiple sloping DD (i.e. uphill street…; not very flat…)

• extending lines in image that are paralell in real world -> intersect at vanishing point

• connection of two horizontally lying vanishing points

• connection between camera center and vanishing point

• -> are paralell to a 3D dominant direction (thus do not differentiate between vanishing and dominant direction)

• vanishing points can be seen as “points at infinity”

• -> ray from camera center through vanishing point is paralell to the 3D lines associated with the vanishing point…

• have the same

• given: two pairs of poitns (each pair lies on a 2D line)

• calculate the actual 2D lines

intersection of these two lines yield:

• least squares version

• -> use more than two lines and find least squares point of intersection

• using a representation on a sphere?

• intersectoin may be far from the camera center since image lines may be roughly paralell (i.e. nearly paralell -> intersect at infinity…)

=> somewhat analogous to short baseline in 3D reconstruction

• use projectoin plane of 2D line in image frame / 3D line in 3D space

• -> intersection of this plane with sphrer creates a “ring” around it

• -> set of great circles of paralell 3D lines intersect on sphere

• -> constitiutes the vanishing point

• => this point and shpere center define the vanishing direction (/dominant direction)

• as vanishing direction estimation

unit sphere:

• bounded space

• unit sphere encodes orthogonality constraint in 3D (-> i.e. one can enforce that three dominant directions are orghogonal to each other)

image:

• unbounded space

• can hardly enforce orthogonality constraint of vanishing directions (as it is in 2D -> hard to ensure three directions are orthogonal….!)

1. census-based methods (old-fashioned)

2. sampling-based methods (efficient)

3. search-based methods (accurate)

• compute vanishing directions on unit sphere (i.e. intersections of rings…)

• -> due to noise -> intersectoin of pair of inliers slighly deviate from grund truth

• -> small area associated with high density of noisy intersections considered vanishing points

• problem: sensitive to outliers

outlier:

• sample three image lines in 3D to compute three great circles

• assume first two lines associated with same vanishing point; third line associated with another vanishing point

• use these two circles to determine first vanishing direction (at point of intersection -> as only two -> no noise as one exact intersection (fyi: two as one is on the other side…)

• use orthogonality constraint and third circle to vanishing direction on third circle orthogonal to first vanishing directoin /dominant direction

• due to already two found dominant directions -> there is exactly one possiblity left that is orthogonal to the other two (implicitly clear from first two…)

=> perform several iterations to guarantee that at least one sampling is valid…

• camera calibration

1. calculate the vanishing points

1. define 3D points at infinity along x,y,z axis as well as origin in manhattan frame (i.e. we use a calibration object that is a cuboid)

1. calculate projectoin matrix (up to scale)

-> i.e. project the known origin of manhattan frame (i.e. [0,0,0,1])

where vx, vy, vz, o are unknowns (o -> some scalar as up-to-scale)

1. determine scale and compute focal length

   • by some DLT method…

• NO

• -> it is a projection of a point at infinity…