C05.2 - Correspondance Estimation Large Motion

• direct: use full patch to compute i.e. brighness difference between pixels

• indirect: determine features (i.e. points or lines) and find matching pairs to estimate patch correspondances

• group of connected pixels

• taht share some common properties (e.g. grayscale value)

• identify and mark blobs (regions)

• intersectoin of two or mroe edges

• localization accuracy:

  • blob: lower

  • cornder: higher

• distincrive:

  • blob: more

  • corner: less

• moravec

• harris

• shi-tomasi

• susan

• fast

• MSER

• LOG

• DOG (SIFT)

• SURFCenSurE

• in regoin aroud corner

  • -> gradient has multiple dominant directoins

  • -> e.g. vertical, horizontal, diagonal

• -> shifting a window in any directoin should cause large intensity changes

• edge: only changes on one axis (i.e. horizontal or vertical)

• flag region: no changes

• cornder: changes in all directoin

  
  

• SSD

• flat: SSD ~ 0

• edge: SSD ~ 0 on one axis; SSD >> 0 on other axis

• corner: SSD >> 0 on all axes

• sum over all pixels

• squared difference between intensity in pixel 1 - intensity of pixel 1 shifted by delta x, delta y

• for two patches (initial pixel patch and shifted one), compute SSD between pixel pairs

• lower SSD indicates higher similarity between patches

• -> consider SSD among multiple directions

  • interest measurement: smallest SSD (i.e. if we find one small SSD -> should not be a corner…)

• if interest measurement > threshold -> is corner

directions:

• horizontal,

• vertical

• both diagonal directions

• we have to physically shift the window

• instead of shifting

• -> approximate surrounding pixels wiht 1st order taylor espansion

• => intensity of shifted pixel is approximately the initial value plus the shift delta times the directional gradient

• as for the shifted patch, we also have the regular inteisity of pixel x,y -> we can remove it altogether as (I(x,y)-I(x,y)-I_x(dx)-I_y(dy))

• -> we simply calualte the SSD based on the directoinal x,y gradients at the positoin (x,y) times their respective delta

• rewrite the elements of the sum so that we partition the deltas out

• where the middle part is our M matrix

• this matrix can be decomposed

• if both eigenvalues are much larger than 0 -> we have a corner…

• harris detector

• harris detector is not scale invariant

• -> same patch size not applicable to different scales of images

• description of pixel information aroud a feature

• -> i.e. blob

• i.e. given we have a detected point (i.e. a corner)

• -> define point descriptors (ile. HOG)

• define distance function that compares two descriptors (i.e. SSD, SAD, NCC or hamming distance)

• brute force matching

• -> we have set of N descriptors in both images

• -> calculate distance between all pairs and choose best fit (O(N^2))

• algo can ocassionally return good scores for false matches

• comute ratio of distances to 1st and 2nd closest descriptor

• -> d1 / d2 < threshold (usually 0.8)

• -> only if distance for closest descriptor is larger than 0.8 (threshold) times distance to 2nd closest descriptor -> consider it a match…

• => distinctive enough

• SSD (sum of squared distances)

• SAD (sum of absolute distances)

• NCC (normalized cross correlation)

• direct pixel correspondances from both patches (i.e. H and F) over all pixels in the patch…

• !always > 0

• always >= 0

• -> 0 only if H and F are identical…

• ranges between -1 and +1

• is exactly 1 if H and F (patches/descriptors to compare) are identical

calculation:

• sum over pixel-wise multiplicatoin

• divided by

• for both patches individual:

• sqrt of sum over squared pixel values

• account for the difference in average intensity of two images

• => subtract mean value of each image

• changes in average illumination

• due to additive illumination changes

• for each patch -> calculate the average intensity (sum over all pixel insentities divided by number of pixels)

• -> introduce in our distance measurement by subtracting this average from the individual values (i.e. direct in the sum)

• zero mean sum of squared differences

• -> i.e. mean corrected (zero mean) pixel values…

• distinctiveness

• robustness to geometric changes

• robustness to illumination changes

  
  

• descriptor is “description” of pixel information around feature

• distnictiveness

  • -> descriptor can be uniquely distinguish a feature from the other features

  • without ambiguity

• scale invariant (for zooming)

• rotation invariant

• view point-invariant (for perspective changes)

• small illumination changes can be modeled with affine transformation

• -> so called affine illumination changes

1. detect keypoints (i.e. patch center)

2. use descriptor to warp patches into canonical space

   1. determine scale rotation and viewpoint change of each patch to do this

3. establish patch correspondances based on similarity of the warped patch (i.e. in canonical space)

1. scale assignment

2. rotation assignment

3. view point assignment

4. wart patch to canonical space

• compute scale using LoG operator

  
  

• use harris or gradient histogram

• to find dominant orientation

• use harris eigenvectors to extract affine transformaiton parameters