[CV] Final 요약 8. Image Alignment

Week 8-1. Keypoints and Features

Keypoint: salience 현저, 중요성, repeatability, compactness and efficiency, locality
flat region, edge, corner

2. Blob detection

blob 얼룩, 물방울 superposition of two ripples 두 파동의 중첩
ripple 잔물결

• spatial selection: Laplacian response의 magnitude는 center of the blob에서 maximum,
  scale of Laplacian is matched to the scale of the blob
  Laplacian 반응이 최대가 되는 지점인 blob 중심을 찾음

Scale Selection
find scale of the blob by convolving it with Laplacians at several scales and looking for the maximum response.
blob 실제 크기에 맞는 필터 scale에서 Laplacian의 크기가 최대가 됨
$\sigma$ = 3에서 가장 잘보이면 blob 크기가 3인 것
Laplacian response decays as scale increases -> scale normalization

Scale Normalization
multiplied by $\sigma^2$
scale increases하면 response 약해지니까 시그마를 곱해서 scale increase할 때 반응이 더 크게 만들어 효과가 비슷하게 만든다

compute at many scales, find max

2D Blob Detection
Normalized Laplacian of Gaussian 사용

3. SIFT Detector

Scale Invariant Feature Transform: 이미지 변형에 영향을 받지 않는 keypoint 추출하는 알고리즘
Laplacian of Gaussian approximated by difference between two different Gaussians
DoG = (s - 1)NLoG
연산량 많으니 DoG로 구함; scale space에서 얻은 같은 옥타브 내에 인접한 두 개의 블러된 이미지들끼리 차이를 구하는 연산

• Extracting SIFT Interest Points

4. SIFT Descriptor

• properties of feature descriptors:
  easily compared, easily computed, invariant- translation, rotation, scale, change in image brightness

SIFT Descriptor
: find dominant gradient orientation over a small image patch
gradient 방향 고르기
divide 16x16 window into 4x4 grid of cells
compute an orientation histogram for each cell
16 cells x 8 orientations = 128-dimensional descriptor
한 grid에서

Comparing SIFT Descriptors
essentially comparing two arrays of data

• SIFT Results:
  Scale and Rotation Invariance, Robustness to Clutter (Occlusion)
  reliable for only small changes in viewpoint

Week 8. Image Alignment and Warping

1. Image Alignment and Warping

fit the parameters of transformation according to a set of matching feature pairs ("correspondences")

1. Direct (pixel-based) alignment: most pixels agree
2. Feature-based alignment: extracted features agree

Image Alignment: what transformation?
${x_i, x'_i}$, x' = f(x;p), find the best estimate of the parameters p

Global Warping/Transformation: transformed output look like?
G(x) = F(h{x}) change domain of image function
transformation T, p' = T(p), matrix; p' = Mp

2. 2x2 Transformation

Scaling
a coordinate means multiplying each of its components by a scalar
Uniform scaling: this scalar is the same for all components
Non-uniform scaling: different scalars per component

Rotation

Shear

2x2 Image Transformations

Translation
matrix representation이 안 됨-> 3x3 transformation

3. 3x3 Transformation

Homogeneous Coordinates
2D -> 3D: x y -> x y 1 = ax ay a(bias)

Translation

Basic 2D Transformation

Affine Transformations
방향, 크기 뿐만 아니라 위치도 포함하는 변환
• Origin does not necessarily map to the origin

4. Determining Unknown Transformations

x' = Mx, M = ?
Least Squares Error

parameters b A
x

linear least squares problem, minimize the error
derivative to 0, solve x

5. RANSAC (RANdom SAmple Consensus)

Intuition: outlier로 만든 line은 나머지 points support 못 받음
Algorithm:
1. Sample randomly the number of points required to fit the model
2. solve for model parameters using samples 선 만들기
3. score by the fraction of inliers within a present threshold of the model, 범위 안 나머지 points 수 세기
Repeat 1-3 until the best model is found with high confidence

Estimating Affine Transformation using RANSAC

• RANSAC loop

1. Get three point correspondences(화살표) randomly
2. Compute M which minimizes $E_{LLS}$
3. Count inliers
4. Keep M if largest number of inliers
   Recompute M using all inliers

• Pros
  simple and general, applicable to many different problems, often works well in practice
• Cons
  Lots of parameters to tune
  Don't work well for low inlier ratios
  Can't always get a good initialization of the model based on the minimum number of samples