[CV] Final 요약 10. Stereo

Week 10-1. Setereo 1

입체사진

1. Camera Model

Perspective Projection

• Forward Imaging Model: 3D to 2D
  image coordinates, camera coordinates, world coordinates

• Arbitrary focal length(f)
  Camera matrix P
  image = P * camera

?: y_i, 빗변: x_i

• In general, the camera and image have different coordinate systems
• In particular, the camera origin and image origin may be different:
camera coordinates 정의

Camera Matrix Decomposition
focal length: 초점 거리
단위 초점거리, 원점 이동 고려 안 함, z = 1, 원점 같다고 가정
P = K[I|0]

World-to-Camera Coordinate Transformation

General Camera Matrix

• sensor not at f = 1 and original shift

2. Camera Calibration

Given a single image, determine the camera’s parameters (camera matrix)

예전에 나옴: Constrained Least Squares
eigenvalue

3. Pose Estimation

Given a single image, estimate the exact position of the photographer (camera)
decouple: 분리하다
노란 박스가 아래로 바뀜

노란 박스를 Kt로 바꿀 수 있음 t: photographer의 위치

4. Binocular Stereo

multi-view geometry 필요, visual cues for 3D- shape from X
Binocular Stereo

Take two pictures of the same subject from two slightly different viewpoints and display so that each eye sees only one of the images

Week 11-1. Stereo 2

1. Epipolar Geometry

Epipolar line
Epipole: projection of o' on the image plane

2. Essential and Fundamental Matrices

essential matrix, epipolar line
Ex = l'

$x'^T Ex = 0$

• point to line
  vs. homograph: point to point

Essential Matrix
x' = R(x-t): rigid motion 강체 운동
x, t, x' 한 plane위에 있음 -> $x^T(t \times x) = 0$
$(x-t)^T (t \times x) = 0$: coplanarity

Linear Algebra Review: Cross Product
Cross product takes two vectors and returns a vector perpendicular to both

Essential Matrix
skew-symmetric matrix to represent cross product
R[t$_{\times}$] = E

Fundamental Matrix
generalization: Identity matrices is removed

3. Estimating Fundamental Matrix

Estimating Fundamental Matrix

one correspondence gives one equation- 9 equations, we need at least 8 points

x: Ax = 0 such that $||x||^2$ = 1
Define least squares problem: $min_x||Ax||^2$ such that $||x||^2 = 1$
Constrained linear least squares problem 앞이랑 똑같이 풀기, eigenvalue

Extracting Rotation and Translation
Compute essential matrix E from known left and right intrinsic camera matrices and fundamental matrix F
$F = K'^{-T}EK^{-1} \\ E = K'^T FK$

l' = Fx

Week 11-2. Stereo 3

1. Simple Stereo

X: 3D point
image plane 주황색선
O, O': camera center
How is X related to x?
X/Z = x/f 왼쪽 초록색 직각삼각형
(X-b)/Z = x'/f 오른쪽 빨간색 직각삼각형
Disparity d = x - x' (wrt to camera origin of image plane)
= bf/Z (inversely propotional to depth)

2. Stereo Rectification

맞지 않는 조건을 simple stereo가 되도록 조정
• Reproject image planes onto a common plane parallel to the line between camera centers
• Need two homographies (3×3 transform), one for each input image reprojection

1. Compute E to get R
2. Rotate right image by R
3. Rotate both images by $R_{rect}$
   에피폴라 라인과 baseline이 만나지 않게(평행하게) 파라미터 조정, 너무 복잡한 내용, 논문 참고
   왼쪽 각도 계산해서 오른쪽 이미지를 각도에 맞게 조정, 그리고 둘다 또 조정

순서 한번 더
1. Rotate the right camera by R (aligns camera coordinate system orientation only)
2. Rotate (rectify) the left camera so that the epipole is at infinity
3. Rotate (rectify) the right camera so that the epipole is at infinity
4. Adjust the scale

3. Stereo Matching

1. Rectify images
   (make epipolar lines horizontal)
2. For each pixel
   a. Find epipolar line
   b. Scan line for best match <- How would you do this?
   c. Compute depth from disparity
   $Z = \frac{bf}{d}$

How do we detect the template in the following image?

• Sum of Squared Differences (SSD)

• Normalized Cross-Correlation (NCC)

+Normalized Correlation (Review)

• Slide a window
  along the epipolar line and compare contents of that window with the reference window in the left image
  Matching cost: SSD, NCC

Effect of Window Size
smaller window: more detail/ more noise
larger window: smoother disparity maps/ less detail, fails near boundaries

• Matching Fail:
  Textureless regions, Occlusions 폐색, Repetition, Specular surfaces 정사면(정 반사성의)

Improving Stereo Matching

• Adaptive Window Method: For each point, match using windows of multiple sizes and use the disparity that is a result of the best similarity measure. 다양한 크기의 winidow, 제일 유사성 큰걸로
• Graph Cuts: similarity score + 얼마나 가까운지, 근처를 segment로 묶어 비슷한 diparity 갖게 함. match & smoothness
• Active Stereo: 추가적인 광원, 깊이 확인
  projectors acts like reverse camera 레이저 여러 개 쏨, 구조화된 광원, 더 정확하게 fundamental matrix, depth 확인 가능

vertical line 보내서

+3D Reconstruction with Internet Images