[CV] 5-2. Photometric Stereo

Week 5-2. Photometric Stereo

1. Gradient Space and Reflectance Map

Surface Gradient and Normal
equation of plane, use p,q variable, surface normal, unit surface normal

Gradient Space
z plane = Gradient Space(pq plane): 모든 점이 특정 표면 방향과 일치함
normer vector(앞에 나옴), source vector, $cos\theta_i$

Reflectance Map
Image intensity (irradiance) at the point (x,y):
Reflectance map: I = R(p,q)

Reflectance Map: Lambertian Surface
Image intensity (Lambertian case):
$\displaystyle I = R(p,q) = \frac{\rho}{\pi}kc cos\theta_i = \frac{\rho}{\pi}kcn\centerdot s$
k: source brightness (radiance intensity)
$\rho$: surface albedo (reflectance)
c: normalization constant (camera gain)
Let $\displaystyle \frac{\rho}{\pi}kc = 1$
then $I = cos\theta_i = n\centerdot s$

contour: 윤곽
gradient map위에 그려진 타원은 모두 같은 image intensity(brightness)

타원 벗어날수록 어두워짐, 90도 넘으면 밝기 0

빛 쏘는 방향이 다름

source direction $(p_s,q_s)$, surface reflectance $\rho$, one image irradiance I(x,y) 알아도 unique surface orientation(p,q) 찾을 수 없다, 같은 밝기의 타원이 있으니까.

2. Photometric Stereo

광도 입체경
idea: surface orientation 방향의 애매함을 해결하기 위해 lighting 다른 여러 이미지 사용
1. Acquire N images with N known light sources
2. Using known source direction and BRDF, construct reflectance map for each source direction
3. For each pixel location (x,y), find (p,q) as intersection of N curves.
This (p,q) gives the surface normal at pixel (x,y)
Smallest N needed depends on the material properties
e.g., N = 3 for Lambertian surface
N 이미지 모아서, reflectance map 만들고 N curves 겹치는 곳에 (p,q)각도 찾아라(픽셀의 surface normal), 표면 특성에 최소N 결정됨

Photometric Stereo: Lambertian Case
성립하지 않는 경우: When S is not invertible 뒤집을 수 없으면, one source direction이 다른 두개의 linear combination이 될 수 있으면($s_3=\alpha s_1 + \beta s_2$)

• more light sources N

Color Images
The case of RGB images
$I_R = \rho_RSn$
$I_G = \rho_GSn$
$I_B = \rho_BSn$
solution: first solve for n using one channel
then n 대입, $(\rho_R,\rho_G,\rho_B)$구함
Or combine three channels and solve for n
$I = \sqrt{I_R^2 + I_G^2 + I_B^2}= \rho Sn$

Results: Lambertian Sphere

Estimated Surface Normal 계산해서 Estimated Albedo 만듦

3. Calibration Based Photometric Stereo

Use a calibration object- size, shape 알고 scene objects와 same reflectance
calibration: 눈금

• Orientation Consistency: Points with the same surface normal produce the same set of intensities under different lighting

surface normal 같은 점들은 다른 lighting에도 같은 intensities 생성
1. Capture N $\geq$ 3 images under N different light sources, produces N image intensities
2. calculate the surface normal (p,q,1) using known size
3. Create lookup table for the N-tuple
all pixels on sphere

$I_1$	$I_2$	...	$I_N$	p	q

4. Capture N images of the scene object under the same N light sources
5. For each pixel in the scene, use lookup table to map ($I_1,I_2,...,I_N)$ -> (p,q)

4. Shape from Normals

Shape From Surface Normals

Estimate surface by integrating surface gradient:
$z(x,y) = z(x_0,y_0)+ \int^{(x,y)}_{(x_0,y_0)}-(pdx + qdy)$
where $(x_0,y_0)$ is a reference point and $z(x_0,y_0)=0$
z(x,y) obtained by integration along any path from $(x_0,y_0)$ 이 좌표에서 아무 길의 적분으로 얻은 z

5. Interreflection Problem

The brightness of a scene point due to light from other scene points
Photometric stereo overestimates albedo and underestimates surface tilt
interreflections within the bowl 생각하면서 계속 추정함