Monocular Dense SLAM with Consistent Deep Depth Prediction

• depth refinement in 2D and 3D aspects
  • handle uncertainty 2D-wise
  • 3D-wise alignment and refinement

논문 링크 | 설명 영상 링크

Refine depth prediction

2D aspect

• areas that are likely to cause uncertainty

1. image boundaries
   : crop (discard)

2. object contours
   : Gaussian weight function (calculates the corresponding weight to refine the predicted depth; 한 픽셀의 주변 픽셀에 대한 color & depth difference 계산)

$\hat{D}_{p_i} = \frac{1}{W_{p_i}}\sum_{p_j \in S_{p_i}}G_{\sigma _s}(||p_i-p_j||)G_{\sigma_c}(||I_{p_i} - I_{p_j}||)G_{\sigma _d}(||D_{p_i} - D_{p_j}||)D_{p_i}$

• $i$, $j$: pixel indexes
• $I_p$: color
• $D_p$: depth
• $\sigma$: parameters to adjust the spatial similarity (spatial, color, depth)
• $W_{p_i}$: for normalization

$\rarr$ sharp color & depth는 강화, 다른 곳은 smoothed

3. pixels far away from the optical centor of the camera
   : ORB2에서 한 것처럼 close & far points 찾고, far points는 local submap 만들 때 사용 안 하고 discard함

3D aspect

• avoiding outliers in depth
• map에서 떠다니는 이상한 애들 없애기
  • threshold(1)을 바꿔주거나 LOF score 구하는 걸 여러 번 해서 더 좋은 결과를 얻을 수 있으려나?

LOF (Local Outlier Factor) score를 모든 3D point에 대해 계산

• $N_k$ : nearest neighborhood
• $lrd_k$ : reciprocal of the average distance from one point to all its neighbors

LOF > 1이면 그 point는 sparse하다는 뜻,
즉 candidate outlier

---

Ego-motion으로 submap alignment

• ICP (Iterative Closest Point) 알고리즘 사용

이 논문에서 영감을 받았는데, 차이점:

1. In contrast to [23] which refines the ego-motion and the depth map alternately, the continuously optimized camera poses of the SLAM system rather than the constant depth prediction from the deep neural network are trusted in our work.
2. Local optimization이나 loop closure 할 때, matrix를 곱해서 다음 submap을 구할 수 있음

• scale factor to refine depth

  • between translation motion and ICP
  • $f = \frac{t_{slam}}{t_{icp}}$

• the initial ICP matrix =