Mini-Splatting: Representing Scenes with a Constrained Number of Gaussians

chaenyangยท2024๋…„ 8์›” 6์ผ
1

3D

๋ชฉ๋ก ๋ณด๊ธฐ
1/9

๐Ÿ“Œlink: https://arxiv.org/abs/2403.14166
methodology ๋ถ€๋ถ„๋งŒ ์šฐ์„  ์ •๋ฆฌ

4. Methodology

densification & simplification โ†’ reorganizing the spatial distribution of Gaussians

4.1. Densification

์ด๋ฏธ์ง€ ๋‚ด์˜ Gaussian ๋ถ„ํฌ๋ฅผ ์ตœ์ ํ™”ํ•˜์—ฌ ์ด๋ฏธ์ง€์˜ ์„ ๋ช…๋„๋ฅผ ๊ฐœ์„ 

  1. Blur Split & Depth Initialization
    - Gaussian ๋ถ„ํฌ๊ฐ€ ๊ณ ๋ฅด๊ฒŒ ๋ฐฐ์—ด๋˜๋„๋ก ํ•˜์—ฌ ์ด๋ฏธ์ง€์˜ ์ดˆ๊ธฐํ™” ๋‹จ๊ณ„์—์„œ ์‚ฌ์šฉ๋จ
    - Gaussian ๋ธ”๋Ÿฌ ์•„ํ‹ฐํŒฉํŠธ ์ฒ˜๋ฆฌ
    - Depth Initialization: ์ด๋ฏธ์ง€์˜ ๊นŠ์ด ์ •๋ณด๋ฅผ ํ™œ์šฉํ•˜์—ฌ ๋” ์ •ํ™•ํ•œ 3D ์žฌ๊ตฌ์„ฑ์„ ๊ฐ€๋Šฅํ•˜๊ฒŒ ํ•œ๋‹ค

  2. Gradient-based Split & Clone Strategy
    - ๋ถ€๋“œ๋Ÿฌ์šด ์ƒ‰ ์ „ํ™˜์„ ์œ„ํ•ด ์„ค๊ณ„
    - ํฌ๊ณ  ๋น„ํšจ์œจ์ ์ธ Gaussian๋“ค์€ ์ตœ์ ํ™” ๊ณผ์ •์—์„œ ๋ณด์กด๋œ๋‹ค

  3. ๊ฐ€์šฐ์‹œ์•ˆ ํˆฌ์˜, ๊ฐ€์šฐ์‹œ์•ˆ ์ธ๋ฑ์Šค ๋ Œ๋”๋ง
    - ๊ฐ ํ”ฝ์…€์— ๋Œ€ํ•ด ์ตœ๋Œ€๋กœ ๊ธฐ์—ฌํ•˜๋Š” Gaussian์˜ ์ธ๋ฑ์Šค๋ฅผ ์‹œ๊ฐํ™”ํ•œ๋‹ค. ์ด๋Ÿฐ ์ธ๋ฑ์Šค๋Š” ๋ฌด์ž‘์œ„ ์ƒ‰์ƒ์œผ๋กœ ํ‘œ์‹œ๋˜์–ด ๊ฐ ๊ฐ€์šฐ์‹œ์•ˆ์„ ๊ตฌ๋ณ„ํ•  ์ˆ˜ ์žˆ๋‹ค
    - ํˆฌ์˜๋œ ๊ฐ€์šฐ์‹œ์•ˆ ์ธ๋ฑ์Šค์™€ ์‹ค์ œ ๊ฐ€์šฐ์‹œ์•ˆ ์ธ๋ฑ์Šค๋ฅผ ๋น„๊ตํ•จ์œผ๋กœ์จ ์–ด๋А ๊ฐ€์šฐ์‹œ์•ˆ์ด ๊ฐ€์žฅ ํฐ ์˜ํ–ฅ์„ ๋ฏธ์น˜๋Š”์ง€ ํŒŒ์•…ํ•  ์ˆ˜ ์žˆ๋‹ค

  4. Deblurring Strategy
    - ๋ธ”๋Ÿฌ ์ฒ˜๋ฆฌ๊ฐ€ ํ•„์š”ํ•œ Gaussian gi๋Š” ํ•ด๋‹น Gaussian์ด ์ฐจ์ง€ํ•˜๋Š” ์ตœ๋Œ€ ์˜์—ญ Si๊ฐ€ ์ž„๊ณ„๊ฐ’ Tblur ๋ณด๋‹ค ํด ๋•Œ ์‹๋ณ„๋œ๋‹ค. ์ด ์ž„๊ณ„๊ฐ’์€ ์ด๋ฏธ์ง€์˜ ํ•ด์ƒ๋„ H ร—W์™€ ํ•˜์ดํผํŒŒ๋ผ๋ฏธํ„ฐ ฮธblur์˜ ๊ณฑ์œผ๋กœ ๊ณ„์‚ฐ๋œ๋‹ค.

โ•์ˆ˜์‹ ์„ค๋ช…

giblur={giโˆฃSi>Tblurโˆงiโˆˆ[1,N]},Tblur=ฮธblurโ‹…Hโ‹…Wg_i^{\text{blur}} = \{g_i | S_i > T_{\text{blur}} \land i \in [1, N] \}, \quad T_{\text{blur}} = \theta_{\text{blur}} \cdot H \cdot W

์ด๋ฏธ์ง€ ๋‚ด์—์„œ ๋ธ”๋Ÿฌ ์ฒ˜๋ฆฌ๊ฐ€ ํ•„์š”ํ•œ Gaussian gi๋ฅผ ์‹๋ณ„ํ•˜๊ธฐ ์œ„ํ•œ ๊ธฐ์ค€

  • ๋ณ€์ˆ˜๋“ค ์˜๋ฏธ
    • gblur: ๋ธ”๋Ÿฌ ์ฒ˜๋ฆฌ ํ•„์š”ํ•œ Gaussian ์ง‘ํ•ฉ
    • gi: i๋ฒˆ์งธ Gaussian
    • Si: i๋ฒˆ์งธ Gaussian์ด ํ”ฝ์…€์— ๊ธฐ์—ฌํ•˜๋Š” ์ตœ๋Œ€ ์˜์—ญ์˜ ํฌ๊ธฐ
    • Tblur: ๋ธ”๋Ÿฌ ์ฒ˜๋ฆฌ๋ฅผ ์œ„ํ•œ threshold
    • ฮธblur: ๋ธ”๋Ÿฌ ์ฒ˜๋ฆฌ์˜ ์ •๋„๋ฅผ ๊ฒฐ์ •ํ•˜๋Š” ํ•˜์ดํผํŒŒ๋ผ๋ฏธํ„ฐ
    • H, W: ์ด๋ฏธ์ง€์˜ ๋†’์ด์™€ ๋„ˆ๋น„
  • Si๋Š” (H, W) ํ•ด์ƒ๋„์—์„œ ๋ž˜์Šคํ„ฐํ™” ๊ณผ์ •์„ ํ†ตํ•ด ๊ณ„์‚ฐ ๊ฐ€๋Šฅ
  • gblur ์ง‘ํ•ฉ์€ Si๊ฐ€ Tblur ๋ณด๋‹ค ํฐ ๋ชจ๋“  Gaussian์„ ํฌํ•จํ•œ๋‹ค. ํ•ด๋‹น Gaussian๋“ค์ด ์ด๋ฏธ์ง€์— ํฐ ๋ธ”๋Ÿฌ ํšจ๊ณผ๋ฅผ ์ค˜์„œ, ์„ ๋ช…๋„๊ฐ€ ๋‚ฎ์•„์ง€๋Š” ์›์ธ์ด ๋˜๊ธฐ ๋•Œ๋ฌธ์— ํŠน๋ณ„ํ•œ ์ฒ˜๋ฆฌ๊ฐ€ ํ•„์š”ํ•˜๋‹ค.

profile
์ž‰๊ณต์ง€๋Šฅ

0๊ฐœ์˜ ๋Œ“๊ธ€