Title

3D Gaussian Splatting for Real-Time Radiance Field Rendering
(SIGGRAPH 2023, Kerbl & Kopanas et al.)

Abstract

Radiance Field

• NN -> costly to train & render
• Recent faster methods -> trade off speed for quality
• No real-time display rate for unbounded scene + 1080p rendering

Proposed Method

• State-of-the-art visual quality
• Competitive training time
• High-quality real-time novel-view synthesis at 1080p

3 key elements

• Represent the scene with 3D Gaussians from sparse points
  • They preserve desirable properties of continuous volumetric radiance fields for scene optimization
  • Avoid computation in empty space
• Interleaved optimization/density control of 3D Gaussians
  • optimizing anisotropic covariance to achieve an accurate representation of the scene
• Fast visibility-aware rendering algorithm that supports anisotropic splatting

1. Introduction

2 ways of 3D representations

Meshes and Points

• most common 3D scene representations
• explicit & good fit for GPU/CUDA based rasterization

NeRF

• continuous scene representations
• typically optimizing a MLP using volumetric ray-marching
• Most efficient radiance field -> interpolating values stored in voxel/hash grids or points
• Stochastic sampling required for rendering -> costly and can result in noise.

3D Gaussian

• Combines best of both worlds.