CityGaussianV2: Efficient and Geometrically Accurate Reconstruction for Large-Scale Scenes

Yang Liu 1,2Chuanchen Luo 4Zhongkai Mao 1,2
Junran Peng 5Zhaoxiang Zhang 1,2,3
1 NLPR, MAIS, Institute of Automation, Chinese Academy of Sciences2 University of Chinese Academy of Sciences3 Centre for Artificial Intelligence and Robotic, HKISI 4 Shandong University5 University of Science and Technology Beijing
Architecture

(a) CityGaussianV2 reconstructs large-scale complex scenes with accurate geometry from multi-view RGB images, restoring intricate structures of woods, buildings, and roads. (b) "Ours-coarse" denotes training 2DGS with our optimization algorithm. This strategy accelerates 2DGS reconstruction in terms of both rendering quality (PSNR, SSIM) and geometry accuracy (F1 score). (c) Our optimized parallel training pipeline reduces the training time and memory by 25% and 50% respectively, while achieving better geometric quality. Here we report mean quality metrics in GauU-Scene dataset.

Abstract

Recently, 3D Gaussian Splatting (3DGS) has revolutionized radiance field reconstruction, manifesting efficient and high-fidelity novel view synthesis. However, accurately representing surfaces, especially in large and complex scenarios, remains a significant challenge due to the unstructured nature of 3DGS. In this paper, we present CityGaussianV2, a novel approach for large-scale scene reconstruction that addresses critical challenges related to geometric accuracy and efficiency. Building on the favorable generalization capabilities of 2D Gaussian Splatting (2DGS), we address its convergence and scalability issues. Specifically, we implement a decomposed-gradient-based densification and depth regression technique to eliminate blurry artifacts and accelerate convergence. To scale up, we introduce an elongation filter that mitigates Gaussian count explosion caused by 2DGS degeneration. Furthermore, we optimize the CityGaussian pipeline for parallel training, achieving up to 10x compression, at least 25% savings in training time, and a 50% decrease in memory usage. We also established standard geometry benchmarks under large-scale scenes. Experimental results demonstrate that our method strikes a promising balance between visual quality, geometric accuracy, as well as storage and training costs.

CityGaussianV2 reconstruct with large-scale scenes with high rendering quality and geometric accuracy! Our rendering is on the left, while extracted mesh on the right. The reconstruction purely relies on multi-view RGB images.

Comparison

Rendering & Geometry Performance

Architecture

Quantitative Comparison with SOTA reconstruction methods. “NaN“ means no results due to NaN error, and “OOM“ denotes out-of-memory error. “FAIL“ means the method fails to extract meaningful mesh due to poor convergence. We introduce TnT-style geometry evaluation protocol for large-scale scenes. Please refer to the paper for more details.

Training Cost and Quality

Architecture

Compared wity CityGS V1 (600M), our V2 significantly reduce the time and VRAM cost in training, while achieving comparable rendering quality (PSNR) and superior geometric accuracy (F1-Score). The tiny version (290M) is compact and efficient, making it well-suited for deployment on low-end devices like smartphones or VR headsets. Results are from Table 2 of the paper.

Geometry

Rendering

CityGaussianV2 achieves SOTA performance both in rendering and geometry. The compared method is on the left, Ours in on the right. For GOF, we compare the mesh from side view to avoid influence of near-ground hull. Try selecting different scenes!

Related Work

[Guédon et. al. 2024] Guédon A, Lepetit V. Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 5354-5363.

[Huang et. al. 2024] Huang B, Yu Z, Chen A, et al. 2d gaussian splatting for geometrically accurate radiance fields. ACM SIGGRAPH 2024 Conference Papers. 2024: 1-11.

[Yu et. al. 2024] Yu Z, Sattler T, Geiger A. Gaussian opacity fields: Efficient and compact surface reconstruction in unbounded scenes. ACM TOG, 2024.