Ref-GS : Directional Factorization for
2D Gaussian Splatting


Youjia Zhang1, Anpei Chen2,3,†, Yumin Wan1, Zikai Song1,
Junqing Yu1, Yawei Luo4, Wei Yang1,†
1Huazhong University of Science and Technology 2University of Tübingen, Tübingen AI Center
3Westlake University 4Zhejiang University

Abstract

In this paper, we introduce Ref-GS, a novel approach for directional light factorization in 2D Gaussian splatting, which enables photorealistic view-dependent appearance rendering and precise geometry recovery. Ref-GS builds upon the deferred rendering of Gaussian splatting and applies directional encoding to the deferred-rendered surface, effectively reducing the ambiguity between orientation and viewing angle. Next, we introduce a spherical mip-grid to capture varying levels of surface roughness, enabling roughness-aware Gaussian shading. Additionally, we propose a simple yet efficient geometry-lighting factorization that connects geometry and lighting via the vector outer product, significantly reducing renderer overhead when integrating volumetric attributes. Our method achieves superior photorealistic rendering for a range of open-world scenes while also accurately recovering geometry.

Results and comparisons

Here we demostrate side-by-side videos comparing our method to top-performing baselines across different captured scenes.

Select a scene and a baseline method below:


Interactive visualization. Hover or tap to move the split.
Note how our method synthesizes accurate reflections of trees that move smoothly across the car surface and windows, while Gaussian-based methods produce blurry reflections that appear and disappear inconsistently with changes in viewpoint.
Note that our model renders reflections with more high-frequency details and is capable of producing realistic near-field content reflections. These include clear reflections of tree branches and buildings in the sphere.
In this relatively diffuse scene with no significant view-dependent appearance changes, we achieved a successful reconstruction of the geometries of the "reflective table center".
Although semi-transparent but reflective surfaces (such as windows) present certain challenges for our method, it is still capable of simulating convincing reflection effects on windshields.

Ablation study

Here we demostrate side-by-side videos comparing our full method to versions of our method where key components have been ablated. See more details in the paper.

Select an ablation below:


Interactive visualization. Hover or tap to move the split.
Using the G-buffer instead of Sph-Mip (i.e. w/o Sph-Mip Encoding) or without deferred shading (i.e., w/o Deferred Shading), sharp details, such as tree branches reflected in the sphere, are not accurately reconstructed. It is necessary to use multi-level spherical feature grid strategies (i.e., w/o Mipmap), otherwise rough surfaces will fail to be reconstructed and artifacts will appear during rendering. Additionally, directional factorization (i.e., w/o Directional Factorization) is essential for modeling near-field inter-reflections.
Using the G-buffer instead of Sph-Mip (i.e. w/o Sph-Mip Encoding) or without deferred shading (i.e., w/o Deferred Shading), sharp details, such as tree branches reflected in the sphere, are not accurately reconstructed. It is necessary to use multi-level spherical feature grid strategies (i.e., w/o Mipmap), otherwise rough surfaces will fail to be reconstructed and artifacts will appear during rendering. Additionally, directional factorization (i.e., w/o Directional Factorization) is essential for modeling near-field inter-reflections.

Shape Reconstruction

Here we demostrate side-by-side videos comparing our method to top-performing baselines across different captured scenes.

Select a reconstruction below:


Interactive visualization. Hover or tap to move the split.

Synthetic results

Here we demostrate side-by-side videos comparing our method to top-performing baselines across different captured scenes.

Select a scene and a baseline method below:


Interactive visualization. Hover or tap to move the split.

Related works

  • Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields
  • GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces
  • 3iGS: Factorised Tensorial Illumination for 3D Gaussian Splatting
  • 3D Gaussian Splatting with Deferred Reflection
  • 2D Gaussian Splatting for Geometrically Accurate Radiance Fields