Project report

Gaussian Splat

Gaussian splats are real-time capable and, above all, located in a 3D space so that they can be used directly in game engines or in virtual production. A large number of photos are sufficient to generate Gaussian splats from the real or digital worlds. To do this, the algorithm places the eponymous Gaussian splats in space, similar to “volumetric pixels”, and refines them by deforming, splitting, changing color and transparency until they resemble the original photos from the respective perspectives as closely as possible. In simplified terms, this can be compared to “multi-view” sculptures, which then not only represent two or three things, but the entire 3D scene from all possible perspectives (see example below).

A surreal visual of a human face, formed by numerous green particles. Emerging from the head are multiple drones flying in different directions, symbolizing a futuristic or technological theme. The drones appear to be emanating from the figure's mind, giving a sense of connection between technology and the human intellect.

Project participants

Alexander Kutter Colin Behrens Jan Pieniak Conrad Dreyer
Marc Schneider

Project status

Konzeptions- / Entwicklungsphase

Revival of a technology for capturing and displaying 3D data

From an initial “try it out and see what it can do”, an entire field of research developed within a year. About a year ago, Kerbl et. al (https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/) published their research work, which has since been followed by numerous other publications. The application of Gaussian splats in the field of real-time rendering generated so much interest. It compensates for some of the disadvantages of NeRFs (Neural Radiance Fields), in which synthetic photos are rendered by a neural network.

Gaussian splatting is a pioneering technology in scanning and visualizing volumetric 3D data. In contrast to traditional methods such as point clouds or mesh structures, this technique generates high-resolution and photorealistic 3D models.

A multi-perspective sculpture, where the shape changes depending on the viewpoint. On the left, the sculpture appears to spell

Example: Multiview sculpture

Image source: https://monicadecardenas.com/markus-raetz/
"Incrocio I", 2001 - 2019, cast bronze, 25,3 x 39,3

A key advantage of Gaussian splatting is its ability to efficiently display detailed surface structures without typical artifacts. This method is ideal for complex surfaces as it can accurately capture both fine details and larger structures. This includes hair, for example, but also vegetation.

Gaussian splatting is also efficient in terms of data processing. The use of continuous functions reduces memory requirements and increases processing speed, enabling faster creation of 3D models, even on cell phones in the browser. This is hardly possible with a similar level of detail with “classic” polygon models. Overall, Gaussian Splatting offers an accurate, detailed and efficient solution for 3D scanning, ideal for applications such as cultural preservation, industrial inspection and medical imaging. Our research focuses on the efficient and (semi-)automated scanning of outdoor and indoor areas using drones, the generation of Gaussian splats from renderings and the scanning of objects.

Matching amounts for this project

Gaussian Splat: Hermann Monument

at 03.09.2024

Project contribution

In our first productive use of Gaussian Splats for the Lippe regional association, the task was to scan the Hermann landmark and make it tangible on a...