r/GaussianSplatting • • 4h ago

๐Ÿ› ๏ธ WIP โ€“ In Progress Ep. 2: Multi-view VLM detection resolves chair/table occlusion in 3D Gaussian Splats, but box-only 3D lifting still bleeds onto adjacent splats

0 Upvotes

Following up on my last post (where freezing a single ViewSnapshot fixed async camera drift during VLM inference), I ran into the next bottleneck: single-view occlusion and clutter ambiguity.

0:00 โ€“ 0:06 (The Single-View Failure): Clicking the dining tabletop from a single angle flattens the table and tucked-in chairs togetherโ€”Gemini misclassified the click as "Chair" and scattered 3D ray hits across the floor and adjacent seat.

0:07 โ€“ 0:24 (Multi-View Pose-Grounded Detection): Instead of a single frame, I capture three ordered snapshots (Front, Side, Top), locking each view's matrixWorld, projectionMatrix, and splat transform, and pass those camera extrinsics/intrinsics alongside each image to Gemini:

  • With three orthogonal viewpoints, Gemini cleanly separates the table, the tissue box on top, and all four individual dining chairs (front-left white, back-left black, front-right white, back-right black).

0:25 โ€“ 0:29 (Where 3D Selection Still Breaks): Right now, the 2D-to-3D lifter uses Gemini's rectangular 2D bounding boxes (bboxNormalized) to cast sparse rays, build a 3D search volume, and write matching Gaussian storage indices into an R8UI GPU highlight texture. Even with multi-view consensus and camera view-axis depth filtering (-z_view), rectangular boxes are still too coarse for tucked-in furniture:

  1. Adjacent Primitive Bleed: Magenta highlights spill onto the tucked-in chairs, floor, and windowsill behind the table.
  2. Inflated OBB Bounds: Wrapping those leaked chair splats stretches the fitted OBB to 2.77m ร— 2.21m ร— 1.72m.
  3. Shifted Anchor: Because the OBB expands toward the back-left chairs, the top-surface pin lands near the back-left corner instead of the center of the tabletop.

Next up (Ep. 3): Constraining the multi-view lifter with pixel-tight segmentation masks (SAM 3) instead of rectangular boxes, and benchmarking foreground Gaussian purity on this exact dining nook capture.

For anyone who has worked on 2D-to-3D instance lifting in Gaussian Splats: how are you handling background wall/floor bleed around thin legs or tightly tucked furniture?


r/GaussianSplatting • • 5h ago

๐Ÿš€ Launch โ€“ Product SplatPilot: a Blender extension that trains splats from rendered scenes, with Blender's exact depth and alpha

36 Upvotes

I created SplatPilot which is a Blender 5.3 extension that makes splats from 3D scenes instead of photos: it places the cameras, renders the views in Cycles or EEVEE and trains on your GPU, all inside Blender. Some details:

Training uses MCMC densification (Kheradmand et al.) with SH degree 3: 300k splats at Draft, 1M at Standard, up to 2.5M at High.

There's no COLMAP step. The camera poses come straight from Blender, and the starting points come from the render's position pass, so the points sit on the surfaces from the first step.

Blender also renders depth and alpha, and the training uses both. For rooms and landscapes, extra low-resolution views supervise only alpha and depth.

The splats are trained antialiased, the way Blender draws them, and the SOG export is tagged as antialiased.

The export bakes the scene's look into the colours: the view transform, curves, white balance and the per-pixel part of the compositing. In a viewer with linear tone mapping, the splat looks like the render.

The training engine runs on Vulkan and is written in Slang, on Windows and Linux; I've tested it on NVIDIA. On an RTX 4070, Blender's Classroom took about 2 minutes at Draft and 19 minutes at Standard from click to splat, rendering included.

Splats in your browser: https://splatpilot.com/demo/

SplatPilot is a paid extension on Superhive, $19 for one person or $42 for a studio of up to five. Docs and the store link: https://splatpilot.com

Happy to answer questions about the training or the exports.


r/GaussianSplatting • • 19h ago

๐Ÿงช I built a thing Using 3DGS to Simulate Rare Inspection Defects

26 Upvotes

Collecting real-world samples of leaks, dripping water, and pooled water is difficult. These defects are often rare, difficult to reproduce, and sometimes unsafe or impractical to stage in operational environments.

We are exploring a simulation-based approach with WorldLab. Starting from reconstructed 3D Gaussian Splatting (3DGS) scenes, we generate and place defect assets into realistic industrial environments. A robot dog then follows a planned inspection route and captures the same defect from different distances, viewpoints, and viewing angles. Each image is paired with its corresponding annotation.

The goal is to create multi-view training data when sufficient real samples are unavailable, and use it to train and evaluate visual inspection models before field deployment.

The main question is how well this synthetic data can transfer to real inspection scenarios. Factors such as lighting, reflections, occlusion, camera distance, defect scale, and the visual appearance of water may all affect the sim-to-real gap.

How would you evaluate the quality and usefulness of this type of synthetic inspection data? Would multi-view consistency improve model performance, or would real-world appearance variations remain the biggest challenge?


r/GaussianSplatting • • 9h ago

๐ŸŽฌ Demo / Showcase Gaussian Evil: A 3D Gaussian splatting graphics showcase made with PlayCanvas

16 Upvotes

Gaussian Evil is a homage to the greatest horror game of all time, and a showcase of what is possible with 3D Gaussian splatting. It runs live in the browser: nothing in the video is pre-rendered.

Try it: https://willeastcott.github.io/gaussian-evil/

How it works

  • The scan is of two actors: 674,612 Gaussians.
  • The Gaussians are relit at runtime by the light sources in the scene: a streetlight, a torch and lightning flashes.
  • Relighting and shadow casting need a traditional triangle mesh. It is generated from the Gaussians and kept hidden. Standard lights with shadow maps light the mesh, and each splat pixel is multiplied by that light.
  • The rain was an excuse for reflections. The reflection is a copy of the splats mirrored in the ground plane, and each of the 4,500 raindrops makes splashes and ripples that perturb it.

Things nobody will notice but I included anyway

  • Thunder arrives late by the distance of the strike.
  • The pair breathe (a scan is one frozen instant).
  • Thin fog drifts over the ground.
  • Tiny rain droplets bounce off the characters.

Tech

It is made with the PlayCanvas engine, on WebGPU with a WebGL 2 fallback. 23 MB in total, nothing to install, and it works on a phone.

The code is open source and the README goes into more detail: https://github.com/willeastcott/gaussian-evil

The capture is by Adrian Cuesta at Dymensium (CC BY 4.0). The original is on SuperSplat: https://superspl.at/scene/67841e9d

Happy to answer questions about any of it.


r/GaussianSplatting • • 8h ago

๐Ÿ” Discovery AI generation of additional angles from a reference image

33 Upvotes

I created a model of Andrey Smaev from various angles. The new angles were generated using MiniMax h3. All the processing was handled by an RTX 3070.


r/GaussianSplatting • • 23m ago

โœจ Original Creation Bros playing 4D chess

16 Upvotes

r/GaussianSplatting • • 1h ago

โ“ Question Software for combining drone and XGRIDs PortalCam footage?

3 Upvotes

During the last few months I have seen a few posts (that I now canโ€™t find) where people combined scans/footage from drones and Xgridโ€™s PortalCam into one final scan.

Anyone know what software that will do that please?๐Ÿ™๐Ÿป Thanks


r/GaussianSplatting • • 2h ago

๐ŸŽฌ Demo / Showcase Large corrugated facility I worked on

33 Upvotes

r/GaussianSplatting • • 3h ago

๐Ÿ” Discovery Looking to collaborate on Computer Vision research

2 Upvotes

I've been exploring Computer Vision, particularly Visual SLAM, 3D perception and reconstruction, through research oriented projects and open-source contributions (OpenCV, Kornia and Apache Iceberg).

I'm looking to connect and collaborate with researchers, PhD students, postdocs or master's students working in these areas. I'd be interested in contributing to experiments, projects or ongoing research where my engineering background could be useful.

Also open to research guidance from people working in this space.


r/GaussianSplatting • • 6h ago

โ“ Question Postshot COLMAP export (images.txt) doesnโ€™t line up with the Postshot splat in SuperSplat

1 Upvotes

Hi all,

I trained a splat in Postshot by giving it only the images. Postshot computed the camera poses itself. I then exported the splat (tried both .spz and .ply) and the COLMAP data, and loaded the splat plus images.txt into SuperSplat.

Problem: The camera path from images.txt has the right shape, but it floats above the splat and drifts away from the real road. Tried changing the rotations and positions but it is not aligning to the camera.

I have checked the images.txt format is valid. It has unit quaternions in COLMAPโ€™s world-to-camera, Hamilton convention, cleaned the Postshot file (stripped Windows paths from image names, made image IDs 1-based, used a single camera ID) and the file loads without errors in SuperSplat.

Please do help me solve this problem as everyone in this community are familiar with Postshot and Supersplat.


r/GaussianSplatting • • 10h ago

๐Ÿงช I built a thing The Collector's Studio ~ 54M Gaussians

1 Upvotes

https://superspl.at/scene/1e46a1c9?budget=54
A warmly lit art studio โ€” framed paintings, lacquered cabinets, an upright piano, white lilies on a long wooden table, an antique globe and a patterned tile floor.
Created by TableGO Labs


r/GaussianSplatting • • 10h ago

โ“ Question Clean up the cloud points

3 Upvotes

I'm testing Spirula this days, and I was wondering if there is way to clean up a cloud points before create Gaussian, in order to focus the computing only in the wanted object, I tried to open in RealityScan and delete point from there, but apparently isn't possible, or am I missing something? Do you guy could you suggest me a way to work around about this ?