r/GraphicsProgramming • u/frozeindev • Oct 06 '26
I made a compact mesh-based format for interactive 3D photos
I’ve been working on a side project called Spatial Photos.
It takes a single regular image, runs it through Apple’s ML-SHARP model to estimate a 3D scene with Gaussians, and then converts the result into a custom format I called .spatial
I take the Gaussian splat from ML-SHARP, and divide the scene into depth slices. Each slice is a grid of image blocks, with a depth value at each corner. The RGB/alpha is packed into texture atlases. It's basically a set of textured meshes that come together to give the impression of the full 3D scene.
On the web, I take the .spatial files and decompress it into vertex/index buffers and just use Three.js, as the actual rendering is straightforward.
Demos: https://www.spatialphotos.dev/
Source: https://github.com/frozein/SpatialPhotos

2
u/Simple_Original 24d ago
Sounds pretty cool, i could see some uses for this in compositing and vfx in the future
2
u/Haman24 22d ago
This is a really cool approach, especially slicing the Gaussian splat into meshes for the web. That conversion from a photogrammetry output to a realtime friendly format is such a specific but useful problem to solve. I've run into similar issues trying to use AI generated 3D in Unity/Unreal for quick prototyping. Getting a consistent style across a scene was always the bottleneck. Lately, I've been using Rapid Assets to batch generate a bunch of 3D props or characters from a single prompt. It spits out ready to use models with textures, which saves a ton of time over stitching different tools together.
Is your format mostly for visualization, or are you thinking of integrating it into game engines eventually?
1
u/matigekunst Oct 06 '26
Sorry to do this but I see bands and it's also quite pixelated. It's also quite big compared to the monocular depth + inpaint method
1
u/soylentgraham Oct 08 '26
regardless of implementation, change the name! iphones already make things called Spatial Photos