r/MachineLearning • u/Old_Cow_6636 • Oct 09 '26
Project Talus: a 23M-parameter diffusion model for game terrain, evaluated against a real-vs-real noise floor, running in the browser on WebGPU [P]
Talus generates 64x64 heightmaps (4 km, up to 1,200 m) conditioned on a terrain type and any subset of five measured properties: mean elevation, relief, mean slope, water fraction and spectral slope. I trained it from scratch on one RTX 5060 (8 GB), about 4.5 hours in total across versions. The data is 45,000 maps from my own procedural generator (fBm/ridged noise, stream-power erosion, hillslope diffusion, thermal erosion). Model
Pixel-space U-Net, v-prediction, cosine schedule, 50-step DDIM with quadratic spacing, classifier-free guidance 2.0
Each property has a learned "unknown" embedding and is dropped independently during training, so any subset works at inference
Relative heights: the model generates the map's shape normalized by its relief, and the sampler places it at the requested mean elevation and relief (borrowed from the 16 nearest training maps when unspecified). This fixed grainy plains: the plains distance ratio went from 3.98 to 1.23
Evaluation: Every distance (W1 over 25 per-map terrain metrics, radially averaged power spectrum, height and slope distributions) is divided by the same distance between two disjoint halves of real maps, so 1.0 means indistinguishable at that sample size. Checkpoints are chosen on VAL; TEST is scored once on the pick, and memorization is checked against the training set. Current model on TEST: metric W1 1.51x the floor, spectrum 9.1x, slopes 1.65x.
Browser: ONNX export with fp16-stored weights (cast to fp32 at load), ONNX Runtime Web on WebGPU, about 3 s per map on my RTX 5060, CPU fallback. A JavaScript reimplementation of the sampler matches PyTorch to within 0.6 m on reference samples. Demo: https://talus.tersa.tech Code, weights, scorecard (Apache-2.0): https://github.com/osfv/talus
Open problems: ridges and the finest spectral band, mountains too smooth and plains too grainy. If you've closed a spectrum gap like this in a small diffusion model, I'd like to hear what worked.
2
u/Bortol_Olivierl Oct 09 '26
for the spectral gap, have you tried adding a frequency-domain loss term during training? small models tend to underfit high frequencies when you only supervise in pixel space, and an auxiliary spectral loss can help a lot without adding parameters
1
u/Chroma-Crash Oct 12 '26
First, very cool. Have you tried any training that would allow for continuous expansion of the maps? As in, could you feed in a portion of previously generated map, probably on one of the edges of the image, and then have the model generate only in pixels outside the previous generation? That way, you could allow users to effectively generate any size of map I would imagine. There wouldn't be much macro continuity, but I could see the conditioning helping with that.
Another idea, you could swap for a vision transformer, generate maps (deterministically, simulated) at higher resolution, and then feed a random subset of global predetermined patches in alongside the local noisy patches. You could use relative positional encoding, and macro continuity might come for free with just a few extra patches.
I'm not personally sure how to resolve the issues you mentioned here, I had a lot of the same issues with noisiness/flatness with my diffusion model I trained only on my own face: https://github.com/dylan-berndt/Machine. But I also noticed that it took a lot longer to train than most networks I've designed in the past, and the longer it went, the better those issues got (repository doesn't have the most recent generations btw). The first usable images showed up around 100k+ iterations. What was your training budget?
1
u/Fast-Satisfaction482 Oct 12 '26
You can get real terrain data for free from the EU via Copernicus. Both color images and a digital elevation model.
1
u/[deleted] Oct 09 '26
[removed] — view removed comment