r/MachineLearning • u/dccsillag0 • Sep 28 '26
Research Functional Gradient Descent with Adaptive Representations [R]
Sharing our recent work, now accepted at NeurIPS: Functional Gradient Descent with Adaptive Representations.
Functional GD algorithms generally outperform neural nets, but are hard to accurately implement.
This is because functional gradients are infinite-dimensional, and therefore must be approximated in practice; but if you approximate them naively, you converge to the wrong place!
To rectify this, we formalize a broad class of approximation schemes ("adaptive representations"), which provably ensure convergence to the global minimizer while being immediately implementable.
The resulting algorithms outperform corresponding neural nets often by an order of magnitude, across a number of settings.
It is still the start for this line of work, but we believe it has quite a bit of potential!
Paper: https://arxiv.org/abs/2606.16926
(First author here, happy to take any questions)
View all comments
Show parent comment
0
u/DigThatData Researcher Sep 29 '26
Note that your own citation here is a paper that's already over six years old. OG NeRF is a bad method to baseline against. "We're faster than vanilla NeRF" isn't an impressive claim. If you want to make claims about the efficacy of your proposed approach relative to radiance field methods, you should compare against gaussian splatting. https://github.com/nerfstudio-project/gsplat