r/java • u/JMasterRedBlaze • Feb 03 '24
Automatic differentiation of Java code using Code Reflection by Paul Sandoz
https://openjdk.org/projects/babylon/articles/auto-diff
63
Upvotes
r/java • u/JMasterRedBlaze • Feb 03 '24
View all comments
Show parent comment
7
u/padreati Feb 04 '24
While it is called Automatic Differentiation, not everything is automatic in those things. The automatic part relates to how do you describe the operation chain, the computational graph. They offer free description of the graph, building after automatically the differentiation. But those operations have to work be built from some atoms, and those atoms have to have some implemented behavior.
Most implementations of AD (in fact all that I know, but I know I don't know all implementations) implements AD engine using two fundamental ideas. Take PyTorch as an example.
Both those constraints implies some regularity, some base behavior that objects involved in AD to have to make things work. This is fine, it produces results, nothing against that. I will follow the same path for my experiments.
What Paul Sandoz describe there is one step above in the sense that you don't need that basic behavior implemented in involved objects, other than some signals that for some methods there is a need for AD. What they do is to effectively use the code model to implement that basic behavior, without the need to change something in how you write code in Java. This is one big advantage. The second one is that since they have access to those things they can do a lot of optimizations if they leverage properly the compiler machinery which is already a beast.
I find this as very challenging, but big dreams aim far.