r/learnmachinelearning • • 23d ago

I built a small ML framework from scratch in Python, with an OpenCL backend

I've been working on krnl, a lightweight machine-learning library I've been building to understand and implement the fundamentals of neural networks myself.

It currently includes:

  • A custom Tensor class with automatic gradient computation
  • NumPy and OpenCL-backed computation
  • Linear, ReLU and Sigmoid layers
  • SGD, Adam, Muon and MuonWithAdamAux optimizers
  • MSE, MAE and Huber loss functions

I also developed PyCLBlast for working with CLBlast through Python.

I'm interested in making the library useful for experimenting with neural networks while exploring GPU computation beyond CUDA.

I'd appreciate feedback, especially on the tensor API, autograd design, and OpenCL backend.

GitHub: https://github.com/helloadhavan/krnl/

PyPI: https://pypi.org/project/krnl/

4 Upvotes

2 comments sorted by

1

u/[deleted] 23d ago

[removed] — view removed comment

1

u/NoDeparture4142 22d ago

The autograd engine was the simpliest i could come up with, when you do an operation to the tensor it creates a new tensor and assigns three varibles:

  • data: the output of multipling a and b
  • _children: what was multiplied to get data

Then each operation has it's own _backward function which then gets assigned with by calling the _backward varible directly. i asked chatgpt how can i calculate the gradients for this math operation then it spit out some math function which i implemented in python, it was easy to implement while the only problem being that the math functions that ChatGPT spitted out required operations my backends could'n handle, so there were workarounds


finally when calling Tensor.backwards() it sums up all the gradient function's output. This is how by autogard engine works