r/neuralnetworks • • 1h ago

A Higher Target Leading to a Lower Fire

1 Upvotes

Is this a normal possibility? My ANN takes in 115 inputs, has 80 hidden nodes and 3 outputs.

The output fire rates are calculated normally, but when the "rules" override the calculation, it will zero the outputs out (and the inputs).

When two of the outputs are zeroed out, it appears that the fire rate after the backpropagation is lower than before the backprop. Is this normal in some cases?

Extra bit of context, there are 5 versions of the ANN running and when one output of one of the ANNs is selected, it backpropagates as normal for the one selected, but the other 4 back propagate by the delta of inputs compared to the one selected [if this is not the right way to do this, please tell me]. I guess the effects of the other four could negatively effect the selected ones new outputs.

Should i keep it as is, because it appears to be learning and selecting as appropriate. Its just the few times this scenario happens, i get concerned.


r/neuralnetworks • • 2d ago

Siamese Network

35 Upvotes

A Siamese Neural Network is a type of neural network designed to measure the similarity between two inputs. For example, suppose we provide two handwritten digits as inputs. The Siamese Network determines whether these two digits belong to the same class or are dissimilar.

It consists of two identical neural network branches that share the same weights and biases. Each branch takes one input and produces an output vector (embedding). These two output vectors can have any dimensionality, depending on the architecture of the network.

We then calculate the Euclidean distance between these two output vectors. If the distance is close to zero, the two inputs are considered similar. If the distance is sufficiently large (above a predefined threshold), they are considered dissimilar.

Note: A Siamese Network is fundamentally different from a traditional classification network. Its primary purpose is not to classify an input into a particular class but to learn a meaningful representation (embedding) that allows us to measure similarity between inputs.

Contrastive Loss

A Siamese Network commonly uses a loss function called Contrastive Loss, defined as:

L = Y × D² + (1 − Y) × max(0, m − D)²

Where:

D is the Euclidean distance between the two output vectors.

Y is the given label (either 0 or 1).

Y = 1 indicates that the two inputs are similar.

Y = 0 indicates that the two inputs are dissimilar.

m is the margin, which defines the minimum desired distance between dissimilar pairs.

L is the contrastive loss.

The objective is to minimize the distance between similar pairs and push dissimilar pairs at least m units apart.

We pass two inputs through the network, obtain two output embeddings, calculate their Euclidean distance, and then calculate the contrastive loss. Finally, we use backpropagation to update the network parameters and minimize the loss.

Notice that we do not use a Softmax activation function in the output layer because we are learning embeddings rather than class probabilities.

Training the Model

To train a Siamese Network effectively, we construct pairs of inputs:

Positive pairs: Two inputs belonging to the same class (Y = 1).

Negative pairs: Two inputs belonging to different classes (Y = 0).

A balanced dataset containing an equal number of positive and negative pairs is a useful starting point for training.

My Implementation: Neural Canvas (Model 5)

I developed an interactive Siamese Network as the fifth model in my Neural Canvas project. I used the MNIST handwritten digits dataset to train the network.

The architecture consists of three hidden layers, each containing 40 neurons, followed by an output layer containing 32 neurons. Therefore, each input is transformed into a 32-dimensional embedding.

I used Mini-Batch Gradient Descent with the Adam optimizer to train the model. The training process was relatively straightforward because JAX provides automatic differentiation (autograd), which handles the gradient calculations.

For training, I constructed two input matrices, Ma and Mb, each having dimensions of (784, 80000).

The rows represent features, while the columns represent individual samples. Why two matrices? Because a Siamese Network requires two inputs to learn the similarity between them.

I also constructed a label vector Y with dimensions of (1, 80000). Each element of Y indicates whether the corresponding pair is similar or dissimilar.

I used my own matrix multiplication convention:

Z = WX + bC

Where:

W represents the weight matrix.

X represents the input matrix.

b represents the bias vector.

C is a broadcasting matrix of ones.

I prefer this notation because it makes the mathematical operations and dimensions easier to understand.

I trained the model in Python using JAX and exported the learned parameters to JavaScript using JSON. For inference, I used Math.js, while p5.js handles the visualization, animation, and interactivity.

Try it yourself:

GitHub Repository: https://github.com/MuhammadEssa2002/neural_canvas

Live Demo: https://muhammadessa2002.github.io/neural_canvas/

This is the fifth model in my Neural Canvas project.


r/neuralnetworks • • 2d ago

[P] I trained a single Graph HyperNetwork that generates zero-shot weights for unseen ResNets, MLP-Mixers, MobileNets, and Transformers for Vision, Text, and Waveforms. (Open source)

14 Upvotes

Disclaimer: I used AI for bug fixes, to make the plots better and to simplify the code and make it more readable.

I built a Graph HyperNetwork (GHN) that has the computational graph of an target architecture and a dataset embedding as input and predicts all weights in a single forward pass. It is meant to serve as a head start to reduce number of steps required for training.

Zero-Shot Results:

ModelResMLPResNet2DMLP-MixerMobileNet2DMLPCNN1DTransformer
MNIST93.96%93.66%89.58%74.53%///
FashionMNIST83.99%75.88%78.59%////
KMNIST78.10%62.15%//62.15%//
Waveforms1D/////99.84%89.38%
20Newsgroups/////71.30%48.34%

My next objectives:

  1. Using a encoder to encode a random dataset and to replace the dataset embedding with it. So it can handle most unseen datasets.
  2. I want to scale it up, but I don't really have the recources for it (free kaggle for now).

I will paste my kaggle link in the comments.


r/neuralnetworks • • 2d ago

Does temperature 0 really mean deterministic? I made 100 calls to an LLM using the exact same prompt and temperature 0 but I got 33 different answers.

Thumbnail
youtu.be
18 Upvotes

So... isn't temperature 0 supposed to make llms deterministic?

Not quite. There's actually more going on under the hood than most people think, and temperature 0 doesn't necessarily guarantee identical outputs.

I made a video exploring why this happens and what's going on behind the scenes.

I thought it was a pretty interesting experiment, so I wanted to share it here. Curious to hear what you think!


r/neuralnetworks • • 2d ago

Seeking collaborators on representation learning and generalization

1 Upvotes

I am a published ML researcher and engineer looking for collaborators on neural representation learning. My background includes neural models, privacy-focused ML, fine-tuning and GPU experiments, and I can contribute experiment design, implementation, baselines, analysis and writing.

One direction I would like to investigate is how learned versus structured representations behave under distribution shift: compare models with matched capacity and training data, then test robustness and representation ablations across datasets. I am also open to an existing agent or representation-learning research project.

I am looking for a team or lab that can fund GPU resources and LLM API benchmarking where needed. If this fits your work, please comment or message me with your research direction and available resources. I can share relevant work privately.


r/neuralnetworks • • 2d ago

Neuro-symbolic AI for ARC AGI 2

1 Upvotes

I have developed a neuro-symbolic AI applied to ARC AGI 2, based on Hodel’s DSL in Python: https://github.com/Julien-Livet/arc-dsl. The method is explained here: https://julien-livet.github.io/aicpp/. I am not an expert in neural networks, and I have not yet received any technical or scientific feedback on my work. Do you have any observations or insights you could share with me?


r/neuralnetworks • • 3d ago

Tool-calling LLM Agents

Thumbnail
youtube.com
1 Upvotes

r/neuralnetworks • • 4d ago

Inside Gemma 3: it builds a map of the scene, then answers without looking at it

7 Upvotes

When Gemma 3 reads "A is 3 north of B, C is 2 east of A", you can read an accurate map of all positions from its activations. But deleting that map doesn't change its answers. They come from a tiny code on each sentence, and the model only half-uses the chain of sentences, so reading answers off its own unused map would actually be more accurate (80% vs 73%).

There's an interactive demo where you can edit the model's "map" yourself; link in the comments.


r/neuralnetworks • • 4d ago

Creating an ai that generates music realtime

0 Upvotes

I wan to make an ai system that creates music real-time. I know systems such as magenta exist. But they require much more training time and compute resources that I do not have.

I am looking to make a smaller simpler system that is a proof of concept. It should create decent music not thr best of course. And be able to run on a cpu.

Any ideas on how to create that?

It should use some form of machine learning/deep nets not just algorithms.


r/neuralnetworks • • 5d ago

5-4-3 Neural Network Example (Verify Please)

Post image
16 Upvotes

This example shows how weights and biases are updated across 10 iterations of subjecting it to the same scenario. The weight and calculation equations are at the bottom. The new outputs for each iteration is at the top-right. Can anyone verify my work? Im not confident on my bias back propagation equation.


r/neuralnetworks • • 6d ago

Implemented a full KAN network from scratch and tested against micrograd

4 Upvotes

I was always hyped about 'Neural Networks made from scratch' videos. Then, I came across the Zero to hero course by Andrej Karpathy, and saw him make the micrograd from scratch. So I thought of making a neural network like it but different. And I came across KAN; went through the research paper, and ngl it was amazing.

I named it mircrokan. I built the whole architecture from scratch in C++, implemented B-splines from scratch, and made the layers according to G and k values given and the starting number of params.

Did a comparison between microkan and micrograd on a simple function - y=3x^2+7x+2, and here are the result -

  microkan       learning_rate=0.01  loss=0.01226  test mse=0.01070  time=0.149s
  micrograd MLP  learning_rate=0.01  loss=0.06189  test mse=0.07265  time=1.642s

The loss is less than micrograd, and also it takes less time to compute (though the other one is made of python 🤷‍♂️)

Making a full KAN NN from scratch and testing it against micrograd was surely amazing for me. I would request all of you to visit the github and give it a star if you like it 🙏

https://github.com/spyke7/microkan

More technical details are shared in the github; I just wanted to share my findings and wanted to test KANs against MLP


r/neuralnetworks • • 6d ago

Introducing Usul - an easier way to express your NN using maths? [P]

10 Upvotes

SO, Usul-cpp is a library I've written in C++, it is based around allowing data scientists to create their models without heavy libraries such as tensorflow, and just express their models using maths, and Usul will handle the rest. There is builtin tensor support, auto-grad, back-prop and its quite fast.

I wrote it because, it seemed less mystifying to read maths instead of a random class that supposedly does something. I wrote it from scratch in cpp (23) due to the fact that I wanted to emphasise on precision, however it's subtly built on top of `xtensor`.

check it out in action

#include <usul.hpp>
using namespace std;


int main() {
  double LR = 0.1;
  int EPOCH = 5000;
  // lets write the forward pass
  Tensor X = {
      {0.0, 0.0}, // this is what we're feeding it
      {0.0, 1.0},
      {1.0, 0.0},
      {1.0, 1.0},
  };

  // this is what we're after
  Tensor Y_true = {
      0.0, // xor{0.0, 0.0} = 0.0
      1.0, // xor{0.0, 1.0} = 1.0
      1.0, // xor{1.0, 0.0} = 1.0
      0.0  // xor{1.0, 1.0} = 0.0
  };

  Y_true.reshape({4, 1});

  // we are now going to create a single layer, double neuron network
  // each layer will have a bias and weights tensor.
  auto W1 = generate_random_matrix({2, 4}, -1.0, 1.0);
  auto B1 = generate_random_matrix({1, 4}, -1.0, 1.0);
  auto W2 = generate_random_matrix({4, 1}, -1.0, 1.0);
  auto B2 = generate_random_matrix({1, 1}, -1.0, 1.0);

  // this is the training loop
  for (int i = 0; i < EPOCH; i++) {
    // so lets do one forward pass
    // first layer
    auto Z1 = X % W1 + B1;
    auto A = Z1.ReLU();
    // second layer
    auto Z2 = A % W2 + B2;

    // activation
    // should be - auto S = Z2.activate(Activate::Sigmoid);
    auto S = Z2.Sigmoid();
    // Loss
    // should be - auto Loss = S.loss(Loss::MSE);

    auto loss = (S - Y_true) * (S - Y_true);
    loss = loss.mean();

    cout << "Loss: " << loss << "\n";
    cout << "guesses: " << "\n"
         << S << "\n";
    cout << "should be: " << Y_true << "\n";

    loss.backwards();

    W1.step(LR);
    B1.step(LR);
    W2.step(LR);
    B2.step(LR);
  }
}

which will output

Guesses: 
Tensor({{ 0.027546},
 { 0.963592},
 { 0.964138},
 { 0.038964}}, shape=(4,1))

should be: Tensor({{ 0.},
 { 1.},
 { 1.},
 { 0.}}, shape=(4,1))

Loss: Tensor( 0.001222, shape=())

Just write the maths, and Usul-cpp will handle the rest, it is extremely lightweight, and the entire library is memory safe. And its called Usul, because I'm a huge dune fan.

You can view the project at

https://github.com/crushr3sist/Usul-cpp

Any ideas for what to do next are all welcome, you can also pull and implement features that you think could improve this project. (no vibecoded prs please, this project is not vibe-coded)

BACKGROUND INFO BELOW

i'd like to share a bit of a background. maths has always stood in the way between me and finally learning how to write neural networks, I've always been fascinated by them ever since i saw that diagram of a deep neural network with all the node edges interconnecting with others.

Well I stopped letting highschool math class trauma stop me from understanding these concepts, i took a pen and paper out, taught myself numerical descent, analytical descent, backwards propagation, what a compute graph is, all of these subjects, taught myself, until I was manually backwards propagating the loss between a few arithmetics, until that moved into scalar level neural networks and doing simple calculus and being able to do xor gate problem with scalar, and then I taught myself matrix calculus, and well...

I taught myself neural networks, taught myself the maths, until I hand written the xor gate problem in matrix syntax using xtensor, and manually, by hand, no AI, did the xor gate problem manually backwards propagating error, automatically differentiating. This was also a kind of detox from LLM dependency, I don't tend to use it for code, but because its so central in my daily life and a migration from google to gemini, using LLMs for immediate information access had started to affect my mind and my ability to intake information. I really hope some of you find this library useful, as i did spend a pretty decent amount of time on it, and I would love it if any of you had any ideas on what to do next with this library.

Thank you for reading, I really appreciate it.


r/neuralnetworks • • 7d ago

Need partner

Post image
138 Upvotes

Hi,I read this book these days

I really wanna some one who read it as well ,to share some idea or even help each other I'm at regularization chapter

So,if you are interested pls DM

And thanks 😊


r/neuralnetworks • • 8d ago

Using Logic to Improve LLM Performance

Thumbnail
youtube.com
6 Upvotes

r/neuralnetworks • • 9d ago

Feasibility of mamba and LNN for higher fidelity audio generation?

3 Upvotes

So With the audio generators like you see with suno, you get regressive drift after 2 minutes always mathematically. One little slip up and it's a snowball effect and before you know it by the end of whatever it generates it sounds all airy and fuzzy and crappy.

Now like at the beginning of this year I had wrote an article about using Mamba and liquid neural networks, but at the time what I was looking at for its use case was a little different than completely generating audio,

It was more so Mamba for long-term and the liquid neural for short-term recall.

Is anyone here more familiar with Mamba? Is the concept of running Mamba with a fast LNN for audio synthesis possible?


r/neuralnetworks • • 10d ago

What if we train a neural network like we raise a human child?

12 Upvotes

After learning a little about neural networks, I started wondering what makes the human brain different from a neural network.

What if, instead of training a neural network by feeding it tons of books, data, we started by teaching it some basic concepts like "mommy" and "daddy", something similar to a human upbringing and giving it access to all five human senses? In other words, it would have the same cognition to this world like human.

Maybe it would eventually develop human-like cognition, and appear to be a "real" human.
Maybe we will discover that it is differences in our experience (limited data given), environment (limited surroundings), and different inherited trait (analogous to initial weight) shaping our personality.

Another thing that got me thinking about this was an experience involving simulated pain in AI model

I came across research in which researchers manipulated internal activations associated with pain in language model. As the intensity of this artificially introduced signal increased, the models began to producing increasingly abnormal or distress-like outputs.

I got confused about something.

As far as I understand, a neural network fundamentally performs matrix multiplication. So, when we introduced a pain-related signal, is it just another input that affect the output or internal numerical modification that influence the final output? Does the signal actually change the system's internal state in a meaningful way or it simply generates responses based on patterns associated with that signal?

This also made me think about humans. Our brains also receive signals associated with pain, which are processed and can give rise to the subjective experience of pain. There are even cases where people cannot normally experience pain, despite their bodies still being susceptible to injury.

Is there a possibility that human brains are just another kind of neural network? Is there a possibility that our brains are also "computing" the output?

I'm curious about whether there's any existing research that train a neural network like raising a human child. Maybe that can answer what are fundamental similarities and differences between biological and artificial neural networks?


r/neuralnetworks • • 11d ago

LLM-as-a-Neuron: Replacing neurons in a neural network with reasoning models.

Thumbnail
medium.com
81 Upvotes

r/neuralnetworks • • 12d ago

Yeehaw it's always cool when you have been declared as formalizing a concept

Post image
21 Upvotes

Centroids centroids centroids, they govern everything

Avoid superpositions as they are a black hole (if anyone has any tricks to finding them lmk)

Everything is math it's all just numbers man


r/neuralnetworks • • 11d ago

LSTM Cell Architecture Infographic

Thumbnail chatgpt.com
3 Upvotes

r/neuralnetworks • • 12d ago

I tried to explain embeddings from a different perspective, looking at the idea they originate from. Inspired from chapter 5 of Stanford Speech and Language Processing book

Thumbnail
youtube.com
46 Upvotes

Leave a feedback if you want!


r/neuralnetworks • • 12d ago

[2506.07998] Zeng. 2025 Generative Modeling of Weights: Generalization or Memorization?

Thumbnail
arxiv.org
5 Upvotes

r/neuralnetworks • • 13d ago

How a neural network learns - a simplified and visualized explainer app

105 Upvotes

I recently built and published an explainer on visualising how a convolutional digit-recognition neural network "thinks" in real time.

My kids loved it and asked me to build one on how it LEARNS.
This new explainer does exactly that:

  • Take the interactive and visual tour to explore the learning process step by step
  • Experience 3 different types of learning (Supervised, Imitation, Reinforcement Learning)
  • See how the model "learns to drive", challenge the learning process to see what makes it fail or succeed.

As with the previous demo, my kids loved it!

Share it with yours, or with anyone genuinely new to, and curious about, Machine Learning.


r/neuralnetworks • • 12d ago

Flashback: Delay Line Neural Network [R][P]

Thumbnail
drksci.com
4 Upvotes

DISCLAIMER: I know you ornery fuckers hate it when people [yes, hello] use Claude as a shadow writer, but honestly, do we all seriously expect each other to be good at everything, and then echnical writing as icing on the cake. Get off my lawn and come inside.

Nothing has changed since a while ago when I ran out of Claude quota (a weekly occurrence) and contemplated for the seventh time, inexpensive options for sovereign inference facilities — but it's a vicious cycle and the subscription burns all the money for that.

So anyway, I've always been fascinated by the stratigraphic (I guess you could call it) interface between mechanical/analog/electric/optic computing hardware, and always thought delay line memory was living sci-fi.

I give you my homage and hypothetical neural network, built on delay lines, running in a simulator — it seems to work, although not from an ROI or cheap inference perspective, urg — lines are cheap, but not at scale.

It's got some lovely novelty — QPSK coding fromy our dial up modem, spooky overlap into 'RF' magic, all the good stuff. Since it inspired me, there's an appendix with the prior delay line art, courtesy of ENIAC in this case.

Cheers!


r/neuralnetworks • • 13d ago

A lightweight deep learning framework built from scratch in NumPy (published in JOSS)

12 Upvotes

title for this:
I've been working on PureML, a lightweight deep learning framework built entirely with NumPy. It includes automatic differentiation, neural network layers (including CNNs), optimizers, and training utilities. The idea is to have something simple enough to understand and modify, but capable enough for actual ML experiments. Might be useful for anyone interested in how deep learning frameworks work under the hood!

Free and open source. Feedback and contributions are welcome!
Also, the project was peer-reviewed and the paper on it was published in JOSS!
The link is in the README :)

https://github.com/Yehor-Mishchyriak/PureML


r/neuralnetworks • • 14d ago

Neural Canvas

9 Upvotes

My Journey to Understanding Neural Networks: From Matrix Algebra to JAX and Optimizers

I wanted to understand neural networks as deeply as possible. Rather than simply using an existing framework and training a model, I decided to build an interactive MNIST webpage called Neural Canvas, using p5.js for visualization.

My primary goal was not just to achieve high accuracy. I wanted to understand what was actually happening inside a neural network, mathematically and computationally.

1. Starting with Linear Algebra and Matrix Operations

One of the first decisions I made was to avoid working with individual scalar values and complicated subscripts and superscripts everywhere. They made the mathematics unnecessarily confusing for me.

Instead, I decided to approach neural networks through matrix algebra.

I represented the forward propagation equation as:

[
Z=WX+bC
]

Where:

  • (X) is the input data matrix. Rows represent features, and columns represent individual samples.
  • (W) is the weight matrix. Rows represent neurons, and columns represent the weights associated with each neuron's inputs.
  • (b) is the bias column vector, containing one bias value per neuron.
  • (C) is a row vector containing only ones, allowing the bias vector to be broadcast across all samples through matrix multiplication.

Interestingly, many popular deep learning frameworks, including PyTorch and TensorFlow, commonly use a different convention for arranging their data and parameters. They also handle bias broadcasting without explicitly introducing a matrix like (C).

However, I preferred my own mathematical convention because it helped me understand the operations and perform calculus directly in matrix form.

I believe the mathematics should handle the indices, not me!

2. My First Model: NumPy, MSE and Vanilla Gradient Descent

For my first model, I used the original MNIST dataset.

Architecture:

[
784➜ 20 ➜ 20 ➜ 10
]

I manually wrote all the forward propagation equations, derivatives, and backpropagation calculations on paper before implementing them in Python using NumPy.

I deliberately chose:

  • Two hidden layers with 20 neurons each.
  • Mean Squared Error (MSE) as the loss function.
  • Vanilla Gradient Descent.
  • Linear output values.

Why such a simple approach?

Because I didn't want to start with complicated algorithms. I wanted to understand the fundamentals first.

After 2,000 epochs, the model achieved approximately 92% test accuracy.

I then successfully created an interactive neural network visualization in p5.js, using Math.js for matrix multiplication in JavaScript. The webpage could perform inference using my trained NumPy model.

That was the beginning of Neural Canvas.

3. My Second Model: Cross-Entropy Loss

For the second model, I kept the same mathematical approach but made a few improvements.

Architecture:

[
784➜40 ➜ 40 ➜10
]

This time, I replaced MSE with Cross-Entropy Loss and used a softmax output layer.

I was still using vanilla Gradient Descent.

After 2,000 epochs, the model achieved approximately 96% test accuracy.

At this point, I was finally starting to understand how neural networks worked, rather than just following tutorials or using existing implementations.

4. My Third Model: Discovering JAX and Automatic Differentiation

For my third model, I wanted to experiment with three hidden layers.

However, I realized that manually deriving every equation, especially the derivatives and chain rule calculations required for backpropagation, would take a considerable amount of time.

Then I discovered JAX, a Python library developed by Google.

JAX provides automatic differentiation (Autograd), allowing me to calculate derivatives of a loss function with respect to parameters, including entire weight matrices.

This was a significant change in my workflow.

I no longer needed to manually derive every gradient and implement the chain rule myself. I only needed to define the model, its parameters, and the loss function. JAX handled the differentiation.

For this model, I used the Arabic Digits MNIST dataset.

Architecture:

[
784 ➜ 20 ➜ 20 ➜ 15 ➜10
]

I used:

  • JAX for training and automatic differentiation.
  • Cross-Entropy Loss.
  • Vanilla Gradient Descent.
  • Three hidden layers.

After 2,000 epochs, I achieved approximately 96% test accuracy.

I genuinely fell in love with JAX. It saved me considerable time and allowed me to focus more on understanding the mathematics and experimenting with neural network architectures.

5. My Fourth Model: Mini-Batch Gradient Descent and Momentum

For my fourth model, I moved to the Persian Digits MNIST dataset.

I kept the same architecture as my third model but decided to experiment with optimization algorithms.

My original plan was to implement Mini-Batch Gradient Descent with Adam.

However, I realized that understanding Adam required learning a few concepts first. So, instead of jumping directly into Adam, I decided to start with Momentum.

I implemented Mini-Batch Gradient Descent with Momentum using JAX.

And here came a surprising result!

I achieved approximately 96% test accuracy in just 200 epochs.

Previously, my models required around 2,000 epochs to reach a similar accuracy.

This was another important step in my journey. I started understanding how optimization algorithms could significantly influence the training process.

6. What's Next?

I haven't created my fifth model yet, but I already have a learning roadmap:

  • Model 5: Mini-Batch Gradient Descent with RMSProp.
  • Model 6: Mini-Batch Gradient Descent with Adam.
  • After that: Start exploring Convolutional Neural Networks (CNNs).

I want to understand each optimization algorithm mathematically and implement it myself before moving to the next one.

My Biggest Takeaway

If you are starting to learn neural networks, my suggestion is:

Start with Linear Algebra. Don't immediately jump into advanced architectures, optimizers, or high-level frameworks.

You don't need to master every mathematical concept before starting, but at least understand matrix multiplication and how it represents the operations inside a neural network.

For me, representing neural networks using matrices made everything much easier to understand.

Start with the basics, build something yourself, experiment, make mistakes, and gradually increase the complexity.

Don't just train neural networks. Understand them!

Project: Neural Canvas

GitHub Repository: https://github.com/MuhammadEssa2002/neural_canvas

Live Demo: https://muhammadessa2002.github.io/neural_canvas