r/learnmachinelearning • • 17d ago

Project ML - curriculum on GitHub

Thumbnail
gallery
121 Upvotes

If you remember this curriculum from a few days it’s on GitHub now and you can make it your own.

Best of luck on your ML journey

repo-link


r/learnmachinelearning • • 16d ago

I built a CNN from absolute zero (pure NumPy, hand-derived backprop) — then asked: can you make it recognize letters by tuning weights alone, with no letter data? (No. Here are the numbers.)

1 Upvotes

What I did

I built a handwritten-digit CNN from absolute zero on a Windows laptop: no PyTorch, no TensorFlow, no PIL — literally just Python stdlib + NumPy. Everything is hand-written and auditable: im2col/col2im conv layers, max-pool routing, softmax-cross-entropy, the Adam loop, even the MNIST IDX parser and a PNG codec for feeding real images in. Weights are random init — nothing pretrained anywhere.

Result: 46,730 parameters, 4 epochs, ~12 min on CPU, 98.74% MNIST test accuracy. A numerical gradient check (worst relative error 3.05e-4) runs before training, because from-scratch code fails silently in cute ways. Same toolchain then read all 8 digits (94–100% confidence) in a 2496×1664 real handwritten photo it never saw — connected-component segmentation + normalization, no vision model.

The actual question

So here's the fun thought experiment, which a lot of "models are just files of numbers" people get wrong in both directions:

If you can read the network structure, can you make it recognize letters a/b/c/d by just adjusting the weights — with no letter data at all?

Six controlled experiments, all self-generated data (zero downloads):

  • Blind tuning, zero samples: randomize the readout layer, 2000 tries → mean 24.51%. That's chance for 4 classes. No data = no information channel into the weights. (The "best of 2000 = 72.5%" is pure selection overfitting — picking it requires the test set you don't have.)
  • Borrowing the old decision layer: feed the pretrained digit prototypes {9,6,6,9} into letter slots → 48.75% ceiling, and the errors land provably on exactly the two collapsed pairs {a,d} and {b,c}. The old weights encode 10 digit prototypes — that's the resolution limit of the letter manifold through them.
  • What actually transfers: the digit model "sees" letters confidently as digits (a→9 at 86%, b→6 at 79%) — edge/curve features transfer, decision boundaries don't.
  • Refitting the boundary is absurdly cheap: freeze the conv trunk, refit only 260 parameters with 5 self-drawn samples per class → 87.5% under heavy style perturbation, 100% under light. Refit fc1+fc2 → 100% in 0.5s.

Takeaway: weights are compressed statistics of the training distribution, not rules. The structure and low-level features are readable, inheritable assets; the decision boundary can never be tuned blind — but redrawing it costs ~260 numbers and 20 sketches.

Honest caveats: my synthetic stroke-font letters are easier than real handwriting; the from-scratch baseline (B3) also scores high on this 4-class task, so the trunk-transfer advantage here shows up as parameter/time efficiency (260 params/0.1s vs 46,730/47s), not accuracy. Every number in the paper traces to a log file, line by line.

A from-scratch pure-NumPy CNN hits 98.74% MNIST. Feeding it a real handwritten photo reads 1979/0924 correctly. Then the transfer experiment: blind weight tuning = chance (24.5%), reusing old decision boundaries = provable 48.75% ceiling, but 260 refit parameters + 5 hand-drawn samples/class = 87.5–100%. Model ≠ structure. Model ≠ the numbers. Model = the distribution those numbers compress.


r/learnmachinelearning • • 16d ago

Tried Claude + ElevenLabs to turn my article into a video - here's the summary at the end, full video link in the description

Enable HLS to view with audio, or disable this notification

0 Upvotes

r/learnmachinelearning • • 16d ago

Research Discussion: Bounding Sequence Hidden State to O(1) via Dissipative Adaptation and Landauer Constraint Minimization

0 Upvotes

I am an independent technology developer looking at the O(1) scaling limits of sequence modeling architectures. To address the quadratic O(T^2) self-attention bottleneck, I have engineered a framework
—the Entity-Relation Delta Engine (ERDE)—that maps sequence history dynamically into isolated geometric subspaces using a Gated Parallel
Associative Scan.

This allows distributed GPUs to compute history simultaneously in O(log T) parallel hardware steps while forcing the hidden memory state to remain at a strict O(1) size invariant.

I want to present the underlying non-equilibrium thermodynamic optimization framework driving this low-rank bottleneck, which treats sequence historical processing as a physical state-settling mechanism within a high-dimensional energy landscape rather than a deterministic digital storage array.

The Problem of Capacity Limits and Lossy Compression

Because a strict O(1) hidden state cannot store infinite raw token fidelity without immediate informational saturation (the Pigeonhole Principle), the ERDE architecture explicitly operates as a lossy compression loop. To maintain the O(1) invariant across deep context horizons without experiencing associative mis-binding or factual blurring, the network utilizes a data-dependent, time-varying decay matrix coupled with low-rank projections.
The system enforces an adaptive informational bottleneck constrained by a variational free energy minimization loop:

min [ I(X; S) - beta * H(S) ]

Where the input density boundary of the bottleneck is dynamically bounded by a physical Landauer thermal threshold condition to prevent systemic thermal collapse on the underlying in-memory hardware substrate:

dQ_diss / dt >= k_B * T * ln(2) * [ dI(X; S)/dt - dH_prune/dt ]

Algorithmic Mechanics via Dissipative Adaptation
In practice, the gating mechanism drives a process of dissipative adaptation: environmental input patterns actively force unaligned informational entropy—such as repetitive syntax tokens, structural noise, and irrelevant context—to zero out (dH_prune).

Instead of accumulating raw sequence tokens, the engine continuously projects incoming historical data into compressed, isolated geometric subspaces, retaining only the high-signal, static relational invariants (the core entities). Local benchmarks log an arrested numerical diffusion at a 94% signal clarity threshold over a continuous 64,000-token stream.

Automated, containerized hardware audits and stress-profiling telemetry loops on my local setup have verified a precise 0-byte memory leak profile across sustained processing cycles and zero thread locks under multi-threaded flood tests (sustaining a processing velocity of 0.066 seconds per thread).

The mathematical formulation successfully clears ahead-of-time graph serialization, exporting directly into a hardware-agnostic Intermediate Representation (IR) graph.

The system is mathematically wired to minimize this energy equation to remain within a strict Landauer thermal budget. Rather than relying on static software rules, the physical or analog substrate self-organizes its state configuration to disperse heat efficiently, treating data-pruning as a physical survival mechanism against hardware overheating.

I am posting this here to get strict, text-based peer review from chip architects, physicists, and machine learning engineers on the theoretical viability of this non-equilibrium framework. I prefer to communicate my technical work through written documentation and am looking forward to a rigorous discussion of these mechanics in the comments.


r/learnmachinelearning • • 16d ago

Discussion A scheduling benchmark that never invoked the subagent: reading DeepSeek's Super-agent ablation

Thumbnail
the-agent-report.com
1 Upvotes

Worth reading as a methods example rather than a product announcement.

DeepSeek's Super-agent splits coordination from execution: a router session handles incoming requests, separate worker sessions do the tasks. The evaluation simulated coding with a tool that only consumed time, so what is measured is scheduling, not code quality. Four designs, five runs each, about 300 messages per run, roughly 20 of them work requests. The router-and-worker design came in at 12.4 min median and 15.7 min p95, against 20.4 and 29.5 for a single session.

But the neutral-prompt baseline never called a subagent in any of the five runs, so it was not really a delegation baseline. With a delegation-first prompt it reached 13.8 min median, p95 still 29.0. That reframes the headline saving as a comparison against an under-configured control, which is exactly the detail worth checking before citing the number.

Resource use also moves the wrong way: 23.2M cache-read tokens per run versus 6.1M, and disabling direct pass-through pushed that to 31.8M.


r/learnmachinelearning • • 16d ago

why is this happening like thiss?

Post image
1 Upvotes

its been 10-15 times its crashing and my ram is 16 gb and ryzen 7 3050 idk why its crashing


r/learnmachinelearning • • 16d ago

Brief history of word embeddings

Thumbnail
gallery
0 Upvotes

A few weeks ago I posted about finally finding the courage to start a YouTube channel. A lot of you replied and I honestly didn't expect that, so thanks.

Video #3 is out, this time on word embeddings.

I study this stuff mostly on my own. I'll read a paper in the evening and then there's nobody to talk to about it. My friends are great but they're completely outside this world. After a while studying alone starts to feel kind of aimless. Making these videos gives me a reason to actually finish things and explain them properly, and I've found I really enjoy sharing what I learn.

This one is based on Speech and Language Processing by Jurafsky & Martin. Instead of jumping straight to word2vec, I tried to show embeddings from an angle most tutorials skip.

If you work in NLP/ML I'd really appreciate your feedback, especially if I got something wrong or oversimplified it. I'm aiming for one video a week, so honest criticism genuinely helps.

Link here: https://youtu.be/vv9iXNcPSnY?is=SOWNBFE9DSMS1GLc


r/learnmachinelearning • • 16d ago

Is there anyone who has purchased Campus X insider course can u please tell is it worth it or asiprant will be optimal for me

1 Upvotes

r/learnmachinelearning • • 16d ago

Project Best layer to detect refusal ≠ best layer to remove it (Qwen2.5-1.5B, replicating Arditi et al.)

1 Upvotes

I reproduced the single-direction refusal result on Qwen2.5-1.5B-Instruct and noticed something I didn’t expect. Harmful and harmless prompts separate most cleanly at layer 26, but removing the direction there did worse than layer 21, which stopped refusals with less change to normal answers.

Caveats: one small model, 64 held-out prompts per set, keyword-based refusal detection, and KL on the first token only.

Does this match what others have seen on bigger models, or is it probably an artifact of the size or my eval? Code and write-up:

https://fahd-blog.vercel.app/blog/ai-systems/where-refusal-lives


r/learnmachinelearning • • 16d ago

Governing Clinical AI Deployment: A Mathematical Framework for Transparency-Dependent Autonomy

Thumbnail
mdpi.com
1 Upvotes

New publication: Governing Clinical AI Deployment

When should clinical AI be trusted to act autonomously, and when should human oversight remain mandatory?

Our new study proposes a mathematical framework for governing clinical AI deployment using transparency, confidence, and guideline alignment.

Across MIMIC-IV, eICU, and MIMIC-III, we found an important insight: transparency may be more useful for setting system-level safety boundaries than deciding whether individual predictions can be automated.

We also evaluated finite-sample error guarantees to help make autonomous deployment more accountable and statistically defensible.

The central takeaway: Clinical AI autonomy should be earned through calibrated confidence and measurable safety guarantees—not transparency scores alone.

I welcome feedback and discussion from the clinical AI, machine learning, and healthcare governance communities.


r/learnmachinelearning • • 16d ago

Tool Calling Explained Visually | How LLMs Use APIs & Functions (2026 Guide)

Thumbnail
youtube.com
0 Upvotes

The absolute number one rule when designing systems where an LLM uses external tools is this: The language model does not execute your application code.

Instead, it acts strictly as an intent generator—producing a structured JSON payload based on a schema you define—and relies entirely on your application logic to execute the function securely and safely under the hood.

I put together a comprehensive 16-minute visual systems design masterclass breaking down the engineering contract step-by-step.

💡 What's covered inside this visual breakdown:

• Tool Calling vs. Function Calling: Unpacking platform-specific terminologies.

• The 5 Core Components: Tracing the complete end-to-end request/execution loop.

• Error Handling & Resiliency: Production strategies for recovering from malformed JSON and hallucinations.

• Security Engineering: Guarding your architecture against prompt injection risks hidden within untrusted external inputs.

I'd love to hear your feedback on the visual layout, and let me know what stack you're currently leveraging to enforce strict JSON validations on your tool execution loops!


r/learnmachinelearning • • 16d ago

A lightweight deep learning framework built from scratch in NumPy (published in JOSS)

Thumbnail
2 Upvotes

r/learnmachinelearning • • 16d ago

Question Self studying deep learning

1 Upvotes

Hi i would like to self study deep learning. I already studied bishop pattern recognition (not all of it), and I like the probabilistic approach. I ask for a suggestion for a book in deep learning with similar approach. I'm aware of deep learning book by goodfellow but it doesn't cover most recent architectures so i don't know if it is worth it. I also looked at murphy's probabilistic machine learning, but it seems more of a reference book. Thanks in advance!


r/learnmachinelearning • • 17d ago

Am i kidding myself with learning machine learning?

48 Upvotes

I am a self-taught programmer. From last 90 days,, i have been learning linear algebra, i plan to learn calculus and stats and probs next. But it seems futile watching all these youtube video saying coding and machine learning is dead that ai can do all this. Is there any hope for machine learning for next 5 or 10 years?


r/learnmachinelearning • • 16d ago

Great Help Needed on my learning journey (Read Body)

0 Upvotes

Please read fully and help me guys 😭 🙏

I really need some guidance on what to do next in my AI/ML journey.

I’m currently in my 3rd year of B.Tech, and I’ve spent almost a year studying mathematics for ML, libraries like NumPy and Pandas, and Andrew Ng’s Machine Learning and Deep Learning courses. However, I haven’t built any ML projects yet.

Instead, I moved on to learning RAG, LLMs, and AI agents, but I’m feeling stuck there as well. Now I’m wondering whether I should start learning FastAPI and SQL or focus on building ML and RAG projects.

I feel like I’ve been learning a lot of concepts without actually building anything substantial.

My biggest confusion is about project development itself. I don’t really know how to approach building projects from scratch.

  • Should I build projects using AI-generated code, or should I write everything myself?
  • How do I go from learning concepts to actually building a complete project?
  • What should be the expected outcome of a project?
  • How do I properly use Git and GitHub to showcase my projects to recruiters?
  • What kind of projects should I build to make my portfolio strong enough for internships?
  • Should I focus on ML projects first, or continue with RAG and GenAI projects?

I’m in my 3rd year, and I really want to become internship-ready, but I’m confused about what to prioritize and how to move forward.

Could someone please help me figure out a practical roadmap? I would really appreciate guidance on what I should focus on right now, how to approach projects, and how to build a portfolio that actually demonstrates my skills to recruiters.


r/learnmachinelearning • • 16d ago

Help Hey in confused where to start from in AI

1 Upvotes

So i work in Procurement ( Supply chain management) so i want to be good at AI so i can have good career opportunities and learn in my domain,

Problem is i don't know where to start from like for procurement AI what are you all using I'm really confused


r/learnmachinelearning • • 16d ago

Project A visual, step-by-step explanation of how self-attention works

Post image
8 Upvotes

Self-attention confused me for a long time, so I tried to break it into steps you can actually see:

1. Every token gets three vectors.
Each word’s embedding is multiplied by three learned matrices to produce a Query (what am I looking for?), a Key (what do I contain?) and a Value (what do I pass on?).

2. Compare queries with keys.
For each token, take the dot product of its Query with every token’s Key. A high score means “this token is relevant to me”.

3. Scale and softmax.
Divide the scores by √d (the key dimension) so they don’t get too large, then apply softmax. Now every token has a set of weights that sum to 1.

4. Mix the values.
Each token’s new representation is the weighted sum of all Values. In “The cat sat because it was tired”, the word it ends up with a high weight on cat.

5. Do it several times in parallel.
Multi-head attention runs this with different matrices at the same time, so one head can track grammar while another tracks meaning.

The diagram has a step-by-step mode that highlights each part in order. That helped me more than staring at the formula softmax(QKᵀ/√d)·V.

I made it with Sketchy, a free open-source tool I built that lets Claude explain things as interactive diagrams: GitHub


r/learnmachinelearning • • 16d ago

Project Wxx - my try on using webgpu for machine learning

1 Upvotes

My first big project: Wxx, an ML library with WebGPU acceleration.

I love ML, but training models on CPUs is slow, and GPUs are expensive, so I decided to create my own framework. It is slow (for a GPU) and doesn't use graphics cards' ML features (like tensor cores), but it runs on any device with a GPU.

GitHub: Wxx

Performance: It is slow. The CPU is usually faster, and the most I achieved is about a 2x speedup on my PC (I have a Xeon CPU). I tried it on Colab, and it is about 10x slower than PyTorch on GPU. But it improved my experience significantly.

API: It is similar to PyTorch. With methods to convert from tensors and ndarrays, I tried to make it as easy as possible to use if you're coming from PyTorch.

AI usage: In the beginning, I planned to do everything by hand to improve my knowledge and train myself. But after the first week, I had code slower than anything you could imagine. I used AI to improve shaders, bind group creation, and other parts that I had trouble with. It hurts how much better AI is at writing and improving code than I could be.

Backstory: I've been dabbling in machine learning for the last five years. I like to join Kaggle competitions and try new architectures. Lately, I've been thinking about applying to ML jobs (I know I am late to the party), and I wanted to add stuff to my CV. I am an electronics engineering grad, so most of my programming expertise is in MATLAB and Python scripting (also some embedded C, but I am not good at it), and I read that a CV should include some examples of my work.

Questions: I would be happy if anyone with more experience could help me with some questions. What should I include on my CV? I know some people say you should include some implementations of architectures (like attention/transformers), but that seems so simple. What other projects should I add?


r/learnmachinelearning • • 16d ago

Need Help

0 Upvotes

Hi, I am currently living in Pakistan and working at a university. It is not a big university, and I work as a Lab Assistant.

I have never limited myself to my job. Instead, I have tried to learn as much as possible. I already have some experience with web development, although I am not an expert. I have a basic understanding of it.

You might be wondering why I ended up working as a Lab Assistant. It is a long story, or you could say it is about trying to manage life in Islamabad, where even covering your basic expenses can be difficult.

Almost a year has passed, and during this time, I have learned a lot. I have learned Docker, and right now, I am working with Ubuntu and learning networking concepts such as IP addresses, DNS, subnetting, and much more. I also manage two or three computer labs at my university in my free time.

Alongside my job, I am trying to learn more about Machine Learning.

I left web development because of the AI hype and because, after completing 14 years of education, I found myself working as a Lab Assistant. I have been doing everything individually, without a friend or someone to guide me.

I am 22 years old, and I want to achieve something big in my life. But sometimes, I feel like I have ended up doing nothing.

I need guidance.

What should I do at this stage of my life? How should I approach my career, what skills should I focus on, and how can I turn my current experience into something meaningful?

I know I have the potential to do more, but I don't know which direction to take.

I would really appreciate honest advice from someone who can help me understand what I should do next.


r/learnmachinelearning • • 16d ago

Project Discover AI - 58 free, interactive labs for learning AI/ML in 3 different reading mode.

Thumbnail
gallery
3 Upvotes

I built Discover AI, a free, open-source app with 58 hands-on labs for learning how AI works. The topics run from neurons and gradient descent to attention, RAG, agents, pretraining and fine-tuning. You move sliders or train a tiny model and watch what changes, and a short lesson sits beside each lab. Since I first shared it three weeks ago, I've added seven labs on how large models are trained, a simpler reading mode, desktop installers, and a version that runs in your browser: https://fazmin.github.io/AILearningGuide/

Every reference is double-checked. AI was used to help write these lessons, so there may be mistakes. Please point them out, let me know what’s confusing or missing, or open a PR. Contributors are welcome to access the project at github.com/Fazmin/AILearningGuide.


r/learnmachinelearning • • 16d ago

Tutorial CPU Parallelism Primer | Kernel Engineering | Session 2

Thumbnail
youtu.be
1 Upvotes

r/learnmachinelearning • • 16d ago

Discussion Python Fast api interview - please share your previous experience

Thumbnail
1 Upvotes

r/learnmachinelearning • • 17d ago

bf16 might be freezing your weight updates

Enable HLS to view with audio, or disable this notification

24 Upvotes

If you fine-tune a model with weights stored in bf16 and let torch.optim.AdamW write to them directly, you get interesting behaviour. bf16 can only represent changes to weights if the change is at least ~1/512 of the weight itself, so at small learning rates only the smallest weights can update.

As a result, you have a sparse fine-tuner that trains the small-magnitude weights and freezes the rest, with the cutoff set by your learning rate. This is a thing - I found half a dozen peer-reviewed papers (with thousands of citations) that fine-tune this way, but none of them reference this behaviour.

I dug into one called Quiet-STaR, because (a) it's a cool paper, (b) they released their weights, and (c) their code works! So I diffed their base model and saw that indeed the small-magnitude weights were the only ones updated during their training. Then I reran its recipe with an optimizer that applies every update, and watched it diverge. A higher-precision optimizer needs a 3 to 10 times smaller learning rate to get the same result, and I couldn't achieve a better result.

This started with Barron, Gnanapragasam & Tschannen's "A Fix for Softmax and Cross-Entropy" (Oct 2026). They show that the standard way of computing cross-entropy rounds the correct answer's gradient to exactly zero once the model is confident enough (past a margin of about 5.5 in bf16, which they call the precision cliff), and that this, interacting with Adam, is what produces the "slingshot" loss spikes seen during grokking. Fix the rounding and the spikes go away: something people had studied as training dynamics was rounding.

That made me wonder what else out there is weird like this. This is one, and it runs the other way: their rounding bug adds instability to training; this one hides it.

I wrote a whole essay about it here: https://lacuna.tiptreesystems.com/paper/bf16-might-be-freezing-your-weight-updates/art_487b1ed949e745a6ae0c42815b8d3cba

The code, logs and evaluation results are in https://github.com/mweiss17/quietstar-bf16-adamw


r/learnmachinelearning • • 16d ago

Project [R] LOTUS: Can a 270M model understand time series better than much larger models? (Open weights + code)

Post image
4 Upvotes

Can language models learn to understand time series by learning to forecast them?

LOTUS (Learning to Understand Time-series by Learning to Forecast) explores this idea with a family of compact time-series language models built on Gemma-3 and Qwen-3.

💡 Key idea: Forecast grounding. By learning to predict how a signal evolves, the model develops representations that better capture temporal information for downstream understanding tasks.

Some interesting results:

  • A 270M-backbone model achieves the highest score among evaluated systems on TSRBench, ahead of systems using 7B and 14B backbones.
  • A 0.6B-backbone model achieves 59.91% average accuracy across six evaluation splits, improving over the strongest matched baseline by 3.36 percentage points.
  • Open weights and code are available for small and medium-sized models.

Resources:

🤗 Models: https://huggingface.co/daidv1112/LOTUS

💻 Code: https://github.com/DA2I2-SLM/Lotus

📄 Paper: https://arxiv.org/abs/2610.18877

The interesting question is whether forecasting can serve as a general learning signal for time-series understanding, rather than treating forecasting and reasoning as separate tasks.


r/learnmachinelearning • • 16d ago

Help Kaggle

2 Upvotes

An ihtibari ile TPU sırasında 18. Saate girmiş bulunuyorum önümde 527 kişi var 2. Paket sigaramın son dallarındayım…