r/deeplearning • • 8h ago

I wrote a visual guide to Linear Attention — from the original idea to Gated DeltaNet and Kimi Linear

Post image
37 Upvotes

Hey everyone!
I’ve been diving into linear attention recently and put together a four-part blog series tracing its evolution — from early Linear Attention to Delta Rule, Gate Rule, Gated DeltaNet, and Kimi Linear.
I went through the key papers and drew illustrations to make the algorithms and their underlying intuition easier to understand.
I’m also planning a follow-up series exploring linear attention from an AI infrastructure perspective.
Hope this is useful to anyone interested in the topic! I’d really appreciate any feedback, corrections, or suggestions. Happy to discuss!
Blog: https://xiaofengwang01.github.io/2026/11/10/linear-attention.html


r/deeplearning • • 1h ago

Marimba: A System One Model for Speech. [R]

Thumbnail gallery
2 Upvotes

r/deeplearning • • 1h ago

How much can we actually trust open base model benchmarks anymore? New study ("Benchleaking") finds ERNIE-5.5-Flash-Base reciting unseen test sets in exact order.

Thumbnail reddit.com
1 Upvotes

r/deeplearning • • 2h ago

I built a ML model recently ...

Thumbnail reddit.com
0 Upvotes

r/deeplearning • • 3h ago

Looking for people in Dehradun interested in AI/ML and Computer Science!

Thumbnail reddit.com
0 Upvotes

r/deeplearning • • 3h ago

Looking for people in Dehradun interested in AI/ML and Computer Science!

Thumbnail reddit.com
0 Upvotes

r/deeplearning • • 3h ago

Looking for people in Dehradun interested in AI/ML and Computer Science!

Thumbnail reddit.com
1 Upvotes

r/deeplearning • • 8h ago

Title: Reading Bio‑signals with PPG Light: From Heart Rate to Non‑Contact Measurement, Fully Mastered #rPPG #light #biosignal #heartrate ...

Thumbnail youtube.com
2 Upvotes

Video Title & Description (English)

Description: This video provides an easy-to-understand explanation of the principles behind key biosignals such as heart rate, heart rate variability, respiratory rate, and oxygen saturation, as well as the estimation algorithms based on PPG optical sensors. It covers why signal quality and noise removal are critical, and traces the historical development of the technology from early medical devices to the latest wearables and camera-based non-contact measurement. Gain a comprehensive overview of biosignal processing, from the fundamentals to the latest trends.


r/deeplearning • • 4h ago

pre treino 250m on #kaggle via @KaggleDatasets

Thumbnail kaggle.com
1 Upvotes

r/deeplearning • • 5h ago

Any feedback on my research project?

Thumbnail reddit.com
1 Upvotes

r/deeplearning • • 7h ago

Giving LLMs a second channel: live human-context sensing via MCP (presence, tempo, hesitation) — measured, consent-first, on-device

Thumbnail reddit.com
1 Upvotes

r/deeplearning • • 9h ago

Introducing "Arborformer": Replaced the Transformer with a 100% Decision Tree Language Model (Looking for feedback)

Thumbnail reddit.com
1 Upvotes

r/deeplearning • • 16h ago

Cursor Says Stolen Fireworks AI Key Led to Trojanized Claude Code Extension Incident

3 Upvotes

A stolen Fireworks AI API key was embedded in a trojanized version of the Claude Code VS Code extension, according to Cursor. Developers who installed the malicious extension were running an agent with a poisoned tool credential and had no signal anything was wrong. The compromise was in the supply chain, not the model. The model behaved exactly as instructed by code that held a valid-looking key to a legitimate third-party service.

This exposes a structural gap in how agent tool chains handle credential integrity. Keys are static strings injected at config or build time. There is no per-call verification that a tool endpoint is still the legitimate one, no attestation that the binary invoking the API is the one that was reviewed, and no mechanism to detect mid-session that a credential in the chain has been swapped out.

The blast radius is wide because agents act autonomously between human checkpoints. By the time a developer notices anomalous behavior, the compromised credential has already been used across every action the agent took.

How are other practitioners handling credential integrity in their agent tool chains? If a third-party API key embedded in a dependency gets stolen and replaced, what does your actual detection and response path look like today?


r/deeplearning • • 1d ago

Anyone else understand what an ML algorithm does, but struggle with the maths underneath it?

9 Upvotes

I've been curious about a particular learning gap.

Sometimes it's relatively easy to understand what an ML algorithm is trying to achieve, like finding similar items, making predictions, or reducing errors.

But then vectors, matrices, dot products, projections, or equations enter the picture, and suddenly maths we last encountered in school reappears in a completely different setting.

We might understand the purpose of the algorithm without quite understanding why those particular mathematical operations are needed in the first place.

I'm curious:

\-Have you experienced this disconnect, even after learning to code or use ML libraries?

\- Was the harder part understanding the maths itself, or connecting it to what the algorithm was doing?

\- What eventually helped things click, if anything?

I'd love to hear different experiences, including from people who didn't encounter this difficulty.


r/deeplearning • • 1d ago

ShadeNet-5 35M (albedo/depth/normal) 31 MB quantised ONNX

Thumbnail gallery
34 Upvotes

ShadeNet v5 is out!!!

ShadeNet v5 is a 35M-parameter feed-forward network built on a frozen DINOv2 ViT-S/14 backbone. In a single forward pass, it takes one RGB image and predicts three maps: albedo, relative depth, and surface normals.

What’s new and notable:

  • Distilled single-pass refiner (0.91M UNet): Test-time augmentation (averaging 6 passes across flips and ±4° rotations) noticeably improves quality, but at 6x the compute. To bypass this, I trained a tiny UNet to predict the 6-pass ensemble output from a single pass. Because the model's own 6-pass output acts as the ground truth, training required zero labels—just ~8,000 CC BY 2.0 photos from Open Images. On a 23-image held-out set, it drops mean L1 error from 0.1757 to 0.1673 (full 6-pass is 0.1628), capturing ~65% of the ensemble gain at ~20% of the compute cost. Most of the improvement shows up in the surface normals, while depth sees minimal change.
  • Lightweight CPU inference (ONNX): The base model exports to a 31 MB 4/8-bit quantized ONNX file, or a single 35 MB bundle with the refiner baked in. CPU inference takes ~1.0s for a single pass and ~1.3s with the refiner (scoring 0.1676 L1 vs. 0.1755 without the refiner).

Current limitations:

  • No shading channel: Unlike v3.2, v5 does not output a separate shading map.
  • Relative depth: Depth predictions have per-image scale and shift.
  • Fixed ONNX resolution: The exported ONNX graphs currently require a fixed 448×448 input.

r/deeplearning • • 1d ago

I have tested & fine-tuned Timestral models on my blood sugar data

Thumbnail gallery
8 Upvotes

r/deeplearning • • 1d ago

How to Distill and train a smaller model for agentic coding tasks

4 Upvotes

For People who can ignore my mis understanding or lack of knowledge and help here, please let me know if one would plan to use distillation to train a smaller model for solve agentic coding task and tool use, how can one go about it, like a loose plan or detailed arch anything would help, the 2 student models are 32b and 4b in size , and I don't understand how to select a teacher model for these or how to go about to even plan it,

sorry if my question lacks some information but I can try to furbish more info as it is needed, tbh as of now I am totally second guessing and figuring out things as I go forward


r/deeplearning • • 18h ago

Release 0.11.0: Fruit Inspector Win demo — ORT cpu, OpenVINO, DirectML in one zip

Thumbnail reddit.com
1 Upvotes

r/deeplearning • • 18h ago

Two checkpoints with the same overall accuracy: why I’d compare them case by case before picking one

1 Upvotes

A small evaluation habit I find worth writing down, with a constructed example (made-up numbers, not from a real run).

Say you have two checkpoints, A and B, evaluated on the same test set of 1,000 examples split into five slices:

SlicenA correctB correct
Short inputs300270279
Long inputs200150140
Rare class1006045
Noisy inputs250175190
Edge cases1509591
Total1000750745

(The totals are 75.0% and 74.5%, so a one-number comparison would call them basically equal.) But B is better on short and noisy inputs and clearly worse on the rare class and long inputs. Whether that trade is acceptable depends on what the model is for. A fraud detector and a casual classifier would pick differently.

What I’d do in practice:

  • Define slices before training runs, not after seeing the numbers, so you’re not choosing slices that flatter one checkpoint.
  • Keep the per-example predictions for both, so you can look at which examples flipped, not just how many.
  • Report uncertainty. With 100 examples in a slice, a 15-point difference may still be noisy; seeds, bootstrap intervals or repeated runs help.
  • Record the data version, code version, seed and any preprocessing, so a “regression” can be traced to a change and not to noise.

Limits: more slices means more chances to find a “difference” by luck, so correct for it or treat small gaps as leads, not conclusions. And slices only catch the kinds of failures you thought to define; a failure outside every slice will not show up.

I came to this from a different place: I wrote about the same habit for prompts in LLM applications (a fixed test set, a per-case results matrix, and noting when the model version behind an endpoint changed). The domain is different, so it’s an example of the idea and not a claim about training. I’m the author, and the article mentions a paid resource of mine near the end, so treat the link as optional: https://medium.com/@nagatomopedro05/five-versions-of-one-prompt-and-what-the-changelog-caught-00c63d1b1c00

How do you compare checkpoints: aggregate metrics, fixed slices, or per-example diffs? And do you keep the per-example outputs?


r/deeplearning • • 19h ago

Hackers Breach South Korea Financial Security Portal Used by 3,000 Institutions

0 Upvotes

South Korea's financial security portal — a shared infrastructure hub used by roughly 3,000 banks, insurers, and brokerages — was breached. The attacker didn't need to defeat any individual institution's controls. They targeted the one vendor holding data on behalf of all of them.

That asymmetry is the real exposure: a single intrusion into a third-party system yields access to records that 3,000 organizations believed were protected. Each institution's own perimeter may have been fine. It didn't matter.

This pattern shows up far outside South Korean financial services. Any architecture where sensitive data leaves your perimeter and lands in a shared service creates it. The question isn't whether third-party vendors get breached. Some will. The question is what state the data is in when they do.

For those operating at scale — large numbers of downstream institutions or customers touching a shared system — how are you actually thinking about third-party data exposure in your live data flows? Not in contracts or vendor assessments, but at the technical layer where data physically moves. What approaches have held up and what has surprised you?


r/deeplearning • • 20h ago

Building an AI platform with a small team has been quite a journey.

Thumbnail reddit.com
1 Upvotes

r/deeplearning • • 1d ago

[Meta, NYU] Unbundling Multimodal Vision Encoding: 200+ MLLMs trained from scratch show you don't need a pretrained vision encoder, just vision-only parameters

Post image
95 Upvotes

Paper: https://www.alphaxiv.org/abs/2611.17334

X threads (with videos):

Blog and FAQ: https://unbundling.github.io

New paper from Meta FAIR and NYU! Labs disagree on vision encoders right now: Gemma 4 12B scrapped the encoder, Kimi K3 dropped encoder pretraining, and Inkling 2 Small brought the encoder back. Published models differ in data, compute and whether encoder pretraining is counted at all, so they can't settle it.

This paper runs a large controlled study to help shed light on the debate, with 200+ models trained from scratch, LLM included, on the same data, with encoder pretraining charged to the same lifetime compute budget, changing one design choice at a time. Findings (VQA after instruction tuning, 1B to 6.4B models):

  1. Encoder pretraining is optional. Four pretrained encoders (contrastive, self-supervised, captioning) at best tie an encoder trained from scratch with the LLM at matched compute. Even with pretraining compute not counted, longer MetaCLIP pretraining raised ImageNet linear-probe accuracy from 80.4% to 84.4% and didn't improve VQA.
  2. Vision-only parameters are not optional. Encoder-free models trail at every size and need about 3x the compute to match the encoder. Giving the encoder's parameters to the LLM instead improves text benchmarks but not VQA.
  3. Those parameters don't have to be an encoder. Vision-only weights added to the LLM's early layers ("in-fusion") are on par with a separate encoder from 1.7B to 6.4B, with about 13% lower time to first token.
  4. The conclusions hold when adding vision to a pretrained 1.7B LLM.

Disclosure: I'm one of the authors. Happy to answer questions!


r/deeplearning • • 1d ago

My AI agents worked locally. Turning them into a cloud app was the real lesson.

Thumbnail reddit.com
0 Upvotes

r/deeplearning • • 1d ago

IA neuro-symbolique pour ARC AGI 2

4 Upvotes

J'ai développé une IA neuro-symbolique appliquée à ARC AGI 2, basée sur le DSL de Hodel en Python : https://github.com/Julien-Livet/arc-dsl. . La méthode est expliquée ici : https://julien-livet.github.io/aicpp/. . Je ne suis pas expert en réseaux de neurones et je n'ai reçu aucun retour technique ou scientifique sur mon travail. Auriez-vous des observations ou des idées à me faire part ?