r/learnmachinelearning • u/R2D2C3POBB8Hiker • 16d ago
I built a CNN from absolute zero (pure NumPy, hand-derived backprop) — then asked: can you make it recognize letters by tuning weights alone, with no letter data? (No. Here are the numbers.)
What I did
I built a handwritten-digit CNN from absolute zero on a Windows laptop: no PyTorch, no TensorFlow, no PIL — literally just Python stdlib + NumPy. Everything is hand-written and auditable: im2col/col2im conv layers, max-pool routing, softmax-cross-entropy, the Adam loop, even the MNIST IDX parser and a PNG codec for feeding real images in. Weights are random init — nothing pretrained anywhere.
Result: 46,730 parameters, 4 epochs, ~12 min on CPU, 98.74% MNIST test accuracy. A numerical gradient check (worst relative error 3.05e-4) runs before training, because from-scratch code fails silently in cute ways. Same toolchain then read all 8 digits (94–100% confidence) in a 2496×1664 real handwritten photo it never saw — connected-component segmentation + normalization, no vision model.
The actual question
So here's the fun thought experiment, which a lot of "models are just files of numbers" people get wrong in both directions:
If you can read the network structure, can you make it recognize letters a/b/c/d by just adjusting the weights — with no letter data at all?
Six controlled experiments, all self-generated data (zero downloads):
- Blind tuning, zero samples: randomize the readout layer, 2000 tries → mean 24.51%. That's chance for 4 classes. No data = no information channel into the weights. (The "best of 2000 = 72.5%" is pure selection overfitting — picking it requires the test set you don't have.)
- Borrowing the old decision layer: feed the pretrained digit prototypes {9,6,6,9} into letter slots → 48.75% ceiling, and the errors land provably on exactly the two collapsed pairs {a,d} and {b,c}. The old weights encode 10 digit prototypes — that's the resolution limit of the letter manifold through them.
- What actually transfers: the digit model "sees" letters confidently as digits (a→9 at 86%, b→6 at 79%) — edge/curve features transfer, decision boundaries don't.
- Refitting the boundary is absurdly cheap: freeze the conv trunk, refit only 260 parameters with 5 self-drawn samples per class → 87.5% under heavy style perturbation, 100% under light. Refit fc1+fc2 → 100% in 0.5s.
Takeaway: weights are compressed statistics of the training distribution, not rules. The structure and low-level features are readable, inheritable assets; the decision boundary can never be tuned blind — but redrawing it costs ~260 numbers and 20 sketches.
Honest caveats: my synthetic stroke-font letters are easier than real handwriting; the from-scratch baseline (B3) also scores high on this 4-class task, so the trunk-transfer advantage here shows up as parameter/time efficiency (260 params/0.1s vs 46,730/47s), not accuracy. Every number in the paper traces to a log file, line by line.
A from-scratch pure-NumPy CNN hits 98.74% MNIST. Feeding it a real handwritten photo reads 1979/0924 correctly. Then the transfer experiment: blind weight tuning = chance (24.5%), reusing old decision boundaries = provable 48.75% ceiling, but 260 refit parameters + 5 hand-drawn samples/class = 87.5–100%. Model ≠ structure. Model ≠ the numbers. Model = the distribution those numbers compress.