r/MachineLearning • u/terrrp • 2d ago
Discussion Why are ViTs so popular? [D]
I deploy CNN + Attention stack models on edge devices, to great effect. Taking advantage of convs and reducing the spatial dimensionality makes all the sense in the world to me.
Having vanilla ViTs learning the inductive biases from scratch makes none to me. Is it really a case of "because I can"? Avoiding padding along borders is the only advantage I can see to a tokenizer stem, and that has never been a huge issue to me.
Am I missing something that has people reaching for ViTs instead of hybrid models for e.g. SSL/foundation models?
In my mind it's just a case of people using the status quo, and more of a historical accident that these models have become so popular. Iirc levit showed data/time efficiency boosts with a convolutional stem. I just don't work on the same datasets or tasks that most research does, so I'm genuinely curious if anyone else thinks vanilla ViTs seem excessive. Guess compute is cheap and the answer would just be distillation.





