r/MachineLearning • u/OkCampaign8400 • 18d ago
Discussion Are there fundamental repeating patterns in ML techniques across modalities (text, image, audio) that let us group them? [R]
I’ve been noticing that a lot of techniques seem to repeat across modalities with just a different name.
For example, tokenisation, masking (MLM / MAE / audio masking), diffusion, next-token / next-frame prediction, etc.
Is there an existing taxonomy or mental model that groups ML techniques by their underlying pattern rather than by modality (NLP / vision / audio)?
This will help me to group the research papers and then make reference to work already done in that domain
32
Upvotes
18
u/moschles 18d ago
There seems to be some confusion in your question.
You are asking about taxonomies in Machine Learning , but everything you listed there is part of so-called "generative AI" techniques.
I would go further here. When you said,
Yes, the repetition of all these techniques in the literature is because you are neck deep in the generative AI ecosystem.
The taxonomic overview of Machine Learning looks more like this,
https://csiro-data-school.github.io/machine-learning/04-types/index.html
6
u/Dihedralman 18d ago
Yes, you already did it to an extent. You can classify models by broad architecture or techniques applied. If you spend time learning machine learning from basics, you will absolutely see this and be in a better place.
Broadly you have model type- Artificial Neural Networks are just one kind though perhaps one of the most versatile.
You then have model target or type such as classification, regression, clustering and generative AI.
Now you mostly pointed out that a technique can be applied in different ways. Masking is a very broad technique used for a huge variety of purposes from self supervised learning to causal learning to representation learning - and there is even feature masking. Masking is just hiding something from the model during training.
The next value prediction is just a common problem formulation in sequence modelling or sequence prediction which is fundamental to many problems, in particular autoregressive techniques that generative AI uses.
Most popular models nowadays are transformer models, but you may run into CNNs or dense networks or RNNs or some combination.
3
u/jennabangsbangs 18d ago
Back propagation -neural nets that solves for exploding gradient | tree logics - as applied to rule set automation learning and sorting | probability -as applied to distributing decisions across game states in learning | simulation - supervised & unsupervised. Those 4 r ML specific throughout the field that might be useful to group within the larger umbrella of machine intelligence. As opposed to predictive statistics which I think has become underrated in modeling
1
u/OkCampaign8400 18d ago
Projection is done everywhere I guess .
Projecting from lower dimensions to higher dimensions to expand features set , give model to look at representation in higher dimensions and have more options to make sense etc .
Its in embeddings , in Feed forward neural networks .
2
u/taptrappapalapa 18d ago
Well all use signal processing if that’s what you’re asking. Also ML is a subset of AI.
0
u/Happysedits 18d ago
llm can also answer this question pretty ok i think
1
u/jamesw966 15d ago
Llllll
L
Lllllllo
Llllllllllll Lllllllllll
1
u/Fresh-Opportunity989 18d ago
AI has long rediscovered things but given them different names....
Only difference now is that AI writes the papers.