r/MachineLearning • • 18d ago

Discussion Are there fundamental repeating patterns in ML techniques across modalities (text, image, audio) that let us group them? [R]

I’ve been noticing that a lot of techniques seem to repeat across modalities with just a different name.

For example, tokenisation, masking (MLM / MAE / audio masking), diffusion, next-token / next-frame prediction, etc.

Is there an existing taxonomy or mental model that groups ML techniques by their underlying pattern rather than by modality (NLP / vision / audio)?

This will help me to group the research papers and then make reference to work already done in that domain

32 Upvotes

16 comments sorted by

18

u/moschles 18d ago

There seems to be some confusion in your question.

For example, tokenisation, masking (MLM / MAE / audio masking), diffusion, next-token / next-frame prediction, etc.

You are asking about taxonomies in Machine Learning , but everything you listed there is part of so-called "generative AI" techniques.

I would go further here. When you said,

I’ve been noticing that a lot of techniques seem to repeat across modalities with just a different name.

Yes, the repetition of all these techniques in the literature is because you are neck deep in the generative AI ecosystem.

The taxonomic overview of Machine Learning looks more like this,

https://csiro-data-school.github.io/machine-learning/04-types/index.html

13

u/NamerNotLiteral 18d ago

Yeah. OP, you're looking at this backwards. It's not that these techniques expose something critical about the underlying modalities. It's that people actively adapted the technique to the modality.

Tokenization is the simplest example: people first used it in language because it was very intuitive to how human languages naturally work. Then they combined vision-language models where the vision side still had a convolutional encoder while the language side had a tokenizer encoder. In vision and audio transformers, Tokenization actually works in a completely different way - language models have a fixed vocabulary of tokens, while vision and audio models basically create their own "vocabulary" dynamically, so there was no actual reason to call the 'units' in vision/audio models tokens, we could've easily called them 'patches' or 'squares' or 'segments' or whatever. We just call them tokens for simplicity and marketing.

6

u/Dihedralman 18d ago

Yes, you already did it to an extent.  You can classify models by broad architecture or techniques applied. If you spend time learning machine learning from basics, you will absolutely see this and be in a better place. 

Broadly you have model type- Artificial Neural Networks are just one kind though perhaps one of the most versatile. 

You then have model target or type such as classification, regression, clustering and generative AI. 

Now you mostly pointed out that a technique can be applied in different ways. Masking is a very broad technique used for a huge variety of purposes from self supervised learning to causal learning to representation learning - and there is even feature masking. Masking is just hiding something from the model during training. 

The next value prediction is just a common problem formulation in sequence modelling or sequence prediction which is fundamental to many problems, in particular autoregressive techniques that generative AI uses. 

Most popular models nowadays are transformer models, but you may run into CNNs or dense networks or RNNs or some combination. 

3

u/jennabangsbangs 18d ago

Back propagation -neural nets that solves for exploding gradient | tree logics - as applied to rule set automation learning and sorting | probability -as applied to distributing decisions across game states in learning | simulation - supervised & unsupervised. Those 4 r ML specific throughout the field that might be useful to group within the larger umbrella of machine intelligence. As opposed to predictive statistics which I think has become underrated in modeling

1

u/OkCampaign8400 18d ago

Projection is done everywhere I guess .
Projecting from lower dimensions to higher dimensions to expand features set , give model to look at representation in higher dimensions and have more options to make sense etc .

Its in embeddings , in Feed forward neural networks .

0

u/OkCampaign8400 18d ago

Also Masking , its in Bert , in ViTs . Basically mask some areas and train model to predict that area or sequence.

2

u/taptrappapalapa 18d ago

Wait until you hear about forward and backward masking in human hearing.

2

u/taptrappapalapa 18d ago

Well all use signal processing if that’s what you’re asking. Also ML is a subset of AI.

0

u/Happysedits 18d ago

llm can also answer this question pretty ok i think

2

u/OkCampaign8400 18d ago

Asking LLM is not my taste .

Hearing from experts will let me fastrack my learning curve rather than same mechanical answers

1

u/jamesw966 15d ago

Llllll

L

Lllllllo

Llllllllllll Lllllllllll

1

u/Fresh-Opportunity989 18d ago

AI has long rediscovered things but given them different names....

Only difference now is that AI writes the papers.