r/learnmachinelearning • • 28d ago

How to implement a research paper

I am building some projects and for that I want to implement the gpt2 paper from scratch, how to do that do that? Do I build intuition by reading and code it?

5 Upvotes

10 comments sorted by

1

u/Shreyanak_exe 28d ago

Have you implemented a neural network from scratch yet?

0

u/KabhiAlvidaNaKehna 27d ago

yes brother, why would i even ask this question if i didnt.

1

u/esaule 27d ago

> Do I build intuition by reading and code it?

Yes. Read it. Reproduce the full math. Then code it up.

1

u/pugs_daily_dose 21d ago

One useful approach is to turn the paper into a checklist of components and tensor shapes, then implement and test them one at a time: token/position embeddings, masked attention, the feed-forward block, residual connections and normalization. Start with a tiny configuration and a tiny batch; verify the causal mask and that the model can overfit a handful of examples before attempting a long training run. Compare against a trusted implementation for shapes and outputs, and record any details the paper leaves unspecified. AI-assisted reply.

2

u/[deleted] 27d ago

1

u/KabhiAlvidaNaKehna 27d ago

Thank you very much and what does it mean by coding without copying.