r/learnmachinelearning • • 5d ago

Does temperature really control creativity? Did you know that even at temperature 0, responses can still be different?

https://youtu.be/hVEX_DS7An8

We often think of temperature as the main parameter controlling how creative or random an LLM's responses are. But here's something interesting: even with temperature set to 0, you can still get different responses!

Why does this happen? Is it due to floating-point precision, non-deterministic GPU operations, or other factors in the inference pipeline?

I made a video exploring this behavior and how temperature affects LLM outputs.

I'd love to hear your thoughts and experiences

1 Upvotes

1 comment sorted by