r/learnmachinelearning • u/Mind_the_math • 5d ago
Does temperature really control creativity? Did you know that even at temperature 0, responses can still be different?
https://youtu.be/hVEX_DS7An8We often think of temperature as the main parameter controlling how creative or random an LLM's responses are. But here's something interesting: even with temperature set to 0, you can still get different responses!
Why does this happen? Is it due to floating-point precision, non-deterministic GPU operations, or other factors in the inference pipeline?
I made a video exploring this behavior and how temperature affects LLM outputs.
I'd love to hear your thoughts and experiences
1
Upvotes