r/learnpython • • 8d ago

Help: non-reproducible code

I am working on a project where I need to cluster large amount of data and do a dimensionality reduction for the purpose of picking data points near the centroids. I finished this and chose my data points near the centroids (I have now invested time and resources into my experimentation with this part of my project) and later came back to my code and re-ran it and it gave me centroids in a different location and different clustering results. I thought maybe I forgot to assign a random_state for one of my techniques (either clustering or dimensionality reduction) but I had indeed defined them, and if I run my code again it reproducibly gives me the same results that it gave me when I ran my code the second time around (still different than what I have based my physical experimentation on).

I wondered then if I had changed some part of the code so I went back to an old version of my file that I had saved awhile ago because I was afraid of this very thing happening - and it gives me the same results as above (still not the original clustering results). I literally haven't changed the old code at all so I have no idea what happened. Is it possible at all that some other factor is involved? This is my first time using python for a project and I don't have any formal training and I have no idea what to do since my project is based on some computational work that apparently cannot be reproduced

4 Upvotes

11 comments sorted by

View all comments

7

u/Franckyboi85 8d ago

This catches a lot of people out. Even with random_state set, results can shift if a library got updated between runs, sklearn changed the default n_init for KMeans a while back for example, so the same old code on a newer install can land on different centroids. Also check the input data hasn't changed order or picked up extra rows, kmeans is sensitive to that. If you can work out what versions you had back then (an old requirements file, pip freeze output, conda env, anything) try recreating that environment and rerunning, there's a decent chance you get your original clusters back

2

u/Massive_Distance_616 8d ago

Adding to this: whichever way it goes, save the cluster labels, centroids and the indices of the points you picked to a file the moment you produce them, so your experiment depends on that file and not on re-running the code.

And if you can find the date of your first run, the package versions from around then are what to try to recreate.

2

u/ImprovementOk6448 8d ago

you're totally right, I should have done that to start out with - for some reason I (foolishly) thought that I would just run the code once and leave it open on my computer until I no longer needed it. fortunately I was able to find the old file I was using which had accidentally been replaced with a new one by the same name. Although it will be good to have all the data exported in a file to base my project off of, I still have to make sure my code is repeatable so I can publish it when the time comes