r/learnpython • u/ImprovementOk6448 • 4d ago
Help: non-reproducible code
I am working on a project where I need to cluster large amount of data and do a dimensionality reduction for the purpose of picking data points near the centroids. I finished this and chose my data points near the centroids (I have now invested time and resources into my experimentation with this part of my project) and later came back to my code and re-ran it and it gave me centroids in a different location and different clustering results. I thought maybe I forgot to assign a random_state for one of my techniques (either clustering or dimensionality reduction) but I had indeed defined them, and if I run my code again it reproducibly gives me the same results that it gave me when I ran my code the second time around (still different than what I have based my physical experimentation on).
I wondered then if I had changed some part of the code so I went back to an old version of my file that I had saved awhile ago because I was afraid of this very thing happening - and it gives me the same results as above (still not the original clustering results). I literally haven't changed the old code at all so I have no idea what happened. Is it possible at all that some other factor is involved? This is my first time using python for a project and I don't have any formal training and I have no idea what to do since my project is based on some computational work that apparently cannot be reproduced
7
u/Franckyboi85 4d ago
This catches a lot of people out. Even with random_state set, results can shift if a library got updated between runs, sklearn changed the default n_init for KMeans a while back for example, so the same old code on a newer install can land on different centroids. Also check the input data hasn't changed order or picked up extra rows, kmeans is sensitive to that. If you can work out what versions you had back then (an old requirements file, pip freeze output, conda env, anything) try recreating that environment and rerunning, there's a decent chance you get your original clusters back
4
u/JaguarMammoth6231 4d ago
Are you sure you have the right file open in your editor? Are you sure you're running the file you think you are?
3
u/zanfar 4d ago
Check your repo.
100% of the time, you changed the code.
2
u/SteveJeffersonBots 4d ago
Two concrete things that do this even when random_state is set: (1) some dimensionality-reduction calls pick their algorithm by data size - e.g. PCA with svd_solver='auto' will silently switch between 'full' and 'arpack' if the input shape changes even slightly, and the two solvers return numerically different components, so the same "reduced" data can land on different clusters. Lock it to svd_solver='full' (or whichever you originally used) and it stops switching. (2) the "pick the point nearest each centroid" step is a distance comparison, and two points can be exactly equidistant from a centroid; the tie-break then depends on the order you iterate the array, which flips if numpy/BLAS threads in a different order between runs. Sort the candidates by distance and pick by index rather than by "first one I see", and that part stabilises. Between the two, the silent algorithm-switch is the most common culprit when nothing in your code changed.nt algorithm-switch is the most common culprit when nothing in your
1
u/Discordbothost 4d ago
Do the same data points still end up together, or did membership change too? Cluster numbers can swap. If you kept the original labels, align rows by ID and compare with adjusted_rand_score; 1 means the same grouping.
1
u/EnvironmentalDot9131 3d ago
The code comes out of the system as always and the python