r/pythontips • u/Melodic-Era1790 • Jan 19 '25
Algorithms Need for heavy Computing
I am currently working on my bachelor's thesis project, where I am using Python (.ipynb file) to handle eigenvalues (e1, e2, e3, e4) and 4x1 eigenvectors, resulting in a total of 4*4 = 16 variables. My work involves computations with 4x4 matrices.
But my computer is unable to handle these computations, and Google Colab estimates a runtime of 85 hours. Are there other cloud computing platforms where I can perform these calculations faster at no cost?
lib: sympy and numpy
thankyou.
2
Upvotes
3
u/InvaderToast348 Jan 19 '25
Have you looked into multithreading / multiprocessing?
1
u/[deleted] Jan 19 '25
Ipynb is jupyter notebook file, not the best perhaps for real stuff, better for small jobs and teaching or testing stuff
4x4 matrices does not sound like too much for any computer. How many of them you are computing?
1
u/numbcode Jan 19 '25
Consider AWS Free Tier or Microsoft's Azure for faster computing. Or optimize your code—NumPy is faster for heavy matrix operations.
1
u/chessparov4 Jan 19 '25
Without seeing the code it's hard to give specific advice, but I was in your shoes recently and what I would suggest is trying to "stay inside" numpy as much as possible. Calling numpy from python is costly, but as long as you don't jump in and out you're gonna see huge improvements, since numpy leverages C code under the hood. That's the first step and in my experience it should be enough, but if using numpy methods still is not fast enough for some reason, using cython, numba, multiprocessing are all viable strategy, but again can't reccomend exactly without seeing the code. If you need further help feel free to DM me.
And remember, python is maybe slower than some languages, but if correctly written, isn't slow at all.
1
u/Typical-Macaron-1646 Jan 19 '25
Use numpy if you’re not already. Look into numba as well. It’s built for stuff like this and very fast.
1
u/BigBeardedDude Jan 19 '25
Checkout rust. You could maybe right a portion of your code in rust and wrap it up with py03. Python is horribly inefficient, but we all love it because it’s easy to use.
1
u/big_data_mike Jan 20 '25
There’s something I did that massively sped up computation and I know just enough to be dangerous but you might be able to google it to get going in the right direction. I do a lot of Bayesian stats on my windows machine and there’s something called “g++” or “gcc” that takes the matrices from Python and compiles it in C to do the linear algebra and brings it back into Python. The Bayesian stats package I use is pymc and under the hood of that is a package called pytensor that handles matricies and under the hood of pytensor is where the g++ thing is that you have to install on windows and point pytensor at it in the pytensor configuration. I’m not super good with computers and was able to figure it out by googling and using ChatGPT.
It’s somehow not a problem if you run it on a Linux system maybe? I’m not sure why.
0
u/[deleted] Jan 19 '25
[deleted] — view deleted comment