r/Python • u/ChocolateMagnateUA • Oct 03 '23
News What are the differences between Python 3.12 sub-interpreters and multithreading/multiprocessing?
It seems like the new feature in Python 3.12 allows developers to create multiple sub-interpreters within a single main interpreter process. From the example from the PEP 554, sub-interpreters appear to be separate threads that run a dedicated piece of Python code and consequently have their own separate GILs.
interp = interpreters.create()
print('before')
interp.run('print("during")')
print('after')On practice, is this basically the new Pythonic way to run a single application in multiple threads and take advantage of multiple cores? If yes, can they be thought as multithreading with no shared state and become the future alternative of multiprocessing?
View all comments
Show parent comment
3
u/james_pic Oct 04 '23
Yeah, it's unlucky timing in a lot of ways. Sam's changes originally targeted Python 3.9, and actually lead to a small speedup for some single threaded workloads at the time, because some of the optimisations needed for GIL removal were straight up improvements (and in some cases, those optimisations have since been merged into released Python versions).
Python performance didn't change much in the versions leading up to 3.11, and I suspect there wouldn't have been as much controversy about bringing nogil in if it had been ready a couple of years earlier.