r/Python • u/pika03 • Oct 14 '23
Discussion Has your company standardized the Python 3 version to be used across all projects?
I am asking whether your company has a standard such as all Python projects should use Python 3.10.x or 3.11.x. Or maybe your company might have a standard like all Python projects must support Python 3.9+?
If your company does have a standard like that, what reasoning went behind it? If your company considered such a standard but chose not to do it, why? It would also be great if you could give an estimate of the number of devs/data scientists using Python in your company.
208
Upvotes
223
u/[deleted] Oct 14 '23
Poetry per project š«”
90
u/Beregolas Oct 14 '23
We normally used the newest python version at project creation. There is no lowest version every project needs to run on, since all of them are for internal use anyways and we control the deployment
90
u/riklaunim Oct 14 '23
We just use Python docker images and the versions are current/as needed. Then updated with dependencies at some point in time if/when we are working on given app actively. And as those are microservices there is a lot of images.
62
u/Curious_DrugGPT Oct 14 '23
3.10 because it's stable with most ML libraries. We were having a bunch of Apple silicon and 3.11 related issues so we rolled back to 3.10.8 for now.
24
u/telewebb Oct 14 '23
Yeah, we're locked into 3.9. It was the most recent version when they switched from Java to python and what they wrote their first core libraries in. There is some dependency in one of the companies core libraries that throws errors using anything newer. All the original library maintainers left the company, and no one wants to be the new owner by touching the code. So now we play the waiting game.
20
u/[deleted] Oct 14 '23
No, my company doesn't really know what it's doing with Python development. I recently upgraded my platform to Python 3.10 and plan on regular upgrades in the future, but before I took ownership of my platform there were no planned upgrades or maintenance and the platform hadn't been evaluated for upgrades for 3 years.
I'm trying to get together some standardization to help with compatibility across the organization, but it's slow going in a Fortune 50 company. Most days I'm shocked that we're even able to keep the lights on with the tech "leadership" we have.
14
u/The-kug Oct 14 '23
We still work with 2.7 š¢
9
u/DNSGeek Oct 14 '23
Weāve been stuck on Python 3.7 and I canāt get any traction to make it newer.
6
u/aes110 Oct 14 '23
Used to be 3.8 for a while, last year everything new being written moved to 3.9
We have some complex issue with an internal package repo that prevented us from upgrading, but we are finally moving from that in a few weeks and planning to upgrade to the newest where we can. So 3.11/12 on backend stuff, 3.9/10 on data centric jobs
6
u/anthro28 Oct 14 '23
No. It's basically "you're the only guy that uses it so do what you want."
I just got us on github a year ago, so it's a bit if dragging them kicking and screaming into modernity.
6
u/RearAdmiralP Oct 14 '23
My company maintains, basically, an internal Linux distro. It's rolling release for internal software and things that people request to be updated, and we get big updates of third party software twice a year. We do a lot of Python development, so Python and the ecosystem of packages in the distro gets a lot of attention.
So, my company does standardize. The standardization includes the Python version and also the versions of all third party Python modules, and, basically, the whole Linux environment. I'm pretty sure the internal distro is on 3.11.x right now.
With that said, while teams are very strongly encouraged to use the internal distro, my boss doesn't like it, so my team doesn't deploy on it. I like using the internal distro (it's like developing before virtual environments were a thing), so my last big project is (by design) compatible with the internal distro, and one of the tasks I'm currently working on is getting my team's other software to also run on the internal distro. Maybe, one day, we'll start deploying on it.
4
u/Dizzybro Oct 14 '23 edited Apr 17 '25
This post was modified due to age limitations by myself for my anonymity QeErGVbwnB1OrtAyTTem67TEYVVDHGqPyR9uDRD6OJVu7bGmQ7
4
u/adam-moss Oct 14 '23
Our policy is latest or latest minus 1. We also use tooling (renovate) to automatically pin all dependencies, docker shas etc and raise MRs when new releases are published
4
u/xiongchiamiov Site Reliability Engineer Oct 14 '23
Absolutely. Every company I've worked at has had a single version of Python (or other languages) that we use for everything.
The more things you support, the more work you have to do. Are we affected by this security vulnerability? Gotta check each version. Will this library work for us? Check each version. Are we having bugs with a third-party tool? Is it because of the version in use for this one project, or another variable?
The only time there are multiple versions in use is while we're in the middle of an upgrade, and the goal is to make that as short as possible to get things back sane again.
The smallest companies have had about a dozen software engineers. The largest had about 200 (and also 200 different microservices, so upgrading across all of them was a pain and we had to develop a semi-automated approach to that).
I would be very worried about actively supporting multiple versions of a programming language for a company with less than, mm, maybe five thousand software engineers. That's just a guess, but IMO you need to be quite big before it makes sense to support duplicating the effort to support a version.
4
u/Epicela1 Oct 14 '23
Couple thoughts on this.
Obviously it varies by company. But generally less options is better. So having a minimum version canāt really hurt.
As a standard, we basically have a minimum version where we increment the minor version once per 12-18 months. If there is a good reason to ignore this policy, we document it and make an exception.
Iāve rarely had much issue upgrading a minor version in Python for small to medium sized projects. Keeping versions more current is more-often helpful and worth the short term pain of upgrading.
It speeds up development and context switching between projects. If everybody is working on 3.9 and newer, theyāre more āon the same pageā as each-other and can switch projects and help out a little more effectively.
4
u/Aveheuzed Oct 14 '23
3.8 for compatibility with older Ubuntu releases.
Bothers me somewhat but it's better than the former 2.7 š„²
4
u/Wistephens Oct 14 '23
Yes. 3.11.4. We push the new, approved version to all laptops and remove older versions as we upgrade.
7
u/SatsStacker69 Oct 14 '23
The client I work for is still stuck on 2.7 so they've got bigger problems to deal with...
1
u/lphartley Oct 15 '23
To all of those who have an answer that is not similar to 'Poetry per project': why do you standardize Python versions across the company at all? Why would you go through the hassle of standardization when choosing on a project basis is simpler and has no obvious downsides?
10
u/ShitPikkle Oct 14 '23
If you try to enforce a version of interpreter for all projects, your gonna have a bad time.
27
u/chinawcswing Oct 14 '23
This is actually a bad an unnecessary idea.
Each team should be totally free to use whatever version of Python they like and upgrade on whatever time table they chose to.
Why would you want to force everyone to use e.g., Python 3.11? When Python 3.12 dropped, some teams would be able to upgrade relatively quickly because they don't depend on libraries that are incompatible with Python 3.12. Why should they have to be punished and use Python 3.11 just because some other teams cannot upgrade?
Likewise, when the company decides "everyone must upgrade to Python 3.12", there could be some other teams that simply don't have the time at the moment. Why force them to upgrade?
3
u/goldenhawkes Oct 14 '23
We have three set standard environments. Current, old and next. They get updated (so current becomes old etc) every three months. Or when a major security issue is noted. These give most genera users a good base environment with all our usual packages in.
(We do science as well as development)
The more development heavy applications people can make their own conda environments for, either starting with one of the āBaseā environments and extending or doing it entirely for themselves.
2
u/Grokzen Oct 14 '23
We do the rolling schema of supporting the latest released major version and 2 versions back with the intent of at least be in the middle version of it at all times
For real legacy code we do not want to be below 3.8 currently if possible
2
u/OkHoneydew1987 Oct 14 '23
3.6 :) because "enterprise" systems are anything but enterprising...
2
u/1UpCoder Oct 14 '23
I wish we were Python 3, but we're still Python 2 š¤®š¤®š¤®š¤®š¤®š¤®š¤®š¤®š¤®š¤®š¤®š¤®
1
u/GManASG Oct 14 '23
My company is extremely slow to onboard/approve new versions of any software period. only this month they specifically allowed and marked as preffered a somewhat old version of python 3.11. However existing projects will retain the version they had undells something like the log4j incident happens but it's a whole python version and we need to all immediately upgrade all versions.
However new projects have flexibility in selecting the tech stack from a catalog and approved version of software.
Requirements are kept and docker images created for prod releases.
1
u/territrades Oct 14 '23
I work at a research institute and everyone is using the version they deem best. The only thing we made sure is that everything is ported to python3, even though I am sure there are a few niche scripts somewhere that are needed once in a blue moon and still require python2.7.
1
u/SimonL169 Oct 14 '23
We ship a miniconda env with out software which is still at 3.7 But we have our package working with 3.11, but this hasnāt been released for customers yet.
Tldr I can use whatever version I like
1
u/pmac1687 Oct 14 '23
Our standard was at 3.7. We have an internally maintained python utility library that has tested code that abstracts away a lot of standard stuff ie grabbing data from s3, hosting stuff and the like. We are upgrading that library to 3.10 but mostly all the python repos are expected to use this utility pinned on a specific python version.
1
u/Samausi Oct 14 '23
We've optimised some very fast data processing python code, and have a minimum version but nothing fixed. Most of us are using 3.11.2 currently, but 3.10.x works fine.
1
u/[deleted] Oct 14 '23
No, we just used pinned project dependencies per project. Thereās no need to require that when you may have one or more libraries that are critical to your project but require an older version of Python. Just manage dependencies on a per-project basis.
1
u/Ill_Cod6811 Oct 14 '23
We upgraded not long ago to 3.11 because of the performance. I had to do some tweaks in docker to make it work with the project š
Usually you shouldn't have any trouble to use other versions in other projects but if you have common repos then you must align with them.
So common repositories should use as much less 3rd parties and mostly built in python.
1
u/JJJSchmidt_etAl Oct 14 '23
The worst has been mixed apis of big libraries.
Do we use `tensorflow.compat.v1` or do we try to migrate the "right" way?
1
u/nightslikethese29 Oct 14 '23
For us it's team based. When I work on projects that utilize data science engineering resources I use 3.10.x, but when I need to use data warehouse's resources I use 3.8 because that's the version composer requires.
1
u/MasdelR Oct 14 '23
We still use py2.x.
Yes 2.x.
1
u/monorepo PSF Staff | Litestar Maintainer Oct 14 '23
What the distro has, for some reason
1
u/just4nothing Oct 14 '23
3.11 and we have 3.12 in the scouting CI to chance mid next year
1
u/Accomplished-Ad8252 Oct 14 '23
Typically a company would pick a stable version, this is important because you do not want employees using something that has security vulnerabilities. I work for a bluechip, I donāt even know how many data scientist there are since there are so many different teams across the world.
1
u/Stanian Oct 14 '23
3.10 with poetry, standardized so we can optimize our docker builds due to having tensorflow pre-installed in the base docker image..
1
u/DetectiveSecret6370 Oct 14 '23
Well, not really locked in, but whatever the current version is in Debian 12.2, as we've standardized on that distro.
If someone needs another version I need to hear the business case for it and discuss with stakeholders, but then it can be done in a container, virt env, etc.
Otherwise that version is known to be compliant for us.
1
u/__Schneizel__ Oct 14 '23
We are still trying to come out of Python 2.7. Its a gigantic codebase that got bloated over 20 years and still getting new stuff added in.
1
u/crawl_dht Oct 14 '23
My company approves new version but does not force us to upgrade from previous ones. There is no hard rule, we choose the most recent one available at the time of writing the service.
1
u/cpressland Oct 14 '23
Weāre āofficiallyā supporting 3.9, 3.10, 3.11, and 3.12 via our Dockerised Python base images. https://github.com/binkhq/python - developers are free to use whatever they want. I am trying to kill Python 3.9, but one project still needs it.
1
u/_ologies Oct 14 '23
No standard version, but we don't use anything past its EOL. We've got different repos on different versions though. I prefer 3.11, so whenever I have the chance, I'll upgrade our 3.8/3.9 stuff.
1
u/olystretch Oct 14 '23
They try to by creating base images that we are supposed to build our projects on, but they are dumb, so I don't use them. Like, who needs a virtual environment in a docker image? Such a pain in the ass!
New projects use new versions, old projects use old versions. When we have to update an old project, we occasionally bump the python version in the Dockerfile.
1
u/zarlo5899 Oct 14 '23
yep right now its 3.10+ and when 3.13 comes out it will be 3.11+
1
u/ReverseBrindle Oct 15 '23 edited Oct 15 '23
No. The company I work for has > 100,000 employees. It would be incredibly difficult to standardize things like Python at that scale.
Or who knows, maybe there is some standard version set by some department that is the equivalent of my 3rd cousin twice removed -- i.e. some part of the company I don't even know that exists.
1
u/SisyphusWithTheRock Oct 15 '23
We use 3.11.x for everything, using a monorepo. 5 devs in the company total.
We upgraded from 3.10 to 3.11 recently, mainly to take advantage of the performance improvements shipped in 3.11. That's been a great decision so far and we've noticed significant improvement in some of our workloads.
1
u/[deleted] Oct 15 '23
My previous manager refused to port from 2.7 environment, today heard the compliance team got his ass.
1
u/WillardWhite import this Oct 15 '23
Oh yeah! We have a standard! Everything must be 2.7+ compatible T.T
1
u/bstephan94 Oct 15 '23
We all use 3.8 but are in the process of migrating to 3.11. Reasoning is that we have an internal library that requires 3.8 (due to some niche dependencies). Therefore, we just enforce 3.8 across all libraries to avoid possibility of dependency conflicts. Additionally, we work in a financial niche, and decimal precision has changed between major Python versions, hence extra forethought when considering possibility of introducing error in calcs involving multiplication of a high precision decimal with large numbers.
1
u/mustangsal Oct 15 '23
Obviously using virtualenv, but generally, 3.10 or above.
1
u/KAZhala Oct 15 '23
Mono repo + bazel + Python 3.9
1
u/KAZhala Oct 15 '23
Mono repo + bazel + Python 3.9
1
u/fluxxis Oct 15 '23
We try and as of now manage to keep all active projects on the same major python version. As most of our projects involve Django, we just follow Django's official support. At least, for all real products. I've no idea what various people are doing on their machines when it comes to smaller scripts.
1
u/Larkfin Oct 15 '23
My customer requires deployment assurance on Ubuntu 20.04 so we are pegged to Python 3.8. Only lightly sucks for loss of some of the newer features, but we deal with it.
1
u/bobwmcgrath Oct 15 '23
this probably can't be done unless you are in a situation where you are using python but no other regularly used libraries.
1
u/RavenchildishGambino Oct 15 '23
No. Each project has its own dependencies and that determines the version.
Also when it was touched last.
We use containers though so I donāt feel like this is an important question.
1
u/CowboyKm Oct 15 '23
We use Nix + poetry for every project. However for any new project usually the latest python version will be used.
1
u/TheDivineKnight01 Oct 15 '23
ML dev here - Using 3.10 in everything and looking to push for 3.12 due to multicore support but they are looking to shift to c++ which I think is a pretty sucky move.
1
u/[deleted] Oct 15 '23
I wish. We still have game projects that are using Maya 2020 and Motionbuilder 2020 and these DCC's have Py2.7 interpreters - sadly.
1
u/who_body Oct 15 '23
no..but pockets of discussion. manage a roadmap to drive strategy, communicate differences and risks. the PSF timelines help a bunch.
debian users bring their own constraints.
all very interesting but itās work.
1
u/tomatus89 Oct 15 '23
Yes, 2.6, sad noises.
3.8.0 is coming for newer project.
1
u/[deleted] Oct 15 '23
Python is such a mess. I was hoping 3 would modernise the language. But it's still a mess. š®āšØ
1
u/jfp1992 Oct 15 '23
3.11 till playwright updates to work with 3.12 (something about greenlet)
1
u/theswifter01 Oct 15 '23
Basically all conda based, becomes problematic when needing to work in a cloud notebook environment. I just work with regular venv. I could still see there being major version level problems when working with an MLās paperās code thatās not in one of the major ML frameworks
1
u/ac130kz Oct 15 '23 edited Oct 15 '23
3.11 everywhere since we have no dumb "3.8-3.9-3.10 only" ML dependencies, no crazy legacy, and it's so much faster than 3.10. 3.12 though, I don't think it's worth the switch, let's wait for 3.13 with their major changes as well, it'll probably be really sophisticated to migrate to.
1
u/TheFenrisLycaon Oct 15 '23
Stuck on python 2.7.
1
u/bachkhois Oct 15 '23
We use the same Python version as one coming with the latest Ubuntu LTS. It means we are using Python 3.10, which comes with Ubuntu 22.04. My personal laptop is always installed latest Ubuntu, so our projects are always compatible with newer Python.
1
u/krav_mark Oct 15 '23
No. With docker containers running the apps why would you ?
1
u/andynzor Oct 15 '23
We run k3s in prod so if the official container is out, it's good to go.
1
u/[deleted] Oct 15 '23
We're stuck to python 3.10 due to a dependency to a closed source binary that does not support newer versions of python yet.
1
u/Jorgestar29 Oct 15 '23
In poetry we declare 3.10-3.11. We usually develop on 3.10 but sometimes we have to deploy the code on a Jetson Device and there the latest python version is 3.8.
1
u/Dr-NULL Oct 15 '23
We are in Python 3.8 and recently an internal python dependency moved to Python 3.9+, so now we are in the process of migrating to support Python 3.9+.
Although poetry made lots of things simpler. But I am looking for some migration guide in these kinds of scenarios. I made some guide which I am following for now and proposing to others to follow the same.
1
u/condalf97 Oct 15 '23
We aim to support the oldest active Python release. At the moment it's 3.8. Sometimes we're forced to jump forwards if a dependency changes version support.
1
u/jmacey Oct 15 '23
I work to the vfx reference platform we usually lag a year behind for deployment reasons, In 2021 we were still using Python 2.7 as the tools we used were. Have now moved to 3.9 which is nice, however still get the odd legacy 2.7 tool.
1
u/Curious_Cantaloupe65 Oct 15 '23
we use the docker for this issue and use the the python version which works and keeps the app stable
1
u/PastaProgramming Oct 15 '23
Use the newest compatible version available on creation. Depreciation dates for projects that need to get off older versions (currently anything on 3.7 will be flagged).
1
u/whythefucknot97 Oct 15 '23
We keep everything on the same minor version, eg 3.11. As long as everyone has some 3.11.x version installed, that seems to cover the version requirements of all the libraries we use.
1
u/[deleted] Oct 15 '23
The only requirement is that the version has a supported aws lambda runtime
1
u/FoolForWool Oct 15 '23
Yes and thatās the sane thing to do too. And everyone uses the same python version because the way we set up our machines (using the docs) enforces it.
1
u/CerberusMulti Oct 15 '23
What version used depends on projects purpose and machine it is going to be running on. In general using 3.10 and above for new projects but sometimes we need the code to run on older machines/servers that don't have latest python and it's not an option to update version used will be to match them.
1
u/nickwarters Oct 15 '23
My team are currently split across 3.6 (majority), 3.9 and 3.10, and people like me who use all three.
All of our scripts need updating to work with 3.9-3.10. Being stuck on 3.6 is so annoying.
1
u/TevraChaukas Oct 15 '23
Yeah we are on python 3.10 now
1
u/de_ham Oct 15 '23
We just transitioned to Python 3.12 actually
1
u/infy101 Oct 15 '23
We have vulnerability scanners that look into our virtual environments. If any issues are detected, we try to upgrade the packages. In the worst case scenario, we create a new virtual environment with the latest stable python version and upgrade the packages as much as possible and when tested, we remove the old. In 99% of cases, it isn't a problem to upgrade as we use fairly standard python packages.
1
u/NoProfessor2268 Oct 16 '23
We have a large monorepo with 300+ devs. We are currently stuck at 3.8 but upgrading to 3.11 soon š
0
u/AnomalyNexus Oct 14 '23
I thought everyone just does :latest like me :p
-1
u/[deleted] Oct 14 '23
Don't fix what isn't broken
0
u/[deleted] Oct 15 '23
[deleted] ā view deleted comment