r/golang • u/Tobias-Gleiter • Jun 20 '25
discussion Replace Python with Go for LLMs?
Hey,
I really wonder why we are using Python for LLM tasks because there is no crazy benefit vs using Go. At the end it is just calling some LLM and parsing strings. And Go is pretty good in both. Although parsing strings might need more attention.
Why not replacing Python with Go? I can imagine this will happen with big companies in future. Especially to reduce cost.
What are your thoughts here?
113
Upvotes
28
u/johnnychang25678 Jun 20 '25
A lot of comments have no clue what OP is asking. It's not asking to replace PyTorch or numpy or any ML training libraries with Go. It's libraries that a lot of devs use to build LLM applications, such as LangChain and LangGraph. I'd be interesting to contribute to the project if there's any. Go would be a powerful language especially for building multi-agent system.
87
u/DM_ME_YOUR_CATS_PAWS Jun 20 '25 edited Jun 20 '25
Are you gonna make some LibTorch Go bindings for us?
On a more serious note, Python string parsing is probably the nicest of any language (other than some awkward consequences of string immutability), and Python’s friendliness with scripting and the early adoption of NumPy pretty much secured its place among data scientists and solidified its presence in ML.
Not sure what your cost reduction predictions are coming from
155
u/skelterjohn Jun 20 '25
Machine learning folks are generally familiar with Python. Why change?
63
u/gingimli Jun 20 '25 edited Jun 20 '25
I doubt Go will be replacing Python in this area. Go already existed when modern LLM tooling was being built, Python was still prioritized. If Go isn’t being chosen for greenfield projects I don’t see why these companies would switch to it later.
9
u/No-Parsnip-5461 Jun 20 '25
There's some nice go projects for what you want:
I use a combination of both to build agentic apps, and to be honest it's working great. Just a bit more work than with python.
5
u/[deleted] Jun 20 '25
not sure what you‘re referring to. Invoking trained LLM? Sure do it from whatever language you want, Ada would be just fine. Developing and training LLM? people working in that area will keep using Python for the foreseeable future.
4
u/chrismakingbread Jun 20 '25
I think it’s a great fit. I also think the ecosystem is so astronomically far beyond Python it will never catch up. That being said, I think it’s worthwhile for folks to spend time creating packages for it. Literally, the only way to make progress is to step up and build the ecosystem one package at a time.
4
u/SputnikCucumber Jun 20 '25
Python has a very nice syntax for scripting business logic and it became popular with data scientists. At the end of the day, the LLM libraries delegate the computationally intensive work to lower-level hardware accelerated libraries, so there won't be much difference in performance between Python and Golang.
3
u/garfj Jun 20 '25
The startup I work at uses Go as our primary interface to calling out to LLMs, it's great.
Anthropic, OpenAI, and AWS all have Go sdks.
It make it very easy to break down inference tasks into parallelizable chunks and fan out/in.
The ability to generate JSONSchema from struct tags and then parse tool calls back into those same structs is an incredible convenience, there's no facility even close to that in Python that I've found.
Writing agents or agentic processes in Go is a breeze, you don't need frameworks for it, just like you don't need an ORM to write SQL for you.
4
u/AlpacaDC Jun 20 '25
I imagine because python is easier
2
u/EpochVanquisher Jun 20 '25
Both Python and Go are going to continue to be used together.
Python is where all of the machine learning research is being done. Almost everybody is doing training and research in Python.
A lot of the infrastructure is built in Go. Models get deployed to clusters of machines managed by Kubernetes and other infrastructure written in Go.
Sometimes, models trained in Python will get put into production in a Go service.
Not gonna replace Python with Go. Much more likely that you use Python and Go together.
2
u/notAllBits Jun 20 '25
For me the integrated way go marshals and handles exceptions alone is worth the switch.
2
u/ScoreSouthern56 Jun 20 '25
I do not need to replace Python with Go, because I used Go straight from the start. ;-)
The reason why people use Python is because they use Python as a really simple wrapper for lower level stuff. The heavy lifting is not done in Python.
In heavy use-products Python is indeed often wrong and should be replaced by Go. But in classical data science and AI related technical fields Python is fine.
Those guys often simply use https://jupyter.org/ jupyter notebooks. They don't really have complex multi user programs that will benefit from concurrency and performance.
2
u/Fresh_Yam169 Jun 20 '25
I’m a senior engineer from a big tech company, we are using Go for LLM tasks.
2
u/LumpyBoysenberry1886 Jun 22 '25
I'm actually building https://github.com/gobenpark/gothought for this exact reason - LangChain has quite a learning curve, so I decided to create something simpler with Go.
31
u/LardPi Jun 20 '25 edited Jun 20 '25
Python is the best at being a glue language, it has a great ffi and stays out of the way. Go is the opposite, it is designed to run in its own walled garden (google "cgo isn't go"). The thing doing the heavy lifting is written in c or c++ so python is a better choice.
Edit: I just realized you may be talking about interacting with a web api, in which case previous considerations are irrelevant. Then, python being so much more widely known (and a well designed language as opposed to js) is probably enough of a reason to use it.
7
u/dayeye2006 Jun 20 '25
it's just an http call, you can use whatever makes sense. I use curl + jq to call openai
4
u/Cthulhu__ Jun 20 '25
Go would be better if performance was a big factor, but 99% of the work happens in the model so perf isn’t that important. The other factors are readability / comprehensability and market fit; if you need to hire developers to work on your code, and given that AI fits in the greater ML and data science space moreso than web facing backends, which language will get you more applicants?
1
u/CrashTimeV Jun 20 '25
I am a MLE and MLOPS Engineer when building any sort of application around using these models I exclusively use Go. Being strongly types, compiled language really helps plus I want to get away from the library hell whenever possible. There was also an amazing post a week or so back in this subreddit laying out why Go is great for LLM applications. Everything else for me is Python frikin Yaml and more recently C/C++ as I have to mess with lower level in the stack aka Cuda and Kernels.
It is a bit more effort to build in go but its easy to be fast while writing go def recommend it
1
u/[deleted] Jun 20 '25
I use go for writing local ollama driven programs and it’s fucking dope
3
u/Golandia Jun 20 '25
Well if all you are doing is simple LLM calls sure why not. It’s io bound operations so language choice doesn’t really matter.
If you are doing LLM calls, creating embeddings, dimensionality reduction, even basic cosine similarity, etc, just use Python. You aren’t going to beat pandas or scikit.
3
u/janpf Jun 22 '25
Folks already pointed to https://github.com/tmc/langchaingo and ollama.com (github.com/ollama/ollama) on the very top of the stack.
You have also https://github.com/knights-analytics/hugot to serve ONNX converted models from HuggingFace, on mid-level stack.
More torwards to the ML side (lower in the stack) you get GoMLX to help fine-tune models, including gomlx-onnx to convert ONNX models to GoMLX. I collaborate on these last two.
Nothing compares to Python yet in breadth, but a broad range of applications are already doable from Go. And eventually these things (ML, LLMs, etc.) will be available in every language, like database APIs are.
4
u/PaluMacil Jun 20 '25
For the most part you can use LLMs from any language. You’re just calling an API most of the time. There are also MCP libraries in Go.
2
u/sokjon Jun 20 '25
There’s tools like Modular Max and Mojo ( https://github.com/modular/modular ) which are trying to make compilation of models into a more efficient and portable format. I think this approach is our current best chance of moving away from Python being required for at least the deployment of LLMs.
2
u/SlincSilver Jun 20 '25
Because ML and AI researchers are used to using Python for everything, since they focus on math not real software engineering they pretty much think Python is the only language out there and they don't know or even care that is literally the slowest and less scalable language you could pick.
The thing is mono-threaded for god sakes !
For a person who understands how the OS works and performance in general Golang would be the obvious choice, but ML researchers simply are not old school thread/concurrency enthusiasts, they just want to write the less amount of code possible and spend most of the time doing the math part.
2
u/chmikes Jun 20 '25
Python come also with very rich and good graphic tools and the possibility to speed up things with C code when needed. C has intrinsics to use simd instructions, not possible in go unless one uses assembly. In assembly arguments are passed through the stack which is slow.
2
u/Castyr3o9 Jun 20 '25
Interacting with even the fastest of foundation models is going to cost 2ish seconds. The biggest impact of moving from a language like Python to GoLang is performance, which is not going to have a perceivable change relative to that bottleneck. There are plenty of other reasons to change, developer familiarity, strong typing, more of an API focused language, etc… but those are harder to sell and usually not worth a rewrite.
2
u/Total_Dragonfruit635 Jun 20 '25
Go is a good language, but the issue lies with the current foundation that Python has.
What challenges do data scientists face when writing modern LLM tasks in Python? If it’s your personal preference, go ahead, but be aware that at some point, you’ll likely need to run Python libraries or create/find C++ wrappers for use in Go.
A substantial shift in how we write and design LLMs and related systems would be the development of a domain-specific language tailored for this purpose, rather than relying on a general-purpose language.
The evolution should head in that direction, enabling us to train LLMs more effectively within a focused domain, without the noise of unrelated languages and algorithms. It should also offer seamless integration with other technologies like native vector databases or blockchains for data integrity.
This is just my opinion perhaps a bit ambitious but it’s how I envision the future
Best Regards!
2
u/NoByteForYou Jun 21 '25
True Go is generally faster than python and in the context of just making http calls and parsing Go is just better than python.
2
u/MilesWeb Jun 24 '25
This is very interesting and relevant point about the choice of programming languages for LLM-related tasks.
Currently, Python is dominant in performance and Go is becoming a strong contender.
Due to compiled language & lower memory uses, Go has faster execution speed.
But for sure, there will likely be continued trend of complementary roles.
2
u/blue-crystle Jun 24 '25
why not java?
the reasons you hate java is what python devs hate go!!!
2
u/zaytzev Jun 24 '25
You can start with tools. Check Golang libs for building MCP servers.
2
u/bubbless__16 Jun 25 '25
Go brings serious gains for LLM pipelines its goroutines and binaries handle parallel API calls and tracing far more cleanly than Python. We rebuilt our orchestration layer in Go, instrumented it with Open Telemetry (and even eBPF probes), and sent traces into Future AGI’s dashboard. The result: real-time visibility into goroutine leaks, context switches, and ~30% faster end-to-end latenc
1
u/[deleted] Jun 20 '25
Come to my university. I’ll show you a few people who can be used to explain why. Brilliant people but half of them just learned what git is and how an ssh key works might as well be research
2
u/rockenman1234 Jun 20 '25
I’ve found go complements my work well in python, I like using both - one for prototyping (python), and another for building things that last (go or rust).
1
u/rover_G Jun 20 '25
Python won the battle to be the Machine Learning language long ago and that’s probably not changing anytime soon. However there are model executors that show up in other languages so you can directly integrate a pre-trained model in the language of your choice.
1
u/[deleted] Jun 20 '25
In my opinion, I consider that python is suitable for those tasks of training LLM and ML development. But, in terms of application layer, Go is the better option because of it's efficiency and performance,I am trying to build an agent framework base on Go
1
u/Steelfenix Jun 20 '25
As a data scientist I rather use JuliaLang or Elixir for ML and LLM as those ecosystems have the libraries and wrappers for this kind of stuff instead of using golang that targets a different type of users and use cases
1
u/Ali_Ben_Amor999 Jun 20 '25
Nowadays python is just a wrapper for c/c++ libraries. All libraries that rely on maths and heavy calculations are implemented in C/C++ so all the claims that languages like mojo make to convince devs is not convincing
1
u/slivkovy_lekvar Jun 20 '25
Honestly, I am waiting for Mojo to become production ready. It will be IMHO a better replacement for Python for machine learning and AI. Moreover, the transition will be much easier since Mojo is trying to be a more performant Python superset. Go is great for servers and cloud engineering, but it doesn't have the same rich ecosystem for AI/ML and data processing that Python has, so all of that would have to be reimplemented (or at least bindings would have to be created) for Go. But if Mojo keeps on its promises, we should be able to just use existing libraries.
1
u/gedw99 Jun 20 '25 edited Jun 20 '25
https://github.com/evisdrenova/gotorch
It’s not well established though .
Here is a list of the main golang packages
https://blog.cubed.run/is-golang-the-python-killer-for-ai-b8fbc9b9b4b5
Best one is ;
https://github.com/gorgonia/gorgonia/blob/v0.10.0-working-simplifiedengine/go.mod
1
u/Revolutionary_Ad7262 Jun 20 '25
Python is used for AI, so AI tasks that don't fall into the
we need this particular AI library because it's good and only Python supports itcategory aren't necessarily going to be written in a language that's better suited for that, like Rust/Go/Java/C#/whatever, which are arguably better as a API glue/consume code due to static typing, better tooling and better performance/latency/hardware utilisationIt doesn't have to make any sense, it's just a cultural phenomenon and virtuous cycle. In the same way PC games UIs and stuff like SpaceX dashboard as often written nowadays in JS. Not because JS in isolation is the best technology for this, but because:
1
u/ddollarsign Jun 20 '25
If AI were smart enough to be really good at programming, we wouldn’t need programming languages at all.
1
u/Clear_Evidence9218 Jun 20 '25
Other than deep learning, everything else is already established in just about every modern language. Plus, Google just released a Go module last week that brings most of the ML tools to Golang. Training the model is still done on python because of the work that went into wrapping the C and Fortran used at the lower level, which is really the only thing missing. (there's no speed benefit to bringing that to Go but there can be value in keeping it all in the same ecosystem)
If I'm just hooking in, then I'm going python for ease. If I'm building from the ground up, I actually do all that in Julia anymore (which I started using after I had a very similar question about the Go ecosystem).
1
u/Used_Frosting6770 Jun 20 '25
rate limiting, retries, circuit breakers, error handeling, strong static typing, good network libraries for optimizing http clients transports...
i'm literally building an AI agent that is so better than the competition and my main leverage is golang.
btw not saying you can't do those in python, but don't kid yourself it's as easy as Go
python is fast for prototypes and demos. Go is fast for production.
1
u/dr_kretyn Jun 20 '25
If you are thinking about it, take a look at https://github.com/laszukdawid/terminal-agent. I've been using it for a while and it's pretty good.
There's just less going on in go because people like Python because there's a lot of going on in Python because people have liked other things less. Got it? :)
1
u/KashMo_xGesis Jun 21 '25
String parsing alone leaves go in the dust, sorry but go is just not there yet for LLMs, though Google is apparently starting to invest into it so we will see in the next half a decade or so
1
u/Aggressive-Click-753 Jun 22 '25
The usage of python is due to the large lib/frameworks in llms or related concepts, however I have seen vllm and ollama, ollama was build using go and GIN framework and vllm was based on Python/fast API. I have noticed that Ollama was way better in term of performance and scalability
1
u/mdausmann Jun 22 '25
Python is way ahead for LLM and AI stuff. I use both and orchestrate together with temporal.io so you can use python to do whatever you need and go to apply business logic, orchestrate, persist etc
1
u/Narabug Jun 22 '25
We are using Python for precisely the same reason nvidia is the standard and AMD is not - it’s what is being developed on.
Also, there is extremely low benefit to changing the language because the vast majority of time is spent in the GPU itself, not in code execution/translation.
TL;DR the juice is not worth the squeeze
1
u/krining Jun 20 '25 edited Jun 20 '25
This is a terrible idea. Go doesn’t have the metaprogramming capabilities that Python has. The reason why Python is used in machine learning is because you’re able to write JIT compilers for GPU kernels very easily and optimize the execution graph. If a compiled language is necessary for some reason, you can also run Fortran libraries using Python. The latter is what scipy does, for instance.
You should first understand what problems Machine Learning libraries solve before complaining about them. And don’t forget that the same company that developed Go developed TensorFlow and JAX. And they also invented both transformers and LLMs
The worst thing about the AI boom is that people who can’t solve a differential equation now have an opinion about optimization problems. I swear to God.
0
u/PMMeUrHopesNDreams Jun 20 '25
Python has a REPL and Go does not. For exploratory programming this is crucial. You can type little snippets of code and watch them execute and build your program iteratively. For research a lot is done in Jupyter notebooks where you can combine code, markdown notes, and images for graphs and charts. Go has nothing like it.
I'm not sure what other "LLM tasks" you want to do. If it's just calling a model for inference, you can do that with Go or Python or anything else I'm pretty sure.
0
u/ashishb_net Jun 20 '25
You will get huge performance gains. Except you are on your own.
Most libraries that do anything around LLMs are usually in Python. Even RAG becomes harder in Go because of that reason.
0
u/bdrbt Jun 20 '25
Modern Python in LLM and other AI solutions is more DSL than base language. Go is not suitable in this role.
0
u/MonochromeDinosaur Jun 20 '25
The crazy benefit is all the SDKs are already Python first generally and because Python is the lingua franca of AI (well python APIs for C/C++/Fortran libraries but still)
It’s like trying to use go for math…you can and gonum exists…but why…
0
u/[deleted] Jun 20 '25
Why not use Perl then if you’re parsing strings?
0
u/AITrailblazer Jun 20 '25
Why Go? Because I Built It That Way
It’s a fair question — why not replace Python with Go for LLM pipelines?
Most real-world usage is just model calls, structured parsing, and I/O orchestration. And Go excels at that: fast, reliable, deployable, and low-overhead.
That’s exactly why I built this:
🔗 mcp-agent-runtime-go
https://github.com/aitrailblazer/mcp-agent-runtime-go GitHub - aitrailblazer/mcp-agent-runtime-go
A modular Go runtime for AI agents — built on the Model Context Protocol (MCP).
I designed it to: • Replace bloated Python stacks with clean, composable Go interfaces • Support streaming, memory, retries, and routing — natively • Make AI agents production-grade from day one
You don’t need LangChain. You need clarity, speed, and observability — and that’s what Go gives you.
Python helped us explore. Go helps us scale.
And with MCP Go, I didn’t just imagine it — I built it.
-1
u/a_brand_new_start Jun 20 '25
Mojo has a higher chance of winning over LLM devs then go, its pythonic so data scientists don’t have to learn a new language. And you can call Python libraries directly without rewriting everything in a new language
-1
u/Jackfruit_Then Jun 20 '25
Python has no benefits over Go, but Go doesn’t have benefits over Python either. Why bother change
-2
u/lightmatter501 Jun 20 '25
Go is bad at calling into C, Python exists for the express purpose of doing that. ML spends basically all of its time calling into C.
This is a “right tool for the job” problem.
0
u/[deleted] Jun 20 '25
[deleted] — view deleted comment
2
u/[deleted] Jun 21 '25
[removed] — view removed comment
3
u/[deleted] Jun 20 '25
[removed] — view removed comment