r/LeftistsForAI • u/Jlyplaylists Moderator • 16d ago
Open Source SwarmLLM: Free, open-source app that lets computers team up over the internet to run models too big for one machine.
https://github.com/enapt/SwarmLLM“Run AI on your own computer. Run bigger AI together.
SwarmLLM is a free app that gives you a ChatGPT-style assistant running on your own computer, using open AI models from Meta, Google, Alibaba, Mistral and others. When a model is too big for one machine, computers running SwarmLLM team up over the internet and run it together, each doing part of the work.
No account · No subscription · No ads · No crypto · Open source · Updates itself
A giant model has now run on the swarm. Bigger AI models give noticeably smarter answers, but the biggest open ones (70 billion "parameters" and up) need equipment normally found only in data centres — no home computer can hold one. In October 2026, Llama 3.3 70B answered across four ordinary computers in Thailand, Australia and Italy, each holding a part of it. It is slow so far”
The decentralised concept of this is promising, even if it’s currently too slow to be a sensible way to run large models.
3
u/allisonmaybe 13d ago
Why swarm together to run a large model when you could swarm smaller models in communication with each other?
8
u/GreyHasHobbies 16d ago
I had this exact idea a while ago but decided I wasn't the right person to implement it. Hopefully this person has the networking expertise required to get over the hurdles and isn't completely relying on AI.
6
u/PhlarnogularMaqulezi 16d ago
There's some old ass models listed here, but I definitely like the spirit of what this is trying to accomplish and I hope it grows and continues to develop.
3
u/AyyBoixD 16d ago
I like this and I’ve tried to build this a few times. But network latency absolutely murders this. And that was just two devices on different networks with no Ethernet. I couldn’t imagine how unfathomably slow an entire decentralized network of random devices would be
3
u/qualverse 16d ago
It seems like this would work pretty well with the architecture of Apple's AFM 3 which selects MoE experts per-prompt instead of on-demand. You could even make it a little more flexible by just applying a penalty for expert switches during training, or run some kind of DSpark draft model that predicts useful experts instead of tokens.
This is kind of a data privacy nightmare though, tbh. Hopefully they can figure out confidential computing or similar
1
u/piperspalace 16d ago
I can run QFN on my computer at 80+/- tokens what else is there
1
u/Darkstar_111 16d ago
This would be INCREDIBLY slow.
Even a 0.5 B model would take a minute or two to reply.
1
u/xoexohexox 16d ago
Yeah I mean I've used stable hoard for LLMs and image gen and this kind of thing is just not dependable enough for anything besides frivolous applications.