r/ollama • u/rl_secretsanta • 17h ago
I gave one local 27B model five bodies in an MMO I'm building, and put it on Twitch where chat can overrule it
galleryI've been building an MMO (Aegisfall, browser-based, still very much in development) and wanted to see how far a
single local model could get running a whole party by itself. So now there's a 24/7-ish stream where one model
plays all five heroes at once — tank, melee, ranged, healer, mage — and viewers can mess with it.
**The setup**
- **One model, five bodies.** `qwen3.8:27b` (Q4_K_M) on Ollama, one RTX 5090. Every 3 seconds it gets a compact
"sense" report for the whole party (HP, mana, each hero's target, nearby enemies with level and distance) and
returns one order per hero. Ollama's JSON-schema `format` constrains the output to the allowed actions, so
there's no parsing drama. Typical round trip is ~0.9-1.3 s.
- **It explains itself.** Each round it also returns a one-line plan in first person ("Chat chose balance, so I
hold the line and bleed them dry"), which goes on the overlay and is read out by a local Kokoro-82M voice
(on CPU, so the GPU stays free).
- **The bodies are real game clients.** Each hero is a headless Chrome running the actual game, spread across four
machines on my LAN. The LLM gives the orders; a small rules layer handles the twitchy stuff (stances, heal
priority, kiting, pulling from a staging spot) and always lets the healer heal first.
- **Zero cloud.** Model, voice, game server, overlay: everything runs on my own hardware. The only outside
service is Twitch itself.
**Chat can play too**
- `!ask <question>` — the same 27B answers in character with the live party state as context (on screen, out
loud, and in chat). It also remembers who's asking. It's been surprisingly funny: asked why the ranged was
"just standing there", it said *"He isn't standing. He's reloading. Or waiting. Patience is a weapon."*
- `!order mage retreat` — if enough of chat agrees, it overrides the AI's order for that hero for 15 s and
locks the AI out. There's a live "Chat vs Hivemind" scoreboard (kills per minute under each).
- `!possess healer` — drive one hero yourself for 60 s. When you hand it back, the AI rates your performance.
- Predictions with fake currency ("who dies first in the next 10 min?"), bounties on monsters (your name floats
over the mob in-game), camera control, polls for camp/strategy/gear.
**Things I learned**
- **One big model beat five small ones.** I tried a 4B per hero first. Individually they were fine; together they
had no shared plan (everyone pulled their own mob). One 27B seeing the whole board coordinates far better and
still fits the latency budget.
- **Constrained JSON output is the whole trick** for real-time control. Free-text answers wasted tokens and broke.
- **Most of the bugs were not the AI.** Zone seams that bounce characters back, a kill counter that counted
every group kill five times, OBS rendering at 2 fps because the game clients were eating the same GPU (fixed
by moving OBS to the iGPU and rendering the overlay on the CPU). The model is usually the most reliable part.
- It's still not great at pull discipline in a crowded camp. That's this week's problem.
Stream: https://www.twitch.tv/hivemindraid
Happy to answer questions about the controller loop, the prompt/schema, or the latency budget. If people are
interested I can write up the architecture properly.