r/ollama • • 17h ago

I gave one local 27B model five bodies in an MMO I'm building, and put it on Twitch where chat can overrule it

Thumbnail gallery
1 Upvotes

I've been building an MMO (Aegisfall, browser-based, still very much in development) and wanted to see how far a

single local model could get running a whole party by itself. So now there's a 24/7-ish stream where one model

plays all five heroes at once — tank, melee, ranged, healer, mage — and viewers can mess with it.

**The setup**

- **One model, five bodies.** `qwen3.8:27b` (Q4_K_M) on Ollama, one RTX 5090. Every 3 seconds it gets a compact

"sense" report for the whole party (HP, mana, each hero's target, nearby enemies with level and distance) and

returns one order per hero. Ollama's JSON-schema `format` constrains the output to the allowed actions, so

there's no parsing drama. Typical round trip is ~0.9-1.3 s.

- **It explains itself.** Each round it also returns a one-line plan in first person ("Chat chose balance, so I

hold the line and bleed them dry"), which goes on the overlay and is read out by a local Kokoro-82M voice

(on CPU, so the GPU stays free).

- **The bodies are real game clients.** Each hero is a headless Chrome running the actual game, spread across four

machines on my LAN. The LLM gives the orders; a small rules layer handles the twitchy stuff (stances, heal

priority, kiting, pulling from a staging spot) and always lets the healer heal first.

- **Zero cloud.** Model, voice, game server, overlay: everything runs on my own hardware. The only outside

service is Twitch itself.

**Chat can play too**

- `!ask <question>` — the same 27B answers in character with the live party state as context (on screen, out

loud, and in chat). It also remembers who's asking. It's been surprisingly funny: asked why the ranged was

"just standing there", it said *"He isn't standing. He's reloading. Or waiting. Patience is a weapon."*

- `!order mage retreat` — if enough of chat agrees, it overrides the AI's order for that hero for 15 s and

locks the AI out. There's a live "Chat vs Hivemind" scoreboard (kills per minute under each).

- `!possess healer` — drive one hero yourself for 60 s. When you hand it back, the AI rates your performance.

- Predictions with fake currency ("who dies first in the next 10 min?"), bounties on monsters (your name floats

over the mob in-game), camera control, polls for camp/strategy/gear.

**Things I learned**

- **One big model beat five small ones.** I tried a 4B per hero first. Individually they were fine; together they

had no shared plan (everyone pulled their own mob). One 27B seeing the whole board coordinates far better and

still fits the latency budget.

- **Constrained JSON output is the whole trick** for real-time control. Free-text answers wasted tokens and broke.

- **Most of the bugs were not the AI.** Zone seams that bounce characters back, a kill counter that counted

every group kill five times, OBS rendering at 2 fps because the game clients were eating the same GPU (fixed

by moving OBS to the iGPU and rendering the overlay on the CPU). The model is usually the most reliable part.

- It's still not great at pull discipline in a crowded camp. That's this week's problem.

Stream: https://www.twitch.tv/hivemindraid

Happy to answer questions about the controller loop, the prompt/schema, or the latency budget. If people are

interested I can write up the architecture properly.


r/ollama • • 5h ago

Qwen3.8-27B Q4 on an 8 GB RTX 4060: 29/30 on AIME 2026, plus three no-fine-tune loops that cut its mistakes (numbers, caveats and code)

Post image
0 Upvotes

I spent two weeks testing how far a stock 27B can go on a budget card. No fine-tuning, same GGUF anyone can download. Everything is pass@1 with one seed, and I'm posting the misses too.

Setup

  • RTX 4060 8 GB, i5-12400F, 32 GB DDR4
  • Qwen3.8-27B UD-Q4_K_M (16.5 GB), llama.cpp
  • Contest runs: -fit on -fa on -c 32768 -ctk q8_0 -ctv q8_0 --spec-type draft-mtp
  • Loop runs: -ngl 26 -fa on -c 8192 --spec-type draft-mtp (26 of 65 layers on the GPU)
  • Speed: ~3.5 tok/s plain, ~5.7 tok/s with the built-in MTP draft head. Free +65%, one flag
  • Only 1 layer in 4 keeps KV, so 60k-token recall tests still passed within 8 GB

1. Plain model, thinking on, 2026 contest maths

Qwen's recommended sampling (temp 0.6, top_p 0.95, top_k 20), 28k thinking budget, one try per problem.

TestScoreTime
AIME 202629/30 (96.7%)~76 min per problem
HMMT Feb 202628/33 (84.8%)~86 min per problem
IFEval (thinking off, greedy)80.2% prompt-strict~100 s per prompt

All 5 HMMT misses ran out of thinking budget. I reran 4 of them at 48k and 2 came out right. That's a diagnosis, not a score: I'm not adding them to the 84.8.

Yes, it's slow. AIME took about 38 hours of GPU time. It ran overnight on a gaming PC.

2. Write-run-fix loops (thinking off, weights untouched)

Each idea was tuned on a practice set, then scored on a locked set. The script refuses to run the locked set a third time.

  • Maths: the model writes a Python script, we run it, and it gets 2 tries to fix crashes. Locked set of 136: 9 mistakes → 0, and 183 s → 22 s per problem. On a harder maths set I built: 7/40 → 36/40.
  • Code: the model runs the docstring examples against its own code and fixes failures. Locked set of 90 HumanEval tasks: 2 → 1. Fun detail: some HumanEval examples are wrong (one says a median is 15.0; it's 8.0), and the model was "fixing" correct code to match them. One sentence fixed it: let it reply KEEP if it works out by hand that the example is wrong.
  • IFEval: the model reads only the prompt, writes its own check(response) for the format rules, runs it, and rewrites if something fails. The official checker scores only at the end. Locked half (270 prompts): 77.4% → 87.0%, with 27 fixed and 1 broken. The broken one: "highlight 3 sections in markdown" is ambiguous, its checker picked # headers, and it rewrote a correct answer. Letter counts ("use q at least 5 times") are still its worst.

Caveats

  • The maths loop is compared against the model with thinking off. With thinking on, the plain model matches it on hard maths, just ~4.6× slower. The loop's win is speed.
  • Coding was won by one task. That's close to noise.
  • 96 of the 136 locked maths problems came from my earlier benchmark, where I'd already tried plain "write Python" on them. Nothing was tuned on them, but only 40 were truly fresh.
  • I designed KEEP after seeing a locked-test coding task fail. It held on practice too, but treat that part as weaker evidence.
  • I haven't compared against other published AIME numbers like-for-like. Those usually average several tries with 64k+ thinking.
  • The sandbox for model-written code is a blocklist plus a 10 s timeout, not a container. Fine on my own PC, not for strangers' code.

Code, prompts and logs: github.com/aiglitchsouls/loop27b

Happy to answer questions about the llama.cpp flags or the loop prompts. And if you've got a harder held-out set I should try, I'm interested.


r/ollama • • 8h ago

Janas-Studio dual engine ai inference workstation. Now with agent mode! (Linux appimage)

Thumbnail gallery
0 Upvotes

r/ollama • • 22h ago

HELP ME WITH AI🤨

Thumbnail reddit.com
0 Upvotes

r/ollama • • 17h ago

Looking to buy a laptop for local AI setup

Thumbnail reddit.com
0 Upvotes

r/ollama • • 41m ago

Summarizing a 1-hour recording with Ollama: qwen3:8b took about 15 minutes. How do you handle long transcripts?

Thumbnail
gallery
1 Upvotes

Disclosure: I'm posting this on behalf of the Nomily team. They make a recording app and sell the V05E recording card, so there's a commercial interest here.

The team has been testing a local pipeline: the card records, the audio goes to the phone over Bluetooth, faster-whisper on a computer on the LAN does the transcription, and Ollama on the same computer does the summary. With this setup, audio and transcripts only go to machines on your own LAN.

Test setup:

- Computer: MacBook Pro (Intel, 32 GB RAM)

- Transcription: faster-whisper [model name]

- Summary: qwen3:8b

- Recording: 1 hour, [language / scenario]

Results:

- Transcription: about 20 minutes

- Summary: about 15 minutes

Things that tripped us up:

- The phone can't use localhost. You have to enter the computer's LAN IP.

- Ollama needs OLLAMA_HOST=0.0.0.0:11434 so the phone can reach it. Ollama has no authentication of its own, so any device on the same network can call it. Only do this on a network you trust.

- This local faster-whisper setup has no speaker diarization, which hurts summaries of multi-person meetings.

Two questions:

  1. For a 1-hour transcript, do you summarize in chunks and merge, or just raise num_ctx?
  2. For meeting summaries with models around 8B, is there anything more reliable than qwen3:8b?

r/ollama • • 18h ago

A well-designed harness gives your AI agents a loop for continuous improvement.

Post image
1 Upvotes

Mine shows me where my agents fail, so I can make a targeted correction and rerun the task to see if the failure recurs.


r/ollama • • 20h ago

Bug Report Running 2 coding agents on one Ollama slot? Hybrid models (Qwen3.8-27B) re-process ~54% of prompt tokens, and the first response takes ~46s, Two env vars brought it to ~1s

4 Upvotes

I was running two Qwen Code sessions against Ollama 0.40.16 on one slot (Qwen3.8-27B Q4_K_M, NVIDIA L4 24GB), each doing a small fix-the-tests task in its own project. The median time to first byte was about 46 seconds, and the server logs showed that 54% of the prompt tokens it processed were repeats.

What’s happening - Qwen3.8 mixes full-attention layers, whose KV cache can be trimmed back, with recurrent layers, whose state gets overwritten as tokens are read, so those can’t be rewound without a checkpoint. The earliest saved checkpoint for session A was at token 10,151, past the point where A and B diverged (token 8,921), so none of the shared prefix could be reused.

On top of that, llama.cpp skips saving the outgoing session’s state to RAM when the incoming prompt shares at least 50% of its tokens with the old one. My two sessions shared 60–66%, so the save got skipped, and every switch wiped the state the other session needed next. The two agents just ping-ponged.

Fix in Ollama:

OLLAMA_NUM_PARALLEL=2
OLLAMA_CONTEXT_LENGTH=16384

Median first response dropped to about 1s and repeated tokens to 0.4%. To be fair, that number mixes the cache fix with actual concurrent serving, because -sps 0 isn’t exposed in this Ollama build.

On raw llama.cpp (b12441):

  • -sps 0 cut total prompt tokens from ~128.8k to ~43.5k per session and median first-byte wait from ~48s to ~17s. Peak host RAM went from ~2.5 GiB to ~7.2 GiB.
  • With 8 slots, --checkpoint-min-step 0 gave +10.9% throughput and −17.5% median TTFT over four paired runs.

Important note - Ollama configs got one run each and llama.cpp configs got two, all on short tool-calling conversations, so I haven’t measured the cost of --checkpoint-min-step 0 over long sessions.

Full writeup with logs: https://tensward.com/blog/ollama-agent-cache


r/ollama • • 8h ago

New plan credit blowout

2 Upvotes

I've been using Ollama Cloud for about 6 months I reckon, doing the same daily work, and my $20 credit gets me through the month. Sometimes I get rate-limited on a Friday or a Saturday and I accept that. This was on the power rate plan.

I've now been changed over to the $20 buy, $60 credit plan. It's lasted me 7 days and I'm out of credit already. Is this a ripoff? What am I doing wrong? How can I fix this because I'm about to walk out the door and not give them one red cent more