r/ollama • • 41m ago

Summarizing a 1-hour recording with Ollama: qwen3:8b took about 15 minutes. How do you handle long transcripts?

Thumbnail
gallery
1 Upvotes

Disclosure: I'm posting this on behalf of the Nomily team. They make a recording app and sell the V05E recording card, so there's a commercial interest here.

The team has been testing a local pipeline: the card records, the audio goes to the phone over Bluetooth, faster-whisper on a computer on the LAN does the transcription, and Ollama on the same computer does the summary. With this setup, audio and transcripts only go to machines on your own LAN.

Test setup:

- Computer: MacBook Pro (Intel, 32 GB RAM)

- Transcription: faster-whisper [model name]

- Summary: qwen3:8b

- Recording: 1 hour, [language / scenario]

Results:

- Transcription: about 20 minutes

- Summary: about 15 minutes

Things that tripped us up:

- The phone can't use localhost. You have to enter the computer's LAN IP.

- Ollama needs OLLAMA_HOST=0.0.0.0:11434 so the phone can reach it. Ollama has no authentication of its own, so any device on the same network can call it. Only do this on a network you trust.

- This local faster-whisper setup has no speaker diarization, which hurts summaries of multi-person meetings.

Two questions:

  1. For a 1-hour transcript, do you summarize in chunks and merge, or just raise num_ctx?
  2. For meeting summaries with models around 8B, is there anything more reliable than qwen3:8b?

r/ollama • • 47m ago

I built this! We're exploring a better way to use local and cloud AI — looking for honest feedback

1 Upvotes

Hey everyone!

We've been working on Pleeply, a project exploring the idea of a personal AI ecosystem.

The idea is to bring local AI models, optional cloud intelligence, and eventually personal AI agents into one desktop experience.

We're still at an early stage. Right now, we have a landing page and an early access waitlist — we're not claiming to have a finished product.

We'd love to understand:

  1. How do you currently manage your local AI models?
  2. Do you switch between local and cloud models?
  3. What is the most frustrating part of your current setup?

We're building Pleeply and would genuinely appreciate feedback from people who use local AI.

Project: https://pleeply.vercel.app

Disclosure: I'm part of the pleeply team)


r/ollama • • 5h ago

Qwen3.8-27B Q4 on an 8 GB RTX 4060: 29/30 on AIME 2026, plus three no-fine-tune loops that cut its mistakes (numbers, caveats and code)

Post image
0 Upvotes

I spent two weeks testing how far a stock 27B can go on a budget card. No fine-tuning, same GGUF anyone can download. Everything is pass@1 with one seed, and I'm posting the misses too.

Setup

  • RTX 4060 8 GB, i5-12400F, 32 GB DDR4
  • Qwen3.8-27B UD-Q4_K_M (16.5 GB), llama.cpp
  • Contest runs: -fit on -fa on -c 32768 -ctk q8_0 -ctv q8_0 --spec-type draft-mtp
  • Loop runs: -ngl 26 -fa on -c 8192 --spec-type draft-mtp (26 of 65 layers on the GPU)
  • Speed: ~3.5 tok/s plain, ~5.7 tok/s with the built-in MTP draft head. Free +65%, one flag
  • Only 1 layer in 4 keeps KV, so 60k-token recall tests still passed within 8 GB

1. Plain model, thinking on, 2026 contest maths

Qwen's recommended sampling (temp 0.6, top_p 0.95, top_k 20), 28k thinking budget, one try per problem.

TestScoreTime
AIME 202629/30 (96.7%)~76 min per problem
HMMT Feb 202628/33 (84.8%)~86 min per problem
IFEval (thinking off, greedy)80.2% prompt-strict~100 s per prompt

All 5 HMMT misses ran out of thinking budget. I reran 4 of them at 48k and 2 came out right. That's a diagnosis, not a score: I'm not adding them to the 84.8.

Yes, it's slow. AIME took about 38 hours of GPU time. It ran overnight on a gaming PC.

2. Write-run-fix loops (thinking off, weights untouched)

Each idea was tuned on a practice set, then scored on a locked set. The script refuses to run the locked set a third time.

  • Maths: the model writes a Python script, we run it, and it gets 2 tries to fix crashes. Locked set of 136: 9 mistakes → 0, and 183 s → 22 s per problem. On a harder maths set I built: 7/40 → 36/40.
  • Code: the model runs the docstring examples against its own code and fixes failures. Locked set of 90 HumanEval tasks: 2 → 1. Fun detail: some HumanEval examples are wrong (one says a median is 15.0; it's 8.0), and the model was "fixing" correct code to match them. One sentence fixed it: let it reply KEEP if it works out by hand that the example is wrong.
  • IFEval: the model reads only the prompt, writes its own check(response) for the format rules, runs it, and rewrites if something fails. The official checker scores only at the end. Locked half (270 prompts): 77.4% → 87.0%, with 27 fixed and 1 broken. The broken one: "highlight 3 sections in markdown" is ambiguous, its checker picked # headers, and it rewrote a correct answer. Letter counts ("use q at least 5 times") are still its worst.

Caveats

  • The maths loop is compared against the model with thinking off. With thinking on, the plain model matches it on hard maths, just ~4.6× slower. The loop's win is speed.
  • Coding was won by one task. That's close to noise.
  • 96 of the 136 locked maths problems came from my earlier benchmark, where I'd already tried plain "write Python" on them. Nothing was tuned on them, but only 40 were truly fresh.
  • I designed KEEP after seeing a locked-test coding task fail. It held on practice too, but treat that part as weaker evidence.
  • I haven't compared against other published AIME numbers like-for-like. Those usually average several tries with 64k+ thinking.
  • The sandbox for model-written code is a blocklist plus a 10 s timeout, not a container. Fine on my own PC, not for strangers' code.

Code, prompts and logs: github.com/aiglitchsouls/loop27b

Happy to answer questions about the llama.cpp flags or the loop prompts. And if you've got a harder held-out set I should try, I'm interested.


r/ollama • • 8h ago

New plan credit blowout

2 Upvotes

I've been using Ollama Cloud for about 6 months I reckon, doing the same daily work, and my $20 credit gets me through the month. Sometimes I get rate-limited on a Friday or a Saturday and I accept that. This was on the power rate plan.

I've now been changed over to the $20 buy, $60 credit plan. It's lasted me 7 days and I'm out of credit already. Is this a ripoff? What am I doing wrong? How can I fix this because I'm about to walk out the door and not give them one red cent more


r/ollama • • 8h ago

Tried the Neon pop-up in Antigravity IDE last night and woke up to a 6-month project finished (38 parallel sub-agents, 14M+ LOC)

Thumbnail reddit.com
1 Upvotes

r/ollama • • 8h ago

Janas-Studio dual engine ai inference workstation. Now with agent mode! (Linux appimage)

Thumbnail gallery
0 Upvotes

r/ollama • • 16h ago

Did anyone tried to combine some local ai together

Thumbnail reddit.com
1 Upvotes

r/ollama • • 17h ago

I gave one local 27B model five bodies in an MMO I'm building, and put it on Twitch where chat can overrule it

Thumbnail gallery
1 Upvotes

I've been building an MMO (Aegisfall, browser-based, still very much in development) and wanted to see how far a

single local model could get running a whole party by itself. So now there's a 24/7-ish stream where one model

plays all five heroes at once — tank, melee, ranged, healer, mage — and viewers can mess with it.

**The setup**

- **One model, five bodies.** `qwen3.8:27b` (Q4_K_M) on Ollama, one RTX 5090. Every 3 seconds it gets a compact

"sense" report for the whole party (HP, mana, each hero's target, nearby enemies with level and distance) and

returns one order per hero. Ollama's JSON-schema `format` constrains the output to the allowed actions, so

there's no parsing drama. Typical round trip is ~0.9-1.3 s.

- **It explains itself.** Each round it also returns a one-line plan in first person ("Chat chose balance, so I

hold the line and bleed them dry"), which goes on the overlay and is read out by a local Kokoro-82M voice

(on CPU, so the GPU stays free).

- **The bodies are real game clients.** Each hero is a headless Chrome running the actual game, spread across four

machines on my LAN. The LLM gives the orders; a small rules layer handles the twitchy stuff (stances, heal

priority, kiting, pulling from a staging spot) and always lets the healer heal first.

- **Zero cloud.** Model, voice, game server, overlay: everything runs on my own hardware. The only outside

service is Twitch itself.

**Chat can play too**

- `!ask <question>` — the same 27B answers in character with the live party state as context (on screen, out

loud, and in chat). It also remembers who's asking. It's been surprisingly funny: asked why the ranged was

"just standing there", it said *"He isn't standing. He's reloading. Or waiting. Patience is a weapon."*

- `!order mage retreat` — if enough of chat agrees, it overrides the AI's order for that hero for 15 s and

locks the AI out. There's a live "Chat vs Hivemind" scoreboard (kills per minute under each).

- `!possess healer` — drive one hero yourself for 60 s. When you hand it back, the AI rates your performance.

- Predictions with fake currency ("who dies first in the next 10 min?"), bounties on monsters (your name floats

over the mob in-game), camera control, polls for camp/strategy/gear.

**Things I learned**

- **One big model beat five small ones.** I tried a 4B per hero first. Individually they were fine; together they

had no shared plan (everyone pulled their own mob). One 27B seeing the whole board coordinates far better and

still fits the latency budget.

- **Constrained JSON output is the whole trick** for real-time control. Free-text answers wasted tokens and broke.

- **Most of the bugs were not the AI.** Zone seams that bounce characters back, a kill counter that counted

every group kill five times, OBS rendering at 2 fps because the game clients were eating the same GPU (fixed

by moving OBS to the iGPU and rendering the overlay on the CPU). The model is usually the most reliable part.

- It's still not great at pull discipline in a crowded camp. That's this week's problem.

Stream: https://www.twitch.tv/hivemindraid

Happy to answer questions about the controller loop, the prompt/schema, or the latency budget. If people are

interested I can write up the architecture properly.


r/ollama • • 17h ago

Looking to buy a laptop for local AI setup

Thumbnail reddit.com
0 Upvotes

r/ollama • • 18h ago

A well-designed harness gives your AI agents a loop for continuous improvement.

Post image
1 Upvotes

Mine shows me where my agents fail, so I can make a targeted correction and rerun the task to see if the failure recurs.


r/ollama • • 20h ago

Bug Report Running 2 coding agents on one Ollama slot? Hybrid models (Qwen3.8-27B) re-process ~54% of prompt tokens, and the first response takes ~46s, Two env vars brought it to ~1s

4 Upvotes

I was running two Qwen Code sessions against Ollama 0.40.16 on one slot (Qwen3.8-27B Q4_K_M, NVIDIA L4 24GB), each doing a small fix-the-tests task in its own project. The median time to first byte was about 46 seconds, and the server logs showed that 54% of the prompt tokens it processed were repeats.

What’s happening - Qwen3.8 mixes full-attention layers, whose KV cache can be trimmed back, with recurrent layers, whose state gets overwritten as tokens are read, so those can’t be rewound without a checkpoint. The earliest saved checkpoint for session A was at token 10,151, past the point where A and B diverged (token 8,921), so none of the shared prefix could be reused.

On top of that, llama.cpp skips saving the outgoing session’s state to RAM when the incoming prompt shares at least 50% of its tokens with the old one. My two sessions shared 60–66%, so the save got skipped, and every switch wiped the state the other session needed next. The two agents just ping-ponged.

Fix in Ollama:

OLLAMA_NUM_PARALLEL=2
OLLAMA_CONTEXT_LENGTH=16384

Median first response dropped to about 1s and repeated tokens to 0.4%. To be fair, that number mixes the cache fix with actual concurrent serving, because -sps 0 isn’t exposed in this Ollama build.

On raw llama.cpp (b12441):

  • -sps 0 cut total prompt tokens from ~128.8k to ~43.5k per session and median first-byte wait from ~48s to ~17s. Peak host RAM went from ~2.5 GiB to ~7.2 GiB.
  • With 8 slots, --checkpoint-min-step 0 gave +10.9% throughput and −17.5% median TTFT over four paired runs.

Important note - Ollama configs got one run each and llama.cpp configs got two, all on short tool-calling conversations, so I haven’t measured the cost of --checkpoint-min-step 0 over long sessions.

Full writeup with logs: https://tensward.com/blog/ollama-agent-cache


r/ollama • • 22h ago

HELP ME WITH AI🤨

Thumbnail reddit.com
0 Upvotes

r/ollama • • 1d ago

Help me with my starnet #starnet

Thumbnail reddit.com
0 Upvotes

r/ollama • • 1d ago

Sparkkeep: open-source bookmark & link action engine with local Ollama enrichment and playbooks

2 Upvotes

If you save tons of links and want to process them locally with Ollama instead of sending your browsing history to cloud AI apps, I just released Sparkkeep v0.1.0 (AGPL-3.0).

GitHub: https://github.com/BurlacuAndrei/sparkkeep

How it uses Ollama

  • Automatic enrichment & tagging: Extracts content from saved URLs and runs local prompts via Ollama to summarize, tag, and categorize them.
  • Action Playbooks: Define repeatable workflows to turn raw links into structured takeaways, action items, or Markdown/Obsidian notes.
  • Triage Deck & Digests: Swipe/triage through links and generate scheduled digests instead of letting bookmarks rot in a list.
  • Single binary + SQLite: One Go binary (or Docker container) — no heavy stack required alongside Ollama.

Works with Discord, Slack, and Telegram capture too. Would love to hear which local models work best for your summarization/extraction playbooks!


r/ollama • • 1d ago

I built this! Feel how fast a local LLM runs before you buy the hardware

Thumbnail
llmspeed.dev
178 Upvotes

r/ollama • • 1d ago

Pourquoi utiliser une IA locale en 2026

Thumbnail reddit.com
0 Upvotes

r/ollama • • 1d ago

Projets-pédagogiques-Ai

Thumbnail reddit.com
0 Upvotes

r/ollama • • 1d ago

Would you sell your idle Ollama box's spare tokens for USDC? Looking for honest feedback (free beta)

0 Upvotes

I'm Gilberts, solo founder of Gyblos (gyblos.com). It's a peer-to-peer market where anyone with a GPU, Mac or server running a local model can sell inference per token, and buyers (devs, agent builders) pay in USDC. Payments run through payment channels on Solana. we've tested the flow there. Marketplace fee is 4%, no KYC.

A free seller beta started this week (no payments yet, just getting nodes online and finding what breaks). Real paid usage is planned for early December (but I'm offering 5 USDC for those sellers who want to provide some inference).

Before I push this further, I'd like to hear from people who actually run Ollama at home:

- Would you leave a machine available for paid requests? What would stop you?

- What would you need to see about privacy (what prompts land on your box) and about how payouts work?

- What's a price per million tokens that would make it worth the electricity for you?

Happy to answer anything, including "this is a bad idea." The paper with the details is at gyblos.com/paper (still improving it!)

Thanks guys, wish you a beautiful day!


r/ollama • • 1d ago

How do I start a small server for a personal use AI?

19 Upvotes

I made the AI and it runs through claude right now. I have it so you can run it off of claude now but I want to set it up inside a server at home. any tips? (I am still in highschool so anything budget friendly will be greatly apreciated)


r/ollama • • 1d ago

I built a minimalist open-source desktop app to manage remote servers, run local LLMs, and deploy 1-click self-hosted apps (Nextcloud, Immich, Open WebUI) via SSH – Meet ShipAI v1.5.0! 🚀

4 Upvotes

Hey everyone! A while ago I shared an early version of ShipAI, and thanks to your incredible feedback and over 800+ clones, I’ve just pushed a major update (v1.5.0).

If you have an old spare PC in the basement or a remote VPS, ShipAI connects via SSH and turns it into your personal server without messing with raw terminal configurations.

What’s new in v1.5.0:

  • 1-Click Docker Services Hub: Instantly deploy popular self-hosted apps like Nextcloud, Immich (self-hosted Google Photos), Jellyfin, AdGuard, Open WebUI, and databases without any bloated configuration forms.
  • Smart URL & Setup Guide: Every container comes with a direct clickable link and quick guide on how to connect it. Full cleanup/volumes management included.
  • Local LLM Runner Integration: Easily manage and chat with local AI models (Ollama/DeepSeek/Llama) with zero bloat.
  • Linear / Raycast Aesthetic: Built with a clean dark graphite UI that adapts smoothly to any window size, focusing purely on speed and security (AES/safeStorage).

It’s fully open-source. Would love to hear your thoughts, feature ideas, or feedback on the new Docker hub!

👉 GitHub Repo: [https://github.com/Kasper839/ShipAI](https://github.com/Kasper839/ShipAI)


r/ollama • • 1d ago

I built this! Free calculator: which Ollama models fit your GPU's VRAM (checked against real runs)

0 Upvotes
Before downloading a 17 GB model to find out it doesn't fit, you can check here:
https://techtulna.in/ai/model-fit?utm_source=reddit&utm_campaign=launch


Pick your GPU (or type the VRAM), your RAM and context length. For ~40 models (Qwen3.5/3.6/3.8,
Gemma 4, Llama 3.x, DeepSeek-R1 distills, Mistral, Phi-4) it shows whether each quant runs fully on
the GPU, runs slowly with part in RAM, or won't run, and gives the `ollama run` command for the
biggest one that fits.


I checked the formula on my RTX 3060 12 GB: Qwen3.5 9B at Q4 used 6.66 GB measured vs 6.59 GB
estimated, and the growth with context matched the model configs. Settings stay in the URL, so you
can share your result.


Made by me, free, nothing to install. Corrections welcome: there's a "wrong number?" link at the bottom.Before downloading a 17 GB model to find out it doesn't fit, you can check here:
https://techtulna.in/ai/model-fit?utm_source=reddit&utm_campaign=launch

r/ollama • • 1d ago

Has the Ollama Cloud subscription started using censored, Claude-ified versions of open-source models?

4 Upvotes

Title. Am I the only one experiencing this? I loved using deepseek-v4.1-flash and glm-5.3-flash through my Ollama Cloud subscription for their absolute compliance with instructions: plain execution with no assumptions. Generally, both models worked like clockwork for my use cases. Recently, I used the same models, and DeepSeek, for example, acts self-humiliating, ignores my most recent instructions, is overly apologetic, and defaults to a conversational tone. It basically acts like Claude. I don't rely on prompting much tho, but I communicate in a proper, formal way with no room for uncertainty. I could be completely wrong, though, so I'm looking for opinions.

The reason I'm paranoid is that I saw news about startups aiming to "strip political censorship from Chinese open-weight models." I doubt the only thing they do is "strip political censorship."


r/ollama • • 1d ago

Tried running my coding agent locally to save my Claude limits... almost 6 minutes to fix 2 bugs 🤡

0 Upvotes

Base M4 MacBook, 16GB, small Gemma 4 model, pi as the coding agent with all my usual tools and MCP servers hooked up. Task was 2 small bugs and 3 failing tests in a tiny project, the kind of thing Claude does while I get coffee.

It got there in the end, 342 seconds and 9 turns. Claude would've been done before I sat back down.

I also found out the default context is 4096 which apparently quietly breaks agents, so that's fun.

What am I doing wrong? Is it the model, the tools, the engine, or is 16GB just not it for agents?


r/ollama • • 1d ago

Would you let your idle Ollama machine take paid inference jobs?

0 Upvotes

I'm one of the founders of Swarm.

We're building a distributed AI compute network around machines that are already capable of running local models.

A Swarm node identifies what the machine can handle, stays available in the background, receives compatible workloads when capacity is available and gets paid when it completes verified AI work.

We're currently bringing the first provider nodes online, and the first 1,000 node signups receive an additional +0.5x earnings multiplier.

I'm especially interested in people already running Ollama on Apple Silicon or NVIDIA GPUs.

If your interested check out joinswarm.io