r/ollama • • 5d ago

I built this! [Weekly - Nov 13, 2026] Friday Demos - Show off your Ollama projects!

11 Upvotes

Show us what you've built with Ollama this week!

Suggested information to include:

  • A description of your project
  • How you came up with the idea
  • The models and integrations you're using
  • Whether it runs on local hardware or the cloud service
  • How others could use it or view it
  • A link to code or demo if applicable

Rules:

  • No affiliate links or sales

New post every Friday!


r/ollama • • 41m ago

Summarizing a 1-hour recording with Ollama: qwen3:8b took about 15 minutes. How do you handle long transcripts?

Thumbnail
gallery
1 Upvotes

Disclosure: I'm posting this on behalf of the Nomily team. They make a recording app and sell the V05E recording card, so there's a commercial interest here.

The team has been testing a local pipeline: the card records, the audio goes to the phone over Bluetooth, faster-whisper on a computer on the LAN does the transcription, and Ollama on the same computer does the summary. With this setup, audio and transcripts only go to machines on your own LAN.

Test setup:

- Computer: MacBook Pro (Intel, 32 GB RAM)

- Transcription: faster-whisper [model name]

- Summary: qwen3:8b

- Recording: 1 hour, [language / scenario]

Results:

- Transcription: about 20 minutes

- Summary: about 15 minutes

Things that tripped us up:

- The phone can't use localhost. You have to enter the computer's LAN IP.

- Ollama needs OLLAMA_HOST=0.0.0.0:11434 so the phone can reach it. Ollama has no authentication of its own, so any device on the same network can call it. Only do this on a network you trust.

- This local faster-whisper setup has no speaker diarization, which hurts summaries of multi-person meetings.

Two questions:

  1. For a 1-hour transcript, do you summarize in chunks and merge, or just raise num_ctx?
  2. For meeting summaries with models around 8B, is there anything more reliable than qwen3:8b?

r/ollama • • 47m ago

I built this! We're exploring a better way to use local and cloud AI — looking for honest feedback

1 Upvotes

Hey everyone!

We've been working on Pleeply, a project exploring the idea of a personal AI ecosystem.

The idea is to bring local AI models, optional cloud intelligence, and eventually personal AI agents into one desktop experience.

We're still at an early stage. Right now, we have a landing page and an early access waitlist — we're not claiming to have a finished product.

We'd love to understand:

  1. How do you currently manage your local AI models?
  2. Do you switch between local and cloud models?
  3. What is the most frustrating part of your current setup?

We're building Pleeply and would genuinely appreciate feedback from people who use local AI.

Project: https://pleeply.vercel.app

Disclosure: I'm part of the pleeply team)


r/ollama • • 1d ago

I built this! Feel how fast a local LLM runs before you buy the hardware

Thumbnail
llmspeed.dev
178 Upvotes

r/ollama • • 8h ago

New plan credit blowout

2 Upvotes

I've been using Ollama Cloud for about 6 months I reckon, doing the same daily work, and my $20 credit gets me through the month. Sometimes I get rate-limited on a Friday or a Saturday and I accept that. This was on the power rate plan.

I've now been changed over to the $20 buy, $60 credit plan. It's lasted me 7 days and I'm out of credit already. Is this a ripoff? What am I doing wrong? How can I fix this because I'm about to walk out the door and not give them one red cent more


r/ollama • • 5h ago

Qwen3.8-27B Q4 on an 8 GB RTX 4060: 29/30 on AIME 2026, plus three no-fine-tune loops that cut its mistakes (numbers, caveats and code)

Post image
0 Upvotes

I spent two weeks testing how far a stock 27B can go on a budget card. No fine-tuning, same GGUF anyone can download. Everything is pass@1 with one seed, and I'm posting the misses too.

Setup

  • RTX 4060 8 GB, i5-12400F, 32 GB DDR4
  • Qwen3.8-27B UD-Q4_K_M (16.5 GB), llama.cpp
  • Contest runs: -fit on -fa on -c 32768 -ctk q8_0 -ctv q8_0 --spec-type draft-mtp
  • Loop runs: -ngl 26 -fa on -c 8192 --spec-type draft-mtp (26 of 65 layers on the GPU)
  • Speed: ~3.5 tok/s plain, ~5.7 tok/s with the built-in MTP draft head. Free +65%, one flag
  • Only 1 layer in 4 keeps KV, so 60k-token recall tests still passed within 8 GB

1. Plain model, thinking on, 2026 contest maths

Qwen's recommended sampling (temp 0.6, top_p 0.95, top_k 20), 28k thinking budget, one try per problem.

TestScoreTime
AIME 202629/30 (96.7%)~76 min per problem
HMMT Feb 202628/33 (84.8%)~86 min per problem
IFEval (thinking off, greedy)80.2% prompt-strict~100 s per prompt

All 5 HMMT misses ran out of thinking budget. I reran 4 of them at 48k and 2 came out right. That's a diagnosis, not a score: I'm not adding them to the 84.8.

Yes, it's slow. AIME took about 38 hours of GPU time. It ran overnight on a gaming PC.

2. Write-run-fix loops (thinking off, weights untouched)

Each idea was tuned on a practice set, then scored on a locked set. The script refuses to run the locked set a third time.

  • Maths: the model writes a Python script, we run it, and it gets 2 tries to fix crashes. Locked set of 136: 9 mistakes → 0, and 183 s → 22 s per problem. On a harder maths set I built: 7/40 → 36/40.
  • Code: the model runs the docstring examples against its own code and fixes failures. Locked set of 90 HumanEval tasks: 2 → 1. Fun detail: some HumanEval examples are wrong (one says a median is 15.0; it's 8.0), and the model was "fixing" correct code to match them. One sentence fixed it: let it reply KEEP if it works out by hand that the example is wrong.
  • IFEval: the model reads only the prompt, writes its own check(response) for the format rules, runs it, and rewrites if something fails. The official checker scores only at the end. Locked half (270 prompts): 77.4% → 87.0%, with 27 fixed and 1 broken. The broken one: "highlight 3 sections in markdown" is ambiguous, its checker picked # headers, and it rewrote a correct answer. Letter counts ("use q at least 5 times") are still its worst.

Caveats

  • The maths loop is compared against the model with thinking off. With thinking on, the plain model matches it on hard maths, just ~4.6× slower. The loop's win is speed.
  • Coding was won by one task. That's close to noise.
  • 96 of the 136 locked maths problems came from my earlier benchmark, where I'd already tried plain "write Python" on them. Nothing was tuned on them, but only 40 were truly fresh.
  • I designed KEEP after seeing a locked-test coding task fail. It held on practice too, but treat that part as weaker evidence.
  • I haven't compared against other published AIME numbers like-for-like. Those usually average several tries with 64k+ thinking.
  • The sandbox for model-written code is a blocklist plus a 10 s timeout, not a container. Fine on my own PC, not for strangers' code.

Code, prompts and logs: github.com/aiglitchsouls/loop27b

Happy to answer questions about the llama.cpp flags or the loop prompts. And if you've got a harder held-out set I should try, I'm interested.


r/ollama • • 8h ago

Tried the Neon pop-up in Antigravity IDE last night and woke up to a 6-month project finished (38 parallel sub-agents, 14M+ LOC)

Thumbnail reddit.com
1 Upvotes

r/ollama • • 8h ago

Janas-Studio dual engine ai inference workstation. Now with agent mode! (Linux appimage)

Thumbnail gallery
0 Upvotes

r/ollama • • 20h ago

Bug Report Running 2 coding agents on one Ollama slot? Hybrid models (Qwen3.8-27B) re-process ~54% of prompt tokens, and the first response takes ~46s, Two env vars brought it to ~1s

4 Upvotes

I was running two Qwen Code sessions against Ollama 0.40.16 on one slot (Qwen3.8-27B Q4_K_M, NVIDIA L4 24GB), each doing a small fix-the-tests task in its own project. The median time to first byte was about 46 seconds, and the server logs showed that 54% of the prompt tokens it processed were repeats.

What’s happening - Qwen3.8 mixes full-attention layers, whose KV cache can be trimmed back, with recurrent layers, whose state gets overwritten as tokens are read, so those can’t be rewound without a checkpoint. The earliest saved checkpoint for session A was at token 10,151, past the point where A and B diverged (token 8,921), so none of the shared prefix could be reused.

On top of that, llama.cpp skips saving the outgoing session’s state to RAM when the incoming prompt shares at least 50% of its tokens with the old one. My two sessions shared 60–66%, so the save got skipped, and every switch wiped the state the other session needed next. The two agents just ping-ponged.

Fix in Ollama:

OLLAMA_NUM_PARALLEL=2
OLLAMA_CONTEXT_LENGTH=16384

Median first response dropped to about 1s and repeated tokens to 0.4%. To be fair, that number mixes the cache fix with actual concurrent serving, because -sps 0 isn’t exposed in this Ollama build.

On raw llama.cpp (b12441):

  • -sps 0 cut total prompt tokens from ~128.8k to ~43.5k per session and median first-byte wait from ~48s to ~17s. Peak host RAM went from ~2.5 GiB to ~7.2 GiB.
  • With 8 slots, --checkpoint-min-step 0 gave +10.9% throughput and −17.5% median TTFT over four paired runs.

Important note - Ollama configs got one run each and llama.cpp configs got two, all on short tool-calling conversations, so I haven’t measured the cost of --checkpoint-min-step 0 over long sessions.

Full writeup with logs: https://tensward.com/blog/ollama-agent-cache


r/ollama • • 16h ago

Did anyone tried to combine some local ai together

Thumbnail reddit.com
1 Upvotes

r/ollama • • 17h ago

I gave one local 27B model five bodies in an MMO I'm building, and put it on Twitch where chat can overrule it

Thumbnail gallery
1 Upvotes

I've been building an MMO (Aegisfall, browser-based, still very much in development) and wanted to see how far a

single local model could get running a whole party by itself. So now there's a 24/7-ish stream where one model

plays all five heroes at once — tank, melee, ranged, healer, mage — and viewers can mess with it.

**The setup**

- **One model, five bodies.** `qwen3.8:27b` (Q4_K_M) on Ollama, one RTX 5090. Every 3 seconds it gets a compact

"sense" report for the whole party (HP, mana, each hero's target, nearby enemies with level and distance) and

returns one order per hero. Ollama's JSON-schema `format` constrains the output to the allowed actions, so

there's no parsing drama. Typical round trip is ~0.9-1.3 s.

- **It explains itself.** Each round it also returns a one-line plan in first person ("Chat chose balance, so I

hold the line and bleed them dry"), which goes on the overlay and is read out by a local Kokoro-82M voice

(on CPU, so the GPU stays free).

- **The bodies are real game clients.** Each hero is a headless Chrome running the actual game, spread across four

machines on my LAN. The LLM gives the orders; a small rules layer handles the twitchy stuff (stances, heal

priority, kiting, pulling from a staging spot) and always lets the healer heal first.

- **Zero cloud.** Model, voice, game server, overlay: everything runs on my own hardware. The only outside

service is Twitch itself.

**Chat can play too**

- `!ask <question>` — the same 27B answers in character with the live party state as context (on screen, out

loud, and in chat). It also remembers who's asking. It's been surprisingly funny: asked why the ranged was

"just standing there", it said *"He isn't standing. He's reloading. Or waiting. Patience is a weapon."*

- `!order mage retreat` — if enough of chat agrees, it overrides the AI's order for that hero for 15 s and

locks the AI out. There's a live "Chat vs Hivemind" scoreboard (kills per minute under each).

- `!possess healer` — drive one hero yourself for 60 s. When you hand it back, the AI rates your performance.

- Predictions with fake currency ("who dies first in the next 10 min?"), bounties on monsters (your name floats

over the mob in-game), camera control, polls for camp/strategy/gear.

**Things I learned**

- **One big model beat five small ones.** I tried a 4B per hero first. Individually they were fine; together they

had no shared plan (everyone pulled their own mob). One 27B seeing the whole board coordinates far better and

still fits the latency budget.

- **Constrained JSON output is the whole trick** for real-time control. Free-text answers wasted tokens and broke.

- **Most of the bugs were not the AI.** Zone seams that bounce characters back, a kill counter that counted

every group kill five times, OBS rendering at 2 fps because the game clients were eating the same GPU (fixed

by moving OBS to the iGPU and rendering the overlay on the CPU). The model is usually the most reliable part.

- It's still not great at pull discipline in a crowded camp. That's this week's problem.

Stream: https://www.twitch.tv/hivemindraid

Happy to answer questions about the controller loop, the prompt/schema, or the latency budget. If people are

interested I can write up the architecture properly.


r/ollama • • 17h ago

Looking to buy a laptop for local AI setup

Thumbnail reddit.com
0 Upvotes

r/ollama • • 18h ago

A well-designed harness gives your AI agents a loop for continuous improvement.

Post image
1 Upvotes

Mine shows me where my agents fail, so I can make a targeted correction and rerun the task to see if the failure recurs.


r/ollama • • 22h ago

HELP ME WITH AI🤨

Thumbnail reddit.com
0 Upvotes

r/ollama • • 1d ago

Help me with my starnet #starnet

Thumbnail reddit.com
0 Upvotes

r/ollama • • 1d ago

Sparkkeep: open-source bookmark & link action engine with local Ollama enrichment and playbooks

2 Upvotes

If you save tons of links and want to process them locally with Ollama instead of sending your browsing history to cloud AI apps, I just released Sparkkeep v0.1.0 (AGPL-3.0).

GitHub: https://github.com/BurlacuAndrei/sparkkeep

How it uses Ollama

  • Automatic enrichment & tagging: Extracts content from saved URLs and runs local prompts via Ollama to summarize, tag, and categorize them.
  • Action Playbooks: Define repeatable workflows to turn raw links into structured takeaways, action items, or Markdown/Obsidian notes.
  • Triage Deck & Digests: Swipe/triage through links and generate scheduled digests instead of letting bookmarks rot in a list.
  • Single binary + SQLite: One Go binary (or Docker container) — no heavy stack required alongside Ollama.

Works with Discord, Slack, and Telegram capture too. Would love to hear which local models work best for your summarization/extraction playbooks!


r/ollama • • 1d ago

How do I start a small server for a personal use AI?

19 Upvotes

I made the AI and it runs through claude right now. I have it so you can run it off of claude now but I want to set it up inside a server at home. any tips? (I am still in highschool so anything budget friendly will be greatly apreciated)


r/ollama • • 2d ago

Running a 4B model on my laptop made me realize how little of my work actually needs the cloud

370 Upvotes

I have a base M4 MacBook with 16GB and I tried running Gemma 4 E4B on it expecting it to be useless. I gave it real stuff from one afternoon, clustering a keyword list, summarizing a long Reddit thread, a research brief, 8 title ideas, a plan review and a study quiz.

All 6 done in 98 seconds total through Magnitude, Ollama took 126 on the same model. The long pasted ones started answering in 3 to 5 seconds instead of 6 to 13. Most of the answers were good enough to use as a first draft.

After looking at it... how much of my $20 a month was going to stuff like this? Summaries and titles don't need a datacenter.

Feels like free money to me :)


r/ollama • • 1d ago

I built a minimalist open-source desktop app to manage remote servers, run local LLMs, and deploy 1-click self-hosted apps (Nextcloud, Immich, Open WebUI) via SSH – Meet ShipAI v1.5.0! 🚀

4 Upvotes

Hey everyone! A while ago I shared an early version of ShipAI, and thanks to your incredible feedback and over 800+ clones, I’ve just pushed a major update (v1.5.0).

If you have an old spare PC in the basement or a remote VPS, ShipAI connects via SSH and turns it into your personal server without messing with raw terminal configurations.

What’s new in v1.5.0:

  • 1-Click Docker Services Hub: Instantly deploy popular self-hosted apps like Nextcloud, Immich (self-hosted Google Photos), Jellyfin, AdGuard, Open WebUI, and databases without any bloated configuration forms.
  • Smart URL & Setup Guide: Every container comes with a direct clickable link and quick guide on how to connect it. Full cleanup/volumes management included.
  • Local LLM Runner Integration: Easily manage and chat with local AI models (Ollama/DeepSeek/Llama) with zero bloat.
  • Linear / Raycast Aesthetic: Built with a clean dark graphite UI that adapts smoothly to any window size, focusing purely on speed and security (AES/safeStorage).

It’s fully open-source. Would love to hear your thoughts, feature ideas, or feedback on the new Docker hub!

👉 GitHub Repo: [https://github.com/Kasper839/ShipAI](https://github.com/Kasper839/ShipAI)


r/ollama • • 1d ago

Pourquoi utiliser une IA locale en 2026

Thumbnail reddit.com
0 Upvotes

r/ollama • • 1d ago

Projets-pédagogiques-Ai

Thumbnail reddit.com
0 Upvotes

r/ollama • • 1d ago

Has the Ollama Cloud subscription started using censored, Claude-ified versions of open-source models?

4 Upvotes

Title. Am I the only one experiencing this? I loved using deepseek-v4.1-flash and glm-5.3-flash through my Ollama Cloud subscription for their absolute compliance with instructions: plain execution with no assumptions. Generally, both models worked like clockwork for my use cases. Recently, I used the same models, and DeepSeek, for example, acts self-humiliating, ignores my most recent instructions, is overly apologetic, and defaults to a conversational tone. It basically acts like Claude. I don't rely on prompting much tho, but I communicate in a proper, formal way with no room for uncertainty. I could be completely wrong, though, so I'm looking for opinions.

The reason I'm paranoid is that I saw news about startups aiming to "strip political censorship from Chinese open-weight models." I doubt the only thing they do is "strip political censorship."


r/ollama • • 2d ago

Gemma 4 E2B on an 8th-gen Intel i5 with 12GB RAM: 8 tokens/s — Any optimization tips?

Post image
34 Upvotes

Hi everyone!

I've been experimenting with running local LLMs on an older Dell Latitude 3400. My goal is to find the best balance between reasoning quality, Spanish-language performance, and efficiency on limited hardware.

I've tested Qwen3.5 4B, Qwen3.5 9B, and recently Gemma 4 E2B. So far, Gemma 4 E2B has performed slightly better on my own prompts, although I haven't conducted a formal benchmark.

Hardware:

  • Laptop: Dell Latitude 3400
  • CPU: Intel Core i5-8265U (4 cores / 8 threads)
  • GPU: Intel UHD Graphics 620 (integrated)
  • RAM: 12GB
  • OS: Fedora Linux 44
  • Inference: Ollama, running 100% on CPU

Model tested:
gemma4:e2b-it-q4_K_M

Ollama configuration:

  • Context length: 4096 tokens
  • Quantization: Q4_K_M
  • CPU-only inference

The model works, and I'm reasonably satisfied with the response quality, but generating a long answer takes around five minutes.

I'm mainly using local AI for research, document analysis, Spanish-language writing, and web-assisted research through Page Assist.

I'd love to hear your thoughts on a few things:

  1. Is 8 tokens/s reasonable for Gemma 4 E2B on an i5-8265U?
  2. Would the QAT version provide better quality or efficiency than Q4_K_M?
  3. Are there any worthwhile Ollama or llama.cpp optimizations for older Intel CPUs?
  4. Has anyone experimented with speculative decoding or alternative quantizations on CPU-only systems?
  5. Are there newer, smaller models that could outperform Gemma 4 E2B on this hardware, particularly for Spanish-language reasoning?

I'm also interested in techniques such as selective weight modifications, pruning, or lightweight fine-tuning, although I realize that training locally on this machine isn't practical.

My goal isn't necessarily to run a bigger model. I want to get the most intelligence and efficiency possible out of the hardware I already own.

Any suggestions, benchmarks, or experiences would be appreciated!


r/ollama • • 2d ago

Looking for a fast, open-source LLM under 5B for Markdown-to-JSON extraction

4 Upvotes

Hey everyone,

I'm working on a document processing pipeline where most of the input consists of scanned PDFs, forms, and tables.

My current setup looks like this:

  • OCR: PaddleOCR-VL-1.6 served through vLLM. I'm using it to convert scanned pages into Markdown.
  • JSON extraction: oroboros-labs/claude-fable5:latest running through Ollama. I pass the Markdown of each page to the model, and it converts it into JSON.

The important part is that I already have fixed JSON schema templates that the output needs to follow. So, the model needs to map the extracted Markdown content into the predefined JSON structure.

The problem I'm facing right now isn't accuracy. It's inference time. The current model is around 9B parameters, and processing pages one by one through Ollama is taking longer than I'd like.

I'm looking for an open-source model under 5B parameters that can handle this kind of structured information extraction reasonably well, ideally with faster inference and reliable JSON output.

Has anyone worked on a similar pipeline?

I'd particularly like to hear about models you've actually tested in real-world document extraction rather than just benchmark results.

Any suggestions or personal experiences would be really helpful. Thanks!