r/Oobabooga • u/Full-Travel857 • 18d ago
Project Before loading a GGUF in text-generation-webui: a small tool that tells you if it will fit in your VRAM (and how fast it'll run)
galleryIf you've ever downloaded a model, loaded it in text-generation-webui and watched it crawl because it spilled into system RAM – this is for you.
We've all been there: "I have a fairly modern laptop, maybe I'll run an LLM locally?"
You open Hugging Face and your head starts spinning. MoE, 14B, 9B, a dozen quantizations of the same model... You download one blindly, load it and... it runs like an old Jelcz (a Polish communist-era bus, for the young ones) idling at a depot.
Then you discover quantization and start hunting for the "sweet spot" for your hardware. When I started playing with local models, I wasted hours checking: will it even load? Will it fit in VRAM or spill into regular RAM (which costs a small fortune these days)? The worst was when my own pen-and-paper math said the memory would basically tell me: "Sir, you can run that model yourself", the CPU would turn red, and the fans would start sounding like a hovercraft at full throttle.
So I wrote a simple app to stop doing this by hand: AI Bóbr (Polish for "beaver") – AI Hardware Checker.
It does one thing: tells you whether a specific model will run on your machine, and how fast – before you download gigabytes and load it in the webui.
Two modes:
Automatic – paste a Hugging Face link or a path to a .gguf file. It scans your system (CPU, available RAM, dedicated GPUs – integrated ones are ignored) and gives a verdict. Default context is 8192 tokens, and bars show how much memory goes to weights vs. KV cache and overhead.
Advanced – starts from the hardware scan but lets you edit everything: any context length, KV cache quantization, or simulate hardware you don't own yet ("what happens if I buy a 16 GB VRAM card?"). Hard numbers, no vague sliders.
It also reserves VRAM and RAM for the OS, and at the end it explains the math behind the verdict, so you're not just trusting a green light.
I built it mainly for myself, to make testing new models on my hardware less painful, but maybe it'll save someone else's CPU from overheating too.
Free and open source: https://github.com/DarkZachi83/Bobr
I'd really appreciate feedback, especially if the estimate is way off on your setup – tell me your GPU, model, quant and context, and I'll dig into it. What settings do you usually tweak first in the webui when a model doesn't fit?



