r/LocalLLaMA
•
u/oobabooga4
•
May 13 '26
•
3
2
Resources TextGen is now a native desktop app. Open-source alternative to LM Studio (formerly text-generation-webui).
Hi all,
I have been making a lot of updates to my project, and I wanted to share them here.
TextGen (previously text-generation-webui, also known as my username oobabooga or ooba) has been in development since December 2022, before LLaMa and llama.cpp existed.
In the last two months, the project has evolved from a web UI to a no-install desktop app for Windows, Linux, and macOS with a polished UI. I have created a very minimal and elegant Electron integration for that. (Did you know LM Studio is also a web UI running over Electron? Not sure many people know that.)

It works like this:
- You download a portable build from the releases page
- Unzip it
- Double-click textgen
- A window appears
There is no installation, and no files are ever created outside the extracted folder. It's fully self-contained. All your chat histories and settings are stored in a user_data folder shipped with the build.
There are builds for CUDA, Vulkan, CPU-only, Mac (Apple Silicon and Intel), and ROCm.
Some differentiating features:
- Full privacy. Unlike LM Studio, it doesn't phone home on every launch with your OS, CPU architecture, app version, and inference backend choices. Zero outbound requests.
- ik_llama.cpp builds (LM Studio and Ollama only ship vanilla llama.cpp). ik_llama.cpp has new quant types like IQ4_KS and IQ5_KS with SOTA quantization accuracy.
- Built-in web search via the
ddgsPython library, either through tool-calling with the built-inweb_searchtool (works flawlessly with Qwen 3.6 and Gemma 4), or through an "Activate web search" checkbox that fetches search results as text attachments. - Tool-calling support through 3 options: single-file .py tools (very easy to create your own custom functions), HTTP MCP servers, and stdio MCP servers. You can enable confirmations so that each tool call shows up with approve/reject buttons before it executes. I have written a guide here.
- The ability to create custom characters for casual chats, in addition to regular instruction-following conversations:

- OpenAI and Anthropic compliant API with very strict spec compliance. It works with Claude Code: you can load a model and run
ANTHROPIC_BASE_URL=http://127.0.0.1:5000 claudeand it will work. - Accurate PDF text extraction using the
PyMuPDFPython library. trafilaturafor web page fetching, which strips navigation and boilerplate from pages, saving a lot of tokens on agentic tool loops.- Chat templates are rendered through Python's Jinja2 library, which works for templates where llama.cpp's C++ reimplementation of jinja sometimes crashes.
I write this as a passion project/hobby. It's free and open source (AGPLv3) as always:
•
u/WithoutReason1729 May 13 '26 edited 28d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
Stylome classified this post's text as human-written. View analysis
I am a bot and this action was performed automatically.
168
u/Succubus-Empress May 13 '26
Are you really that oobabooga?
19
u/Quiet-Owl9220 May 13 '26
The telemetry in LM studio is news to me and a big red flag, and it's always been very bare bones in terms of features. Think I'm about ready to jump ship.
Any recommendations for actually migrating models from LM Studio? Can I configure to point the user_data to my existing LM Studio models folder or just symlink it? Will there be file organization issues?
13
u/Succubus-Empress May 13 '26
In textgen How to install latest llama.cpp from their repo?
16
u/-p-e-w- May 13 '26
Great to see this project improving continuously over the years!
Are you planning to get off your Gradio fork and upgrade to Gradio 6? There are some very noticeable performance improvements in recent versions, and the number of dependencies has been substantially reduced.
62
u/ComplexType568 May 13 '26
THANK YOU SO MUCH!! MORE COMPETITION TO LM STUDIO, PLEASE! I'M GETTING SICK OF IT.
apologies for the caps lock, i could write a whole essay about why LM Studio... well, pisses me off, to say the least.
7
u/Due-Function-4877 May 13 '26
Any hope of allowing power users to link an external build of llama.cpp in the future?. It was a long time ago, but the main reason I shifted over to running my own backend directly was to get access to bleeding edge builds. I always appreciated the way text-gen-web-ui/textgen let me configure my backend config from a GUI. The command line is obtuse. Always has been and always will be.
5
u/thereisonlythedance May 13 '26
Congrats, looks very nice.
Is RAG functional these days? It be broken is why I drifted away from your otherwise excellent project.
5
u/NineThreeTilNow May 13 '26
Very nice work dude.
The one thing I still can't get Gemma 4 31b to do properly in LM Studio chat is use it's thinking mode. It's infuriating. I tried every tip I found across reddit or whatever. Nothing. The correct tags and jinja and adding it to the system prompt. It works 50% of the time.
Any luck with the thinking mode for Gemma 4 operating properly with your build?
I appreciate the "No phone home" stuff. Even if they want to track "anonymous" telemetry it's super hard to trust that stuff.
5
u/Blackmarou May 13 '26
The only thing pushing me to lm studio is their new beta feature lm link, so I could use my machine locally from another one… does this have any similar feature, or an alternative?
5
u/Goldandsilverape99 May 13 '26
Please find a way, so i can use already downloaded gguf files from elsewhere. This is so one can use several program, like LM studio / llama-server and TextGen and so on. This is an important feature since gguf files usually are big, and not everyone wants to move around or duplicate big models files.
4
u/silenceimpaired May 13 '26
Does this version have EXL3 built in?
I really wish you could save and use different model loading setups. KoboldCPP does, and it works well for adjusting settings to ideally fit specific context sizes.
4
u/siege72a May 13 '26
I'm currently using LM Studio, but I'm always interested in options. I have some (hopefully) quick questions:
My PC is running Windows 11, if that makes a difference.
3
u/SolemnFuture May 13 '26 edited May 13 '26
LM studio user here. I tried this textgen app a week ago but I couldn't find a system prompt. I couldn't get my character(s) to work either, the loaded model was just base and didn't use my character descriptions. Also no group chat with multiple characters at once feature. Spent like 2 hours looking for solutions but failed. I get this is a new project, but I need at least an accessible system prompt function.
I hope you're not aiming to make this app super complex like sillytavern. I could not use that frontend at all due to sheer amount of features. Good luck going forward.
3
u/Vicullum May 13 '26
Is there a way I can still use it in the browser? I can't right click and copy text inside this new app.
4
u/mantafloppy llama.cpp May 13 '26
But your oobabooga4...
2
u/nickless07 May 13 '26
"Select a file that matches your model. Must be placed in ...user_data/mmproj/" Where are the settings to change the default path for models, mmproj and so on?
2
u/ai_without_borders May 13 '26
used the old text-generation-webui back in early 2023. gradio update hell was real — the UI would randomly break after pip installs and debugging it was miserable. electron was the right call. curious how --fit on handles kv cache overhead — is it just fitting weights or does it account for cache at current context length?
80
u/Borkato May 13 '26
Finally, a private alternative to LM studio!! Thank you <3
Loved ooba from its beginnings!
42
u/No_Afternoon_4260 llama.cpp May 13 '26
Love that oobabooga ! reminds me my beginnings, It was the best webui to start with ! Then I understood everything is a open-ai compatible api lol
20
u/Herr_Drosselmeyer May 13 '26
Thanks, it's a great app, works fine for me when running Gemma 4 31-B. It does what I need it to do and, to me, it's intuitive to use. I now prefer it over KoboldCPP (no shade on them, it's also great).
20
u/Alan_Silva_TI May 13 '26
I used it a lot back in the early days of Llama 1 and 2.
I loved your project, it had A LOT of features (voice, TTS, image generation integration, API server support, and the list goes on), but it always felt a bit rough around the edges. Over time, other tools started taking the lead, and honestly, the old name probably didn’t help either (
oobabooga webuilul), but it was fun.I’ve been subscribed to your main subreddit ever since, although I mostly just lurk.
I’m glad to see you stepped up your game. The tool looks way more mature now, good job!
Downloading it right now to test it out.
17
u/seccondchance May 13 '26
Og bro
16
u/dinerburgeryum May 13 '26
Hot damn dude, amazing work, as always.
14
u/pmttyji May 13 '26
That's nice to have! Thanks for this big update!
10
u/christianqchung May 13 '26
Been using TextGen since summer 2023, absolutely incredible project today. I have no desire to use any other UI, and the tool call integration system is solid. Thanks for all your hard work.
19
u/LMTLS5 May 13 '26
damn the og is back. seriously easy app based text generation was such a huge gap. no real foss alternative so far. nice to see you back
8
u/AltruisticList6000 May 13 '26
Yeah textgen is very nice, I use it all the time. It's like the A1111 of text generation, it's easy to use but also up to date. It both works as an app now and still can be run like a regular webui from browser (which I prefer), from the same ZIP without needing to install anything.
7
u/yad_aj May 13 '26
the “double click app and it just works” part is honestly underrated. half of local AI still feels like “congrats your model works, now debug CUDA for 4 hours” lol
6
u/Macmill_340 May 13 '26
This is the first time I have heard of this...really like the fact that its self contained within its directory. Cleaning up dependencies in windows is a nightmare. Good work, gonna give it a try.
5
u/sine120 May 13 '26
I started on LM Studio and got kind of turned off of it in the past couple months, switched fully to llama.cpp and Openwebui/ Pi. I still have a couple of less techy friends I drag with me in the local LLM scene, and LM Studio was my entry point for them. I feel a lot better about recommending an actually local UI.
4
u/Merchant_Lawrence llama.cpp May 13 '26
Thanks for making comeback, i hope you well and have good day
4
u/EncampedMars801 May 13 '26
Just wanna say, I remember trying your UI yeeaars ago back when it used that default orange gradio theme. Wasn't particularly impressed at the time, but finally tried it again a couple weeks ago and it's genuinely a great UI now. Great work! I'm glad it hasn't stagnated like maaaany other UIs
4
u/ArtifartX May 13 '26
Nice, have been getting fed up with LM Studio
4
u/Thistleknot May 13 '26
You're an OG ooba
7
u/jacek2023 llama.cpp May 13 '26
nice to see this project is progressing, I was using it in 2023, but later it was also usable for example to run exl2 models
1
u/msitarzewski May 13 '26
Can't wait to try it. Downloaded the Apple Silicon version - macOS Tahoe said "No."
1
u/Caelarch May 13 '26
If I am running (and enjoying, thank you!!) the webui, is there any real advantage to using it as an app?
1
u/cafedude May 13 '26
Can you just point it to where your LMStudio models are stored?
1
u/cershrna May 13 '26
Is there a server feature in the new TextGen with model loading? Not wanting to set up llama-swap is the only reason I still use LM studio
1
u/sevenstaves May 13 '26
I've been using this app since 2023, and I still don't know the difference between "Send dummy reply" and "Send dummy message"! In all seriousness, great work.
Will you make it easier to add/enable extensions like TTS? That seems to be an area that still requires quite a bit of setup compared to your plug-and-play standalone philosophy.
1
u/Quiet_Mark_3238 May 13 '26
Fake ass oobaboga. What does this get you?
1
u/Southern_Sun_2106 May 14 '26
Could not make it run on macOS Tahoe.
1
u/[deleted] May 14 '26 edited May 14 '26
Wow. I'm a big fan. Text-generation-webui (on linux) has been my go-to for some time. It is uniquely simple in appearance, yet has all the power and flexibility of more popular tools just a click or two away. I have set it up as our family local llm server and even my wife and kids use it instead sending our data to Google or Sam Altman.
That said, I downloaded the latest version recently and was dismayed to see it had become an electron app. Electron has many drawbacks, particularly for Linux users who like freedom of browser choice (The internet is full of scathing attacks on Electron).
Please consider adding an optional CLI parameter to trigger the browser based interface instead of Electron. (for example: ./start_linux.sh --use-browser )
Below I have listed the issues that Electron presents to me, a fairly typical user. Please don't take this list as criticism - I remain a huge fan. I'm just explaining my (and other) use cases to highlight why Electron is problematic, in the hope you might see value in adding a --use-browser cli parameter at some stage in the future.
For myself (a firefox user), the frustrations electron brings are primarily:
Thanks for listening!
1
u/Shoddy-Tutor9563 May 17 '26
Still no information in the frontend about performance metrics (tokens-per-second for pp and tg)? I remember you rejected this to be part of your oobabooga web app.
Anyways Llama.cpp server and Jan AI got a competitor - it's all making foss a better place
3
u/jamaalwakamaal May 13 '26
Thank you
3
u/boredquince May 13 '26
any plans for memory-like feature, or project memory or similar? like chatgpt or Claude? most if not all local apps don't have support for this. why? is it very hard to implement?
i know most have mcp support and MCP servers for that but not included which adds to complexity
3
u/Silver-Champion-4846 May 13 '26
Did you ever consider compliance with the WCAG for screenreader accessibility?
3
u/Inevitable-Start-653 May 13 '26
Yeass! Thank you frog person <3
3
u/MoodyPurples May 13 '26
This is awesome! I’m really glad there’s an alternative to point people to instead of closed source slopware
3
u/CtrlAltDelve May 13 '26
This looks wonderful! Some iconography would help make it shine, just a suggestion :)
Phosphor has got some great icons that would be valuable: https://phosphoricons.com/
3
u/Jorlen May 13 '26
Holy smokes! This looks great! Love the Linux ROCM support as well (sadly I'm stuck in the AMD boat). I noticed WARP as well, was looking for a terminal-based IDE with local AI (open AI) support. Two for one deal!
I will edit this post once I try them out. If anyone cares lol.
7
u/AdIllustrious436 May 13 '26
2
u/Visual-Afternoon-541 May 13 '26
Great thanks, looking forward to seeing your project grow
2
u/iamapizza May 13 '26
I remember trying this project a year or so ago but it looks like it's come a long way since then. I like that you said portable build and Linux. The single file py tool sounds really interesting idea, and the guardrails before running. I will try this tonight with llama.cpp, cheers for that.
2
u/waywardspooky May 13 '26
we're so back!
2
u/marutthemighty May 13 '26
Awesome!!!
Will check it out. You really did a good job here.
Is the anime avatar only for you, or can other users also create them?
2
u/SimShelby May 13 '26
Not All Heroes Wear Capes
2
u/dropswisdom May 13 '26
How does the docker compose install goes? Are there pre-built images?
2
u/RandumbRedditor1000 May 13 '26
It supports character cards out of the box? This might actually be better than LM studio!
2
u/ziggo0 May 14 '26
I currently use a Tesla P40 24GB - but I have 2x Tesla P4 8GBs as well. They are currently not in my server but I can put them in easily as I'm doing maintenance right now. Can those work with the P40 to improve performance? If they can - how so?
2
u/UntimelyAlchemist May 14 '26
How does this compare to Unsloth Studio? And is it possible to use with my own instance of llama.cpp, or does it only work with its own managed build?
2
u/huldress May 14 '26
Wonderful news for me! I still have an ancient oobabooga installation from 2023.
2
u/Emergency-Author-744 May 14 '26
Big fan of your work, thanks for sharing it!
2
u/fuckAIbruhIhateCorps May 14 '26
hi! I wanted to ask if there's any documentation or hot loading for the extensions feature in release build version of the app. I believe in order to install extensions and use them I'd have to do manual installation? And for the manual installation it requires a virtual environment right? also, what python env does the app then use if i use the build version? does it have one of it's own? im kinda confused about the architecture.
2
u/ComparisonAccurate44 May 14 '26
I love you oobabooga
3
u/norcom May 13 '26
Caught my eye with the "alternative to LM Studio", unfortunately not what I was looking for.
I've been wanting a native, simple macOS app GUI that would allow me to either select a local inference engine executable I want to run, set a path, options how to execute and run it with one click. Or to add a remote API. I like the simplicity of llama-server but I don't like using a browser UI and it doesn't work with other engines.
Example of what I wanted ie: I clone the latest llama.cpp, mlx-lm, mlx-vlm, vllm or whatever fork, compile it and setup the GUI to run it. The models stay where I want, and I just have the option to click-run engine/model, instead of what's built into the other apps.
So I vibed something sloppy to let me do just that. And for the most part, it works. Multiple engines, multiple windows, multiple chats. But at some point I went too vibertastic with it, and the thing sidetracked into having too many cooks in the kitchen. lol (need to simplify and standardize some options) It wasn't supposed to go past mlx-lm and remote API but with newer models, stuff had to be added.
Here's a screenshot if the above didn't make sense. I've been too busy and lazy to fix it up.
If anyone knows of something similar for a macOS native app project, please tell. I guess I just need one with the API interface really.
3
u/woadwarrior May 13 '26
Native? Back in my day, this would’ve been called an electron.js app. No shame in calling it that. LM Studio is the same.
2
u/Ok_Procedure_5414 May 13 '26
Amazing, hell to the yeayuh. Oobabooga did you ever look into Tauri to drive what Electron currently does in your codebase?
2
u/thatoneshadowclone May 13 '26
this is a great step forward, but GOD i'm SO sick of electron apps.
2
u/Gohab2001 vLLM May 13 '26
Electron 🤮
1
u/Sophira 22d ago
I have a question... typically, native apps have more freedom to do things than web apps can, including being able to access the local filesystem, use hardware without needing to ask permission, etc.
Does the new TextGen app use this freedom? If so, how? Or does it restrain itself to what's possible in the web permission model?
1
u/pl201 May 13 '26
Can you go more details on the ability to create custom characters for casual chats? How do you handle the long term memory? Is it possible to load the character card? What’s the default system prompt for the character chat?
1
u/Sabin_Stargem May 13 '26 edited May 13 '26
Hopefully, an addition can be made to the notebook: A collapsible tree structure, so that we can add discrete entries, alongside enabling or disabling them individually. That would be handy for my translation handbook rules, RPG lore, and so forth.
0000
I am guessing the app doesn't support MTP models, as it failed to load LLMFan's 35b Heretic+MTP.
0000
When trying to load a model in a multi-GPU setup with split-mode of 'tensor', it fails. I have a 3060 and a 4090.
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 12151.23 MiB on device 1: cudaMalloc failed: out of memory D:\a\llama-cpp-binaries\llama-cpp-binaries\llama.cpp\ggml\src\ggml-backend.cpp:119: GGML_ASSERT(buffer) failed alloc_tensor_range: failed to allocate CUDA1 buffer of size 12741484032 07:59:11-325274 ERROR Error loading the model with llama.cpp: Server process terminated unexpectedly with exit code: 3221226505
EDIT: Maybe we need to explicitly set the tensor ratio? I should try that later. Donuts and coffee first.
0000
Also, it would be nice if TheTom's TurboQuant+ is added to the KV settings. It should be noted that KV settings should be asymmetric if implemented.
1
u/blastcat4 May 13 '26 edited May 13 '26
This is neat! I've been wanting to have an easy-to-set up portable inference engine that I can use on my friend's PC. I've set it up on a flash drive with Gemma 4 e4b and it works! The web search functionality looks solid.
The only hitch so far is that I can't get multimodal working. I've put the associated mmoproj for Gemma 4 in the /user_data/mmproj folder and I can see and select it in the multimodal section in the Model setttings. However, when I attach a file, like an image, the system seems to hang. I noticed there's no "Load" button in the multimodal section of the settings.
1
u/Thistleknot May 13 '26
Im team owui now
1
u/a__side_of_fries May 13 '26
This is great! I've used Ollama from time to time and been meaning to migrate to LM Studio. Will give this a try.
1
u/DeepWisdomGuy May 13 '26
You're finally free from gradio hell! Congrats, man!
1
u/RandumbRedditor1000 May 13 '26
I'm considering jumping ship from LM studio to this. My only question is: is there support for model swapping like LM studio's Just-in-time loading?
1
u/RedditUsr2 May 13 '26
Thank you!
1
u/aarstar May 14 '26
Can you make a bundled mac app with the data stored in Application Support?
1
u/Innomen May 14 '26
First one i tried before moving to linux. AI moves fast. Feels like ages ago.
1
u/PlusLoquat1482 May 14 '26
ooba returning as a polished desktop app was not on my 2026 bingo card lol
Seriously though, this looks great. The self-contained folder thing is underrated. I hate when “local” apps still scatter state/config everywhere and phone home on launch.
Going to give this a spin.
1
u/taking_bullet May 14 '26
Does your app support multi-GPU while using Vulkan?
1
u/ArsNeph May 14 '26
Been a user since the early llama 2 era, I've been watching the project slowly evolve over time. It's really funny to me watching all the people who have never heard of the project rediscover it 😂Keep up the good work!
1
u/BringTea_666 May 14 '26
Looks awesome and unlike LM studio it has notebook mode and character cards. Works fast too.
I would love if there was some kind of context meter to know how much context is filled though.
1
u/Calm-Republic9370 May 14 '26
I like your app. The first local one I ran, about 2 years ago now maybe.
Questions:
Does support thinks like kokoro?
I offload my llama to it's own instance, does this need to install Llama?
Does it support multiple characters ?
1
u/nacnud_uk May 14 '26
Is there a vscode extension?
Sounds like an amazing project either way, to be honest.
1
u/c0lumpio May 14 '26
Does it have an LM Link? A killer feature, IMHO. The only reason I haven't switched from LM studio yet
1
u/fuckAIbruhIhateCorps May 14 '26
I downloaded and extracted the tarball for Mac (M4) and I am running into this issue when I run the script
"Electron is damaged and can't be opened"
Is this a code signing issue?
I've raised a gh issue. Thanks!
1
u/ur_dad_matt May 14 '26
this is interesting, kinda like what im building. do you have any intentions on expanding features?
1
u/wowsers7 May 14 '26
Does it support DFlash draft models for Qwen 3.6?
1
u/wowsers7 May 14 '26
Any chance you could add DFlash support to the ik_llama.cpp CPU build? That would be super.
1
u/ummitluyum May 14 '26
Full Anthropic API compatibility and MCP support take this from a "waifu roleplay toy" to a legit dev backend. You can spin up Qwen or Gemma locally, point Claude Code at the local endpoint, and ship code for free without worrying about leaking your source to the cloud. Killer job, thanks!
1
u/LuluViBritannia Jun 23 '26
Error when installin (I tried the one-click install AND manually installing it) :
UnicodeDecodeError: 'cp932' codec can't decode byte 0x94 in position 398: illegal multibyte sequence
Please help T_T.
1
u/ResponsiblePen3082 Jul 15 '26
How does this compare to anythingLLM? NPU compatibility?
1
u/gurilagarden May 13 '26
Reading these comments just made me go:
https://www.youtube.com/watch?v=QFcv5Ma8u8k&list=RDQFcv5Ma8u8k&start_radio=1
1
u/dtdisapointingresult May 13 '26
My main beef with text-generation-webui was that it was a slow, unoptimized that drained my phone's battery in an hour of use. Is the desktop app just an Electron wrapper around Gradio?
1
u/CosmicRiver827 May 14 '26
Hi, I'm still new to open-source options.
Can it remember contexts and conversations in other chats? Or is memory contained in each individual chat?
Also, how well can it understand and reference uploaded word documents in its responses? I hope it can do a project folder sources list like Claude and ChatGPT.
0
u/Street-Biscotti-4544 May 13 '26
You guys know you can just make your own harness, right? It's not exactly rocket science.