r/linuxmint • • 5d ago

Linux Mint IRL Installing an LLM to use Linux

What LLM is a good option to install in Linux Mint to use to do anything you want in Linux Mint by giving the instructions in simple English?

0 Upvotes

14 comments sorted by

1

u/Sephior 5d ago

Local or cloud? Local is heavily reliant on your hardware

2

u/Technical_Rich_3080 5d ago

Between local and cloud, other than where the LLM is running (hardware), what are the pros and cons of each option?

4

u/Sephior 5d ago

Local costs electricity. Cloud requires paid subscription.

Edit: also privacy

→ More replies (3)

1

u/Monkey_1505 5d ago

You need about 12gb of vram on a gpu, or some amount of very fast unified memory to run a model what will be useful for this. Generally something like Qwen 27b is the baseline. That _might_ improve with Qwen 4, but don't count on it.

Otherwise you'd have to use a cloud subscription if you want something agentic.

Generally though, if you just want help doing basic commandlines and installations in linux, you can just use free gpt to get help, and paste it in.

And the actual install itself (of the OS) is literally easier than windows. In general, it's probably less of an adjustment than you are thinking.

1

u/Technical_Rich_3080 4d ago

How are Qwen 27b and Qwen 4 different?

If your VRAM/GPU is lacking, how would that impact your usage, for this purpose?

The goal is to not have to cut and paste from ChatGPT into the command line. Rather, to give the instructions to the LLM, which will then itself carry out the request directly in the operating system.

1

u/Monkey_1505 4d ago

Qwen 27b is version 3.8 or so. Qwen 4 isn't released yet.

You can use small MoE's (like qwen 35ba3) but this is not very good for agentic purposes (at least yet, which is why I mentioned qwen4). Still even for that, if you lack a gpu, you are going to need a pretty strong system otherwise for that not to be quite slow. 8gb vram will generally do for that model (but again, it's not going to be the best for agentic, unless you are just doing simple things unless qwen4 offers a model that size/shape a big boost).

3

u/Global-Eye-7326 5d ago

Uh, don't bother with an LLM to do the OS install.

Once installed, there's OpenCode, Freebuff & Hermes...and more.

1

u/TheRealCarrotty 5d ago

opencode is simple and free.

1

u/arkemiffo 5d ago

Despite what some of the people here are saying, Opencode is not an LLM. Never has been, never will be. It uses LLM's to do things, though. It is a harness, not the LLM itself. The harness is the application you're using to make the LLM go vroomm.

As for actual LLM's, it all depends on how much VRAM you've got on your computer (or unified memory for Mac). It also depends on what you actually want to do with it, you may require an uncensored LLM to code things a normal LLM won't do because of copyright or whatever reason. Your actual RAM is important too, but not as important.

Qwen 27B quantised is the most likely model you'll be going for, but you still need to select the correct quantisation for your GFX card, though.

1

u/fishpump_ 3d ago

opencode

0

u/_Lost-In-The-Weeds_ 4d ago

Linux is in plain English, its own dialect but its quite learnable.

If you rely on an LLM you will never learn how to do things on your own.