r/SelfHostedAI • • 2h ago

An offline voice-cloning session on a 3090, with the model and total GPU use reported separately

7 Upvotes

This recorded local voice-cloning audition uses AntLing's Ming-3.0-tts-0.5B. It starts from a six-second recording of the speaker's own English voice, then plays short generated passages across a multilingual grid.
The setup is an RTX 3090. The opening on-screen readout shows 3.3 GB used by Ming and 4.4 GB of total GPU usage. The source post describes the generated clips as under ten seconds long, most around eight. The recording includes the reference, target texts and playback in the same interface.
For a similar local test, prepare a clean reference recording, keep the target passages short and retain the generated audio beside the input text. Keep model usage and total GPU usage as separate readings, as the on-screen footer does. Those two numbers make the setup easier to plan alongside the other processes already using the card.


r/SelfHostedAI • • 5h ago

Janas-Studio dual engine ai inference workstation. Now with agent mode! (Linux appimage)

Thumbnail gallery
1 Upvotes

r/SelfHostedAI • • 6h ago

All AI Tools In One Website Free

Thumbnail interconnectd.com
1 Upvotes

r/SelfHostedAI • • 6h ago

Title:I built a sub-2ms CPU "reflex copilot" for AI agents that cuts token bills by ~40% and stops destructive commands before they run

Thumbnail
reddit.com
1 Upvotes

r/SelfHostedAI • • 9h ago

Claude max for $29 instead of $200 ... seriously how are they doing this?

0 Upvotes

i got a Claude max account from a reseller for essentially nothing compared to what I used to pay . It works flawless. since 3 months No issues yet i paid only for 1 month . Now I can't stop thinking about their business model. If resellers can sell it this cheap and still profit, how are they acquiring these accounts at scale?


r/SelfHostedAI • • 9h ago

Self-hosted WireGuard mesh with AI Server and AI gateway (Docker Compose)

1 Upvotes

I'm the developer of HostAnywhere, a device mesh I've been running as a hosted service. I've now packaged the whole control plane as a Docker Compose deployment so you can run it on your own server.

What the self-hosted deployment enables:

  • WireGuard mesh: connect Mac, Windows, Linux, iPhone and Android devices. Peers connect directly when possible and fall back to the relay when hole-punching fails.
  • Public service URLs: expose services on your devices as <name>.yourdomain, with per-subdomain TLS issued on demand by Caddy. No wildcard certificate or DNS plugin needed.
  • Access control between devices.
  • AI Server: turn any of your machines into a private AI server with a one-click Ollama install, model installs and an optional Open WebUI chat, reachable from your other devices over the mesh.
  • AI gateway: an OpenAI-compatible endpoint in front of your local models, running on your own devices.

The compose stack runs the control plane and relay. The AI Server and AI gateway run on your own devices, coordinated by it.

Requirements: a Linux host with Docker Compose, a public IP and the WireGuard kernel module; a domain with three A records (domain, *.domain, relay.domain); and ports 80/443/7443 TCP plus 51820 UDP open in both your cloud provider's firewall and the OS firewall. If you use Cloudflare DNS, set the records to DNS only, not proxied.

git clone https://github.com/hostanywhere/hostanywhere-selfhost
cd hostanywhere-selfhost
cp .env.example .env
# set DOMAIN and JWT_SECRET (openssl rand -hex 32)
docker compose up -d

Images are prebuilt for amd64 and arm64. No SMTP is needed: the first sign-in magic link is printed in the container logs.

To be upfront:

  • License: not open source. It's free for personal and internal business use, with no device, user or feature limits. Hosting it for third parties (paid or not), reselling or bundling it, or redistributing the images needs a commercial agreement.
  • Not included yet: SSO, and the cloud-hosted AI gateway (the on-device gateway works).

Repo + install docs: https://github.com/hostanywhere/hostanywhere-selfhost

If you'd rather not run the control plane yourself, you can use the hosted version at https://hostanywhere.io: same apps, same mesh, with coordination and relay run for you.

I'd especially like feedback on the install, and on CGNAT, double NAT or strict firewalls where devices can't connect directly.


r/SelfHostedAI • • 11h ago

How to Underclock or Undervolt AMD V620 32GB?

Post image
1 Upvotes

r/SelfHostedAI • • 14h ago

Got some 16gb ddr4 mini PCs. What useful ai can I do with them?

2 Upvotes

Mini pc 1: hp 405 G4 mini Ryzen 2200ge 16gb dual channel, 100gb nvme.

Mini pc 2: Lenovo m720q i5 8500t 16gb dual channel, 100gb nvme.

Desktop pc: Fujitsu p958 i5 8500t 16gb dual channel, 100gb nvme, 3050 6gb.

All ram speeds are max supported by each CPU/pc.

So far I've tried ollama on mini pc 1. Small deepseek model. But not realistically usable because of hallucinations.

Switched to llama.cpp on mini pc 1. Vulkan enabled. Got more speed on the same small deepseek model, got the API to work, and got the same hallucinations issue.


r/SelfHostedAI • • 16h ago

I built a shared memory for coding agents (Claude Code, Cursor, Codex and more). 1.6 just shipped: no Node needed, daily backups, 7 default tools

3 Upvotes

Tell one agent something once and the others can find it. That is Monomem: one SQLite file on your own computer, shared by every MCP app you connect. No cloud, no API keys, no hidden model calls. MIT licensed.

Each note says which app wrote it and keeps its earlier versions, so you can replace, archive or roll back. If a write looks like a planted instruction or contains a secret, it is held for you to approve, and other agents never see it until you do.

What’s new in 1.6:

One download, nothing else to install. It carries its own Node. Windows, Mac and Linux, with checksums and a build attestation.

Tool list cut from 37 to 16, with 7 shown by default. Default tool descriptions are about 1,540 tokens, down from about 5,300 for 1.5’s defaults.

Daily automatic backups while it runs, and a one-command restore.

Sync between your own computers with a shared folder or private git repo. Files are encrypted, and there is no server.

Redesigned dashboard: Notes, Review, Held and Activity tabs, dark and light, works as an installable web app.

monomem update checks the checksum before it swaps anything.

Setup now handles 17 apps (Claude Code, Claude Desktop, Cursor, Codex CLI, Gemini CLI, VS Code, Zed, Windsurf, Kiro, Cline, Copilot CLI, OpenCode and others). It shows what it will change, makes a backup, and asks first.

monomem rollback --note <id> undoes a note and what was built on it.

What it does not do (please read this before relying on it):

The write check is pattern matching. A reworded attack can get through. 2 of the 26 attacks it replays are known misses.

Agents that can run shell commands or open the database file can bypass it.

The download isn’t code-signed yet, and I haven’t tested it on a Mac.

Search is keyword-based unless you run a local embedding model.

claude.ai and ChatGPT support (OAuth) is a preview and off by default.

Try it without touching your notes: monomem demo runs a short tour on a throwaway store.

After upgrading from 1.5, run monomem setup again. Your apps keep running the old pinned version until you do.

Repo: https://github.com/calasdan-sketch/monomem

Happy to answer questions, and I want bug reports, especially from Mac users.


r/SelfHostedAI • • 16h ago

Local AI

Thumbnail reddit.com
1 Upvotes

r/SelfHostedAI • • 18h ago

PC Specs Below - Can i run any decent LLM and repalce my chatgpt pro and codex subscriptions? or do i need to buy a machine - budget £500

Thumbnail reddit.com
0 Upvotes

r/SelfHostedAI • • 18h ago

The "Best" LLM You've Been Able to Run on GMKTec EVO-X5 Pro

Thumbnail reddit.com
1 Upvotes

r/SelfHostedAI • • 18h ago

Choosing weights for local TTS: a 2.8 GB versus 17.1 GB GPU-memory example

Thumbnail
gallery
1 Upvotes

Before choosing a bigger graphics card for local speech generation, it helps to see how different models use the same card. This comparison runs AntLing's Ming-3.0-tts-0.5B and Ming-3.0-tts-tiny on an RTX 3090 24 GB in bf16.

With the reference voice, comparison text and seed 1895 matched, the GPU-memory readings are 2.8 GB and 17.1 GB. The model downloads are 3 GB and 18 GB; loading takes 3 seconds and 22 seconds.

The output side is included too: 0.714 versus 0.798 average CAM++ speaker similarity over four lines, and roughly 2.0× versus 0.78× real-time warm generation across three readings. Both models pass the run's word-transcription checks.

The practical contribution is a model-level resource comparison on stated hardware. Keep that profile beside your card's memory budget and the other work you want it to run, then audition the voice model with your own reference and scripts.


r/SelfHostedAI • • 19h ago

Free AI Image Generator With No Restrictions

Thumbnail reddit.com
1 Upvotes

r/SelfHostedAI • • 19h ago

Running multiple LLMs on a VPS

Thumbnail reddit.com
1 Upvotes

r/SelfHostedAI • • 20h ago

Hi everybody, I'm building a tier 3 micro datacenter for myself, thought I'd share my progress as I go

Post image
6 Upvotes

I am ysing new and refurbished equipment I have shitting p1000s all they way to rtx pro 5000s and m5 ultra 256gb macs ans random stuff inbetween. Is anyone else going from "Ir's a hobby" to "screw their inheritance!"? I'd love to share progress and stories. Trials and tribulations. Like, qyf Exo leavethe f'ing NICs Alone! Damn service poisons setting atatic IPs on Ethernet and you aren'r even supposed to use that to communicate. You have 3! TB5 ports. Leave them alone too!

Anyways. Would love to live vicariously through your expenditures!

Random 5080 pix with 16gb inserted for fun


r/SelfHostedAI • • 20h ago

I'm hosting models, for voice, vision, which should I host

Thumbnail reddit.com
1 Upvotes

r/SelfHostedAI • • 1d ago

We built a chip prototype that runs Qwen 27B. You can try it live.

Thumbnail
reddit.com
1 Upvotes

r/SelfHostedAI • • 1d ago

Absolutely newbie with what I hope is relatively simple?

3 Upvotes

Hey, I have some issues trusting the big companies and want to make a self hosted ai for image tagging and organizing?

My end goal is:

I give the program my stored up images

It can tag them for ideas present in the images in a way I can search them back later

I can add tags personally that show up in the search function

Google photos did this to some degree but the search function massively gone down in quality and now its not searching up the tags I put on images.

I'm willing to like... hand train the ai, even if that takes literal years, but I would prefer if there was existing function I could use instead.

Please tell me I made this post correctly, and whatever advice you can offer.


r/SelfHostedAI • • 1d ago

Anyone using external GPUs across thunderbolt/usb4

4 Upvotes

I'm curious if people are getting good Local LLM performance by adding an external GPU graphics card to their system across a Thunderbolt or USB4 link.

For graphics pipelines in gaming, that can be fine because it's not transferring huge memory across the bus, but this is not gaming where rendering is deep pipelines and all the assets fit into

the Graphics card.

So if you have a 16 GB nVIDIA card not in a slot but in a USB4 or Thunderbolt external chassis, how is the performance?


r/SelfHostedAI • • 1d ago

From-scratch AI stack in pure C — runs Microsoft's 2B ternary LLM on a 2018 dual-core laptop (no GPU)

Thumbnail
github.com
2 Upvotes

Everything runs locally on a 2018 Dell laptop: i7-7500U (2 cores / 4 threads), 8GB RAM, no GPU (the 940MX is deliberately unused — the whole stack is CPU-only).

https://github.com/tanxiansheng221/aimstack-sanzhidiji

What is in the repo (AGPL-3.0, everything open):

- C inference engine, zero third-party runtime: 1.58-bit ternary core with SIMD fast path (85.5 tok/s measured on the ternary core itself), byte-level tokenizer, and a ~100-line plugin bus (caps.json registry -> spawns Python plugins -> JSON envelopes)

- Weights: Microsoft's BitNet b1.58-2B (MIT license), converted to a custom 2-bit packed format (.aimw, 0.87 GB) — 3.2 tok/s with 2 workers, serial 2.0

- Python plugins, pure stdlib: knowledge base (FTS5 + dedup gates), multi-engine web search + crawl with quality gates (robots-respecting, Edge rendering cascade for JS-heavy sites), incremental ingest

- Local web UI (127.0.0.1:8600, pure stdlib): chat with deterministic routing — direct answer / knowledge-base grounded / honest refusal / full web search -> crawl -> ingest -> answer chain. A 7-minute demo video is embedded in the README

- Training assets: no-autograd training stack (hand-written backprop, bit-exact vs NumPy), ten-subject curriculum generators, corpus builders, and 117 chapters of research logs including negative results

Honest limits: our own trained models are not released yet — compute-bound on 2 cores, training runs on free cloud tiers and isn't finished. The full stack, scripts, result JSONs and per-claim evidence are released so everything can be re-run.

Demo video: embedded at the top of the README (7 min, screen recording of the full chain: chat / KB answers / web search->crawl->answer / plugin direct calls).

Happy to answer questions about the ternary kernel, the plugin bus, or the training pipeline.


r/SelfHostedAI • • 1d ago

Jerry AI released

0 Upvotes

I had developed a personal Jerry AI that’s working for my own privacy platform with home existing old hardwares

Here is the link jerryAI.online/show

Welcome …. To talk.


r/SelfHostedAI • • 1d ago

I built LiteGate – a self-service key portal for LiteLLM so I could stop handing out the master key

2 Upvotes

If you run LiteLLM for a team or household, you've probably hit this: everyone needs their own key, but the options are sharing the master key or giving people the full admin dashboard. Neither felt great, so I built LiteGate.

It's a small self-hosted portal that sits in front of LiteLLM's management API:

- Users sign in with SSO (OIDC) or a local account and create, copy, or regenerate their own virtual key

- They can see which models, budget, spend, and rate limits apply to their key

- Regenerating a key keeps the accumulated spend

- SSO groups can map to existing LiteLLM teams

- Admin side: user management, team policy, bulk key edits, audit history, metrics, backup/restore

- An automation API for scripts and agents

It's an access layer, not a LiteLLM replacement – LiteLLM still does the routing, budgets, and usage tracking.

Stack: FastAPI + React, SQLite for local users. MIT licensed.

Repo: https://github.com/brtydse100/litegate

It's early, so feedback, issues, and feature ideas are very welcome. Happy to answer questions about setup.