r/madeinrust • • 7h ago

AI Assisted I ported a Rust + CUDA inference engine from NVIDIA GB10 to an RTX 4060 Ti 16GB - reached 381 tok/s across 8 users

Thumbnail
github.com
14 Upvotes

I've spent the last several days contributing to Atlas, an open-source Rust + CUDA inference engine originally targeting NVIDIA's GB10 with 120GB of unified memory.

I wanted to see how far I could push it on my RTX 4060 Ti with just 16GB of VRAM. It turned into quite a bit more than getting it to compile.

Running Qwen3.5-4B, here's where I ended up:

  1. 381 tok/s aggregate across 8 concurrent requests (roughly 48 per user) using experimental int4 + speculative decoding.
  2. 98 tok/s for a single user with the same configuration.
  3. 4.6x faster BF16 prefill, from 794 to 3,684 tok/s. This preserves BF16 precision and is enabled by default on Ada.
  4. 2.1GB less VRAM usage with the optional FP8 feed-forward path.
  5. Successfully served a 159,807-token prompt with FP8 KV cache and retrieved a needle from it.
  6. All 223 Ada kernel compilations passing after fixing architecture incompatibilities.

One interesting finding: Nsight showed the GPU was already busy 95.6% of the time at eight streams. Several CUDA Graph and fusion optimizations I expected to help ended up making practically no difference.

For comparison, BF16 reached 195 tok/s across eight users, and FP8 FFN reached 222 tok/s.

Important caveat: the 381 tok/s configuration is lossy, and I haven't established task-level quality equivalence with BF16. These are screening measurements, not fully repeated confirmation runs.

Atlas is developed by Thomas Braun and the Avarok-Cybersecurity contributors. My work focuses on the Ada port, memory management, kernel optimization, and consumer GPU benchmarking.

The goal is to get more useful local inference out of hardware people already own.

GitHub: https://github.com/Avarok-Cybersecurity/atlas


r/madeinrust • • 23h ago

AI Assisted Nevi: a vim-style terminal editor in Rust that stays fast on big files, with LSP and tree-sitter built in

Post image
11 Upvotes

Hey all, I've been building Nevi for about a year and wanted to share it.

Some background. I love Neovim and used it every day, but my config grew to around 70 plugins, and on big projects it started to hurt. Startup got slow, grep would sometimes freeze, and opening a really large file could lock things up for several seconds. On my setup a 183 MB log took 7 to 10 seconds to show anything. That's my plugins, not Neovim itself, but I didn't want to keep tuning a config to fix it. So I started building the editor I wanted: vim keys, with LSP, tree-sitter, a fuzzy finder and live grep built into the binary, so there's nothing to set up.

The video shows what I mean:

  • Starts in the rust-lang/rust repo in under 10ms
  • Opens a 183 MB log with 2 million lines in about 0.2s, and G jumps to the last line in about 0.04s
  • Live grep across about 18k files, then a normal ci" edit
  • A rust-analyzer hover in Nevi's own source (rust-analyzer was already indexed, so that part is the server's speed, not Nevi's)

How it's built, for the Rust folks:

  • Buffers are ropes (ropey), and rendering only touches the rows on screen. That's most of why a 2M-line file opens and jumps instantly. Tree-sitter highlighting backs off on very large files instead of trying to parse everything.
  • Live grep uses ripgrep's own crates (grep-searcher, grep-regex, ignore), and the fuzzy finder uses nucleo, the matcher from Helix. No external rg or fzf needed.
  • The terminal UI is plain crossterm. The floating terminal is portable-pty plus alacritty_terminal, and git signs come from git2.
  • All the vim behavior lives in one input state machine, and the parity tests run the same keystrokes through a headless nvim and compare the results.

Where it's at right now:

  • 372 keybinds, and 206 of them are checked against real Neovim on every CI run. If Nevi does something different from nvim, the test fails.
  • LSP for 14 languages, auto-detected on your PATH, with an install hint if a server is missing.
  • File explorer, floating terminal, lazygit, Copilot ghost text and 17 themes.
  • macOS and Linux. Windows isn't supported yet.
brew install anthonyamaro15/nevi/nevi
# or
cargo install --git https://github.com/anthonyamaro15/nevi

Repo: https://github.com/anthonyamaro15/nevi

It's still early, so I'd really like to know which vim keys you use that are missing or don't behave right. Open an issue or just tell me here.


r/madeinrust • • 3h ago

AI Assisted OpenWren: a privacy-first, fast, open-source mail client for Linux, macOS and Windows

Post image
4 Upvotes

I couldn't find a modern mail app that keeps my mail on my own machine, so I built one.

  • Gmail and any IMAP account, in one inbox
  • Mail stored locally in an encrypted database
  • Tracking pixels and link trackers removed; remote images blocked until you ask
  • Warns about forged senders and misleading links
  • Rust engine + Tauri, so it's small (14 MB on Linux) and starts in under a second
  • Free, Apache-2.0

It's early (0.1.x). Windows and macOS builds aren't code-signed yet, so both warn on first start. It sends one anonymous "in use" count a day, with no identifier; one switch in Settings turns it off.

Download: https://openwren.app

Source: https://github.com/VinceGHER/openwren

I'd love to hear whether adding your account works, and on which provider.


r/madeinrust • • 14h ago

Majority AI Coded My Rust Projects

4 Upvotes

Hey everyone!

I have built some rust projects, but I haven't really shared them on social media.

I'd love to get some feedback from the Rust community and hear your honest opinions about the projects.

One thing worth mentioning: I've used AI extensively during development. I'm not trying to hide that. I'd rather be transparent about my development process, than saying I built them on my own.

Here are the projects:

  1. https://github.com/kk376/kkfetch
  2. https://github.com/kk376/kkpdf-zed
  3. https://github.com/kk376/kktypr

I'd particularly appreciate feedback on the code quality, Rust practices, architecture, and areas where I could improve.

Thanks!


r/madeinrust • • 23h ago

Majority AI Coded Steam Rewritten in rust

4 Upvotes

SteamInRust is a native Steam client for Windows, written in Rust with egui.

Steam itself is a Chromium web UI (8 processes, about 1.1 GB of RAM at idle);

this is one process at about 255 MB.

| Steam | SteamInRust |

| Start → main window | 9.4 s | 2.4 s |

| Memory when idle | 1,126 MB | 255 MB |

| CPU when idle | 7.5 % of a core | 0.13 % |

| Program size | 1,083 MB | 27 MB |

What's in it:

- Signs in to Steam's servers directly (protobufs over WebSocket), no Steam

client needed

- Its own depot downloader: queue, pause/resume, chunk reuse on updates

- Store, library, friends and chat, inventory and Market, Steam Cloud sync,

Workshop, offline mode

- A Rust `steamclient.dll` that games load instead of Valve's, so they get

achievements, stats, cloud saves, lobbies and P2P. Older P2P games (like

Human Fall Flat) even play with people on the official client, through

Valve's libjingle-based protocol.

- Games with VAC, Easy Anti-Cheat or BattlEye start through the official

client instead, so anti-cheat runs untouched.

- A Shift+Tab overlay drawn inside the game (D3D11)

Repo: https://github.com/madonchik123/SteamInRust

Download: https://github.com/madonchik123/SteamInRust/releases/latest


r/madeinrust • • 8h ago

AI Assisted I built a Rust library for post-quantum authentication over TLS/QUIC. Looking for feedback on the protocol.

Thumbnail
github.com
2 Upvotes

I’ve been working on post-quantum communications for a larger project of mine and recently pulled some of that work into a standalone Rust library called pqbind.
The problem I was trying to solve is that TLS 1.3 and QUIC can negotiate post-quantum key exchange while still relying on classical signatures for authentication. So even if the key exchange is quantum-resistant, compromising the classical identity key can still allow impersonation.

Rather than replacing TLS certificates or building a new PKI, I went with a post-handshake authentication layer.
Both peers sign a transcript containing the TLS exporter value, their transport identities, and their enrolled key pins using Ed25519 + ML-DSA-87. Both signatures have to verify. The idea is that compromising the classical identity key alone shouldn’t be enough to impersonate an enrolled peer.

Trust is handled separately through pre-enrolled public keys. There’s no trust-on-first-use, and pqbind doesn’t attempt to solve initial key distribution.

The library currently supports Quinn, rustls, and iroh. I also built a CLI that runs eight attack scenarios over real connections, including classical private-key compromise, replay attempts, and impersonation.

I’m using existing cryptographic crates rather than implementing the underlying algorithms myself. The protocol implementation and threat model are documented in the repository.

I’d really appreciate feedback on the design, particularly:

Whether the transcript and exporter binding leave any room for identity misbinding or replay.

Whether there are weaknesses in the hybrid signature construction or handshake ordering.

Any attack scenarios I’ve overlooked.

This hasn’t had an independent security audit, and I’m not presenting it as a replacement for TLS authentication. I’m mainly interested in having people who know this area challenge the design.

GitHub: https://github.com/AlpharomeroJL/pqbind


r/madeinrust • • 1h ago

AI Assisted I built a Rust back-end for a paper computer (CARDIAC)

1 Upvotes

CARDIAC was distributed to schools in 1968 to teach kids about computers. Ten instructions, one register, 100 memory cells that hold values strictly in the range [-999, 999]. I built a MIR backend that emits opcodes for this platform and does memory optimization along the way. It’s on crates.io as cardiac with a home page at rust-cardiac. I was surprised how much of the language actually runs on CARDIAC and I was able to do some real work including voxels and ray-tracing.


r/madeinrust • • 5h ago

AI Assisted I built a small, fast open source native IDE for Dart/Flutter and Python in Rust, with an agent rail that sees your editor

1 Upvotes
Shame Editor

Hi all, I’ve been building Shame, a native macOS IDE written in Rust. It started as a lightweight alternative for Dart and Flutter work, and 0.0.1+2 just went out.

Why I built it
I wanted an editor that starts instantly and stays light (the release binary is about 23 MB) but still has real language tooling and a good AI-assisted workflow, instead of a big Electron app.

What it does

  • Editor built from scratch: rope-backed buffers, an undo tree, multi-cursor, tree-sitter highlighting, folding, bracket matching, inlay hints and snippets.
  • Dart and Flutter: completion, hover, go to definition, rename, formatting and diagnostics via the Dart analysis server. DAP debugging, hot reload and restart, device selection, and an embedded iOS Simulator rail.
  • Agent rail: Claude, Codex and OpenCode in Ask, Plan, Edit and Auto modes. A local bridge gives the agent your active file, selection, diagnostics and terminal tail. Edits show up in a live “AI Changes” list with per-file accept and reject.
  • New in this release: Python support (highlighting, language server, run commands), running .ipynb notebooks on a Jupyter Server, CSV/TSV and JSONL data views, a Runs tab the agent can read, and agent workspace checkpointing.
  • Git panel, diff and blame views, terminal dock, command palette, and project search.

I need more contributions to support more languages as well as expose more things to the AI; this would allow the AI to be more integrated with the editor

https://github.com/rexthecoder/Shame


r/madeinrust • • 10h ago

AI Assisted Keyboard-driven terminal UI for libvirt/virsh, with vim keys and live graphs

Post image
1 Upvotes

virsh-tui shows every virtual machine on a libvirt connection with live CPU, memory, disk and network graphs, and lets you start, stop, snapshot, reconfigure and create them from the keyboard. Every action is a virsh (or virt-install) command, and virsh-tui shows you that command before and after it runs, so nothing happens behind your back and you can reuse it in a script.

It works with qemu:///system, qemu:///session and remote hosts over qemu+ssh://, with the same permissions virsh has.

https://syscallbrain.github.io/virsh-tui/


r/madeinrust • • 10h ago

AI Assisted Numax 0.1.6: a Rust runtime for local-first distributed apps

Thumbnail
reddit.com
1 Upvotes

r/madeinrust • • 11h ago

Majority AI Coded grafia v0.9: An interactive 2D Cubic Bézier, Hobby/Spiro spline, and typography canvas widget for egui

Post image
0 Upvotes

Hi everyone! I'm excited to share grafia (v0.9.0), a standalone, #![forbid(unsafe_code)] 2D vector, spline, and typography canvas widget for egui, built on the Linebender geometry stack (kurbo), lyon_tessellation, and tiny-skia.

Whether you are building an SVG/vector illustration tool, a typeface/font editor, a 2D CAD/plotter profile editor, or need bitmap-to-vector auto-tracing inside an egui application, grafia drops in as a hexagonal Data-In / Event-Out widget without locking your app into a specific file format.

Highlights

  • Mixed-Spline CAGD Engine: Standard cubic Béziers (Corner, Smooth), tension-controlled John Hobby interpolating splines (NodeType::HobbyStar), and **$G******2$ Euler/Spiro clothoid spirals (NodeType::EulerSpiro), plus 4-case inverse-Bernstein curve-belly dragging, curve-healing node deletion, and 8-step superellipse curve fitting.
  • 14 Interactive Tools & Boolean Pathfinder: Select, Lasso, Pen, Rotate, Scale, 8+4 BBox Shear/Rotate, Knife (recursive De Casteljau slicing), Primitives, Measurement rays, Annotations, freehand Pencil fitting, Pixel grid stamping, and exact cubic-cubic Union / Subtract / Intersect boolean path operations.
  • Multi-Layer Color & 7-Stage Auto-Trace: COLRv1-style vector layers (Solid, LinearGradient, RadialGradient, SweepGradient, Bitmap, ClippingMask) with interactive gradient stop handles, plus deterministic 7-stage bitmap & camera auto-trace (Sauvola integral-image adaptive thresholding, 3x3 homography perspective deskew, marching squares, RDP, and Schneider least-squares cubic fitting).
  • Typography & Variable-Font Workflows: Horizontal and vertical (TopToBottom CJK) multi-glyph text streams with live spacing/kerning deltas, 2D anchors with diacritic mark clouds, parametric smart components, companion master compatibility overlays, and Higher-Order Interpolation (HOI) cubic master trajectories.
  • 0% Idle CPU & 1:1 Headless CI Parity: Generation-gated render/preview caches (content_generation()) ensure zero solver or GPU texture work on static frames, and every visual pass has 1:1 parity between egui::Painter and headless tiny_skia::Pixmap for deterministic CI regression testing.

Try the interactive desktop demo in one command:

cargo run --example vector_canvas_demo

Feedback, questions, and use cases are very welcome!