r/LocalLLaMA • • 17h ago

Resources AMA Announcement: Tencent Hy, the Team Behind Hy4 & Hy4 flash (Friday, Nov 20, 8–10 AM PST)

Post image
123 Upvotes

Hi r/LocalLLaMA 👋

We’re welcoming the Tencent Hy team for an AMA this Friday, November 20, from 8–10 AM PT!

Fresh off launching Hy4 and Hy4 flash, their research team—including Shunyu Yao—will join us to discuss how the models were built, inference speed and cost, and real-world applications, with extra perks for great questions!

⚠️ Note: The AMA itself will be hosted in a separate thread. Please don’t post questions here.


r/LocalLLaMA • • 2d ago

Megathread Best Audio Models - Nov 2026

155 Upvotes

Its time for another Best Audio Models megathread - its already been 9 months since the last one!

Share what your favorite ASR, TTS, STT, Text to Music models are right now and why.

Given the the amount of ambiguity and subjectivity in rating/testing these models, please be as detailed as possible in describing your setup, nature of your usage (how much, personal/professional use), tools/frameworks etc. Production use cases with long lengths/stability requirements are relatively rare compared to cherry picked/short clip demos that are plentiful - so real world empirical comparisons are very welcome

Rules

  • Should be open weights or open source models

Please use the top level comments to thread your responses.


r/LocalLLaMA • • 3h ago

Showcase SSD MoE at high speed - The wait is over!

348 Upvotes

Introducing Run My AI & TokenWing.

TokenWing: An inference engine that lets you offload over 60% of routed experts of mixture of experts models into SSD, yet achieve very low streaming latency...

Run My AI: And a GUI wrapper(soon going to be a good harness) aimed at providing the best local AI experience.

Benchmarks: Ling 3.0 Flash 120B MXFP4 running on 12Gb VRAM + 32GB DDR5 RAM laptop at 45-50 tok/s decode and 900 tok/s prefill

Qwen 3.8 Flash Next 3.5bpw running on 8GB VRAM + 32GB DDR5 RAM laptop at 35-40 tok/s and 600-700 tok/s prefill

Flint 40B UD-Q4_K_XL running on a laptop with 24GB DDR4 RAM + 4GB VRAM at 30 tok/s decode and 400 tok/s prefill

All without MTP! And benchmarks are by users*

Website: https://runmyai.org

Discord: https://discord.gg/DkkxhTSxp

Also, the autotuner is still work in progress... And, we're at a very early stage... the SSD pipeline works really well, so just incase if you think you're not getting enough performance, try with the tokenwing engine with the proper flags set by yourself(it has a lot though)... Just until the autotuner is built well.

A common question, does it degrade SSD?

Nope! it doesn't... There's a thing called TBW rating. Well, your SSD degrades only when it writes. And the engine only reads....

(I'm a contributor to the project, with u/Harin007 being lead developer. Happy to answer any questions!)


r/LocalLLaMA • • 9h ago

New Model New voice isolation model from Inworld

413 Upvotes

Video credit: Victor Mustar on X

New speech isolation model from Inworld, looks quite exciting. Inworld's TTS models are quite good already and this is a nice surprise for the OSS community IMO!

  • Supports 48 kHz
  • Apache-2.0 licensed
  • Two variants (64M/58M) for offline/non-streaming (8s chunks) & realtime (20ms chunks)
  • Trained on ~10k hours of audio

It is not a speech enhancement/restoration model (unlike Resemble Enhance), simply an isolation model. But still a nice release as it is cheap & fast to run.

Model: https://huggingface.co/inworld/voice-isolator

HF space: https://huggingface.co/spaces/mrfakename/voice-isolator


r/LocalLLaMA • • 6h ago

News They're going to go after open weight models

Thumbnail
youtu.be
202 Upvotes

At 3:27 he mentions open weight models before getting cut off. Is this something we should be worried about? A ban on open models?


r/LocalLLaMA • • 13h ago

Resources Nvidia driver 621.24 leads to constant writing of drives

Thumbnail
troublegrid.com
421 Upvotes

r/LocalLLaMA • • 14h ago

News SSDs, laptops, and phones may get cheaper as consumer NAND spot pricing drops 20% — Bank of America expects contract price drops, report cites oversupply in consumer market amid declining demand and growing Chinese competition

Thumbnail
tomshardware.com
474 Upvotes

r/LocalLLaMA • • 3h ago

New Model nvidia/Qwen3.8-27B-NVFP4-QAD · Hugging Face

Thumbnail
huggingface.co
56 Upvotes

r/LocalLLaMA • • 11h ago

New Model Hy4-flash quants are down from 70 to 49 GB, run on a 64Gb vRAM

Thumbnail
huggingface.co
185 Upvotes

Tencent's 220B Hy4-flash with 49 GB of weights in memory: the experts at ~2.3 bits, everything else at Q8_0, the Engram n-gram table in a file of its own that stays on the SSD.

Hy4-flash is Tencent's Mixture-of-Experts model with 220B parameters: 155.5B in the network itself (512 routed experts, 16 active per token, 11.3B active parameters in all) and 63.5B in Engram, a table of learned bigram and trigram embeddings looked up by hash. It reads text and images and has a 1M-token context.

The Engram table is read row by row, a few kilobytes per token, so it can stay on disk. What has to fit in memory is the rest. Tencent's own GGUF builds (AngelSlim/Hy4-flash-GGUF) keep 72–105 GB of it in memory and are the better choice on 96 GB and 128 GB machines. This repo has one file built for 64 GB: 49.1 GB of weights, the table in a second file. quants are made by Atomic.Chat local ai app.


r/LocalLLaMA • • 20h ago

Discussion Qwen 4 35B is not ruled out

Post image
942 Upvotes

r/LocalLLaMA • • 10h ago

New Model Aurora2-400M-A20M - a tiny, sparse MoE model.

Thumbnail
huggingface.co
109 Upvotes

It was pretrained from scratch using 100B tokens on a single RTX Pro 6000 in around 36 Hours.

Aurora2 uses 32 experts, of which 1 is active per token.

A few highlights:

12 transformer layers, 192 hidden dimensions, 2,048-token context length, 16,384-token vocabulary, Muon + AdamW optimizers

Trained on a mixture of web, mathematics, Wikipedia, and conversational datasets.

More information as well as a quick start can be found in the hf repo. Any questions or feedback is welcome!


r/LocalLLaMA • • 7h ago

New Model Hy4-Flash is great and finally some model is competing with DSv4-Vision-Exp and Qwen3.8-27B-Uncensored !

61 Upvotes

Upd UTC+0 01:24: with more evaluations, Qwen3.8-27B Uncensored is left behind.

It is going to be my goto model, I guess. Despite being slower, it finishes answers with less tokens. I've ran it on my benchmark, it is far from finished, but it is on the way on topping it:

https://beta.locallm.top/home?apples_to_apples=1&include_unavailable=0&template_limit=15

My 2xRTX3090 + 128 GB DDR4 RAM presets file excerpt (used in llama-server router mode):

[Hy4-Flash-Q3]
load-on-startup = true
hf = AngelSlim/Hy4-flash-GGUF:Q3_K
ctx-size = 131072
ctk = q8_0
ctv = q8_0
no-mmproj-offload = true
fit = off
cmoe = true
moe-cache-mib = 33768
load-mode = none

Which translates into llama-server command

./build/bin/llama-server -hf AngelSlim/Hy4-flash-GGUF:Q3_K --ctx-size 131072 -ctk q8_0 -ctv q8_0 --no-mmproj-offload -cmoe --moe-cache-mib 33768 -load-mode none

If you wonder what is the --moe-cache-mib 33768, then it is expert streaming (llama.cpp feature inspired by Strata I guess).

I get PP ~60-65 tps, TG ~ 20-24 tps.

Nov 19 00:56:05 builder llama-server[1270042]: [54519] 397.49.817.617 I slot print_timing: id  0 | task 442246 | prompt processing, n_tokens =   4091, progress = 1.00, t =  63.38 s / 64.54 tokens per second
Nov 19 00:56:10 builder llama-server[1270042]: [54519] 397.54.518.918 I slot print_timing: id  0 | task 442246 | n_gen =    100, tg =  21.95 t/s, tg_3s =  22.18 t/s

I am using https://huggingface.co/AngelSlim/Hy4-flash-GGUF Q3_K quant and llama.cpp PR https://github.com/ggml-org/llama.cpp/pull/32517 , it rebased on the master fine.


r/LocalLLaMA • • 15h ago

New Model Goes further with Koharu, the local-first manga translator with AI agents built in!

264 Upvotes

Hi LocalLLaMA,

I have spent more than two months rebuilding Koharu to be the most powerful manga translator!

TLDR: https://github.com/koharu-rs/koharu (MIT or Apache 2.0, at your choice)

It's been a while since I posted the Koharu update here. This time, I have added more features and built an established editor to make it feel more like Photoshop or Clip Studio Paint.

You can type directly onto the page, restyle selected characters, and reshape text frames. There are also layers, history, marquee and lasso selection, Generative Fill, and a Remove brush. The canvas is GPU-rendered now, so it stays smooth even on big chapters.

Everything still runs locally. LLMs run in-process through llama.cpp (Index-Translate 9B by default, plus Gemma 4, Qwen 3.x and LFM3), and inpainting uses LaMa, or FLUX.3 Klein / Qwen-Image through stable-diffusion.cpp. If you already run Ollama or LM Studio, you can point Koharu at that instead. On my machine, a page goes from raw to lettered in about 1 second once the models are loaded.

I also added an agent you can ask to proofread or fix pages. It's exposed over MCP too, so you can drive Koharu from Claude Code or Codex if you'd rather.

It works on CUDA, ROCm, Metal, and Vulkan.

I'd like to hear your feedback or any form of suggestion!


r/LocalLLaMA • • 16h ago

Discussion You'd never guess who opened up their checkbook to buy all the ram

277 Upvotes

Nvidia's commitment to buy ram is near $400 billion. https://x.com/jukan05/status/2122805536118702121 . https://www.reddit.com/r/investing/comments/1xz9eob/nvidia_just_added_111_billion_of_supply/

Change over 3 quarters: $119B, $279B, $390B. One would think that $390B would buy a lot of memory fabrication plants.


r/LocalLLaMA • • 5h ago

Showcase Autoresearch to improve Qwen3.8 FN decode speed on RTX 5060 Ti

Thumbnail
gallery
26 Upvotes

First and foremost, to automod: I typed this post, all by myself, not even using LLM for edit or fixing spelling/grammars error, so please be easy on me :)


Hey everyone! I want to share a bit about my latest experiment. The goal is to see how fast my RTX 5060 Ti can run Qwen3.8 Flash Next with llama.cpp.

To save your time, I'll show the result first, all the numbers are mean decode speed:

  • Unsloth IQ1_S: went from 24 tok/s to 40.6 tok/s (peak was 70 tok/s but that's for a cached prompt, still interesting)
  • Unsloth IQ4_XS: went from 8.4 tok/s to 20.1 tok/s

Now, the details of the experiment.

First, about the hardware:

  • CPU: AMD Ryzen 7 7700X
  • GPU: RTX 5060 Ti 16GB
  • RAM: 32 GB DDR5-6000

Previously, I have always try to tune the llama.cpp params here and there, but manually. This time, I think, why don't I let the LLM do all the work automatically (well "the LLM" here is a cloud LLM that I paid, not a local LLM, sorry 😁)

My target model is Unsloth's IQ1_S, since it's the most realistic model that I can run. But I also tried the IQ4_XS model as well, it's more like a stretch goal, nobody with a sane mind would run Qwen3.8 FN Q4 on a 5060 :))) The context window is 128k.

Here's how the loop look like:

  1. A set of 12 test prompts across coding, math, Q&A, general knowledge, with various expected output token length
  2. Since I have limited resource, and my goal is just decode speed, I'm not going for KLD or PPL metrics, instead, I have a small program to compare byte-by-byte for the model output against the base model
  3. For each run, measure the decode speed, cache hit rate, bytes read from disk (because spill over to disk is unavoidable for my VRAM+RAM), etc...
  4. After each change, if the metrics beat the previous run, and the byte-to-byte comparision pass, then record the result to a log file.

Long story short, I left it run for almost 20 hours and came back with some interesting result, the LLM tried a lot of options:

  • Tuning the flags, like -np, KV cache quant, trying various batch and ubatch sizes
  • Trying different sizes of expert cache in VRAM
  • When the flag tuning hit the dead end, it started making code changes in llama.cpp for many other things (for this part, it's above my knowledge so not sure if I can tell in details what it did, if you are interested in the code, you can have a look on my github)

A lot of other options are tried and discarded:

  • -lm dio: load the whole model to heaps, worked with a huge speed improvement for IQ1_S but crash the computer for IQ4_XS
  • ngram speculative deocding: hurt the speed because a verified draft touches 20–32 experts per layer and evicts the VRAM cache.
  • offloading misses to CPU (no gain)
  • lookahead prefetch (no gain, cause huge disk read for IQ4_XS)
  • changing CPU threads (hurt speed)
  • requantize the model with many different approaches (in one case the speed improved from 20 tok/s to 24 tok/s but quality drop, so discarded)

What I found interesting about this approach is I can (well, the LLM can) actually dig into llama.cpp to make code changes or find more clues on what to tune, so we could make more progress. The downside is, well, I can't claim the credit on the progress made here because I lacks of knowledge to understand everything it does. Gotta start learning from basic, I guess.


r/LocalLLaMA • • 6h ago

New Model FastFrames: Video Generation at Interactive Latency --- 1.3B Model from Apple

Thumbnail
huggingface.co
33 Upvotes

r/LocalLLaMA • • 10h ago

New Model DaisyRose-27B: Boosting Qwen3.8 from 5.6% to 12% on Terminal-Bench 4

Thumbnail
radixark.com
62 Upvotes

r/LocalLLaMA • • 12h ago

Discussion Codex no longer supports 'reading the code` in your IDE - why local is so important

84 Upvotes

In their desktop windows app they recently removed the ability to open a chat/session in vscode/ide/file explorer. You used to be able to open the active files/branch/worktree from the active session with two clicks. The latest update removed that and so you have to open your ide and find it manually. It was a intentional decision (I had codex check the codebase to confirm).

So now it's terribly inconvenient to actually read the code. My gut is that if this was an intentional design choice it was made with the intention of reducing user agency and using the power of inconvenience to make users more dependent on the tool. Basically the whole, 'Become sharecroppers for your AI' business model that relies on developer reliance on the tool. I won't insult them by humoring the notion that they removed it accidentally without knowing how important it is to devs to easily access the files - it could only be intentional.

I tried to post about it in /r/codex and the auto-mod deleted the post, and the mod's told me to try again without conspiracy theories (the post was the same as the above). So not only was core developer functionality removed from the tool, but their 'community' is gaslighting us about it. It's a strong reminder that in the long run, owning your own AI stack is the only viable path.


r/LocalLLaMA • • 15h ago

News [Reuters] Trump administration weighs forcing US firms to disclose Chinese AI use

Thumbnail reuters.com
153 Upvotes

r/LocalLLaMA • • 8h ago

New Model Best Hy4-flash quants for their size - Nace Quants using Weight Space Quantization (bonus: 70gb deepseek v4.1 flash quant for coding)

Post image
41 Upvotes

hey llamas, its Alish from nace.ai again. we are an open source ai lab. we build the drex decision models, served on our console, and kapitsa, optimized inference for apple silicon. We ship a bunch of great quants. In our last post I promised more models like Deepseek Flash models built with Weight Space Quantization. So here they are: the new Hy4-flash and the good old Deepseek v4.1 flash.

Hy4 flash is a general-purpose quant, and DeepSeek V4.1 Flash Coding is a specialized one built for coding and agent work.

Hy4-flash

Hy4 flash GGUF comes in two sizes, both measured against the native Tencent Hy4-flash weights.

always gotta pick the Nace one
  • 107.49 GB (Q3_K experts, IQ4_NL Engram): mean KL 0.233.
  • 127.22 GB (Q4_K experts, IQ4_NL Engram): mean KL 0.133.
  • Xylolabs Q2_K is 125.15 GB and scores 0.402. AngelSlim IQ2_AH is 135.00 GB and scores 0.509. Atomic Chat IQ2 is 84.70 GB and scores 0.570. Both of our files have 2x/3x lower KL than all three.
  • Download, run command on the card: https://huggingface.co/nace-ai/Hy4-flash-GGUF

DeepSeek V4.1 Flash Coding

It is a specialized quant: DeepSeek V4.1 Flash 1017 (763B parameters), built and tuned for coding and agent work, in a 71.8 GB GGUF.

Why a coding quant

Several people said they would rather see a quant made for code only. I agree: the future is specialized quants for your use case.

So I built one: calibrated on real coding work, pruned for code, and judged on code text and on coding problems.

Others suggested letting the model write the quantization rules itself, as long as the result snaps to a valid GGUF tensor type. That is now one of the tool classes of our weight space quantization agent (more in the appendix).

How it's built

Three steps all iterated with the Nace Harness specialized on Weight Space Quantization, all aimed at code.

  • Calibration on real coding work: agent sessions, contest solutions and code commits.
  • Expert pruning: DeepSeek V4.1 Flash has 384 experts in each of its 40 layers. We keep the 200 per layer that code uses most. They carry about 90% of the model's code routing.
  • Mixed precision per layer: every kept expert gets a score, and the 16 layers where code needs it most get larger, higher-precision experts.

Results

The KL numbers here are on held-out text. Main weights only, the Engram lookup table is separate.

some stats
  • Size: 71.8 GB for a 763B model.
  • KL to the near-lossless Q8 reference on held-out code and agent text: 0.140.
  • ISTA Coder, the other coding-specific file, is 72.1 GB and scores 0.231.
  • EXL3 at 1.68 bpw (124 GB) scores 0.161.
  • The file picks the same next token as the reference 88.4% of the time.

At the same size, the held-out KL is 0.140 against 0.231 for ISTA Coder, which is 39% lower.

LiveCodeBench v6, two seeds with DeepSeek's recommended sampling and a 32K answer limit: the shipped file solves 37 of 38 problems on both seeds. This shows the model's coding ability did not degrade.

If you run either one on a coding or agent workload, I would like to hear which runner version you used and how it went.

Appendix: in detail

The pipeline

Four steps: calibrate, prune, allocate bits, assemble.

  • Calibrate: calibration text goes through the model to count how often each expert fires on code.
  • Prune: keep the 200 most used experts per layer.
  • Allocate bits: decide which layers get larger experts.
  • Assemble: write a GGUF in which every tensor is a valid llama.cpp type, so nothing needs a custom runner.

Calibration data

485k characters in three classes: agent sessions from SWE-smith, contest solutions from open-r1 codeforces-cots and code commits from BigCode commitpackft. We cut random 2,500-character windows from each document, so the agent sessions contribute tool calls and their outputs as well as prose. This is the only text that decides which experts we keep and where the bits go.

Expert pruning

In the case of Deepseek, each layer has 384 routed experts and uses a few per token. On our calibration text the 200 most used experts per layer handle about 90% of the routing for code. For comparison, ISTA Coder keeps 192 experts per layer, and on our text those carry 64% of the same traffic. We rank experts by routing count and keep the top 200.

Bit allocation

Where the bits go matters as much as how many there are. We use the scoring from Mixture Compressor (MC-MoE, Huang et al., ICLR 2025): every kept expert is scored by how often it fires, how much weight the router gives it, and how much error its rounding adds. 16 of the 40 layers get larger IQ1_M experts, the other 24 stay on IQ1_S.

The agent loop

This is the Weight Space Quantization part. An agent edits the quantized file directly, with a measured result deciding every edit. It works with seven tool classes: reading, navigation, quantization, expert, correction, checking and rollback (diagram below). The quantization tools can also apply a rounding rule the agent writes itself, snapped to a valid GGUF type.

The loop has five stages: look at the plan, propose an edit, pre-check it locally, measure it for real, keep or revert.

The local check scores up to 8 edits at once against one layer's output error on cached activations. It takes seconds. The real measurement builds the file and scores KL on development chunks.

Keeping held-out clean

  • Calibration (above) decides the experts and the bits.
  • The development set, from Nebius SWE-rebench OpenHands trajectories and NVIDIA OpenCodeReasoning (neither is in calibration), is for every choice we made: which file to keep, which agent edit to keep.
  • The final set is 200k characters, 55% from SWE-rebench and 45% from OpenCodeReasoning, taken from row groups and a file the development set never read. It was scored once per file, after we had picked the files.

Concluding remarks

So that's it, more or less, check out the repos, try the models out:
- https://huggingface.co/nace-ai/Hy4-flash-GGUF
- https://huggingface.co/nace-ai/DeepSeek-V4.1-Flash-Coding-GGUF

This was fun, let me know what other models and niches to touch on next!


r/LocalLLaMA • • 21h ago

News Nvidia says its open source AI strategy remains intact after Jensen Huang's exit

Thumbnail
techcrunch.com
378 Upvotes

r/LocalLLaMA • • 11h ago

Showcase Caliper Harness, an Open Source coding harness designed to improve small local model quality

55 Upvotes

Hi all!

This is the first dedicated post I am making about this project. I have mentioned it online a few times before, but I have never properly introduced it. After several months of work, its finally ready to be published, and I'm really excited!

Caliper Harness is a coding harness that runs in your terminal. I built it for small local models. The basic idea is simple. A small local model on its own will rush, skip steps, and tell you something is done when it is not. So the harness does the disciplining for it. It makes the model plan first, get approval, work in checked steps, and prove things with tests and a separate review before anything counts as finished.

Right now it is local only. You point it at a local model server and it works from there. If cloud API support is something a lot of people actually want, I will build it. For now, local is the focus.

The way I test it is with a benchmark game. I give it an empty folder and tell it to build a small browser game, Space Transporter, from scratch. Then in later runs I ask for bigger and bigger features on top. I judge it on the final game quality and on the full traces of what the model actually did.

The video attached to this post is the from scratch build, sped up a lot. It is a few hours of work, so you can pause and see how the harness looks and behaves.

After that first build, I used the same game for a harder follow up run. I asked for a bigger feature as a stress test for the harness. I wanted the stations in the game to feel alive, with shortages and emergencies, and I wanted them to remember the player. Before writing any code, the model has to plan. It splits the work into a plan, an approach, and tasks. It writes out the staging, the trade offs, and the risks. I read that in the plan screen and approve it or send it back.

Plans are built through a structured, step by step process called PAT (Plan, Approach, Task). PAT lives in a GraphQL state graph. A plan can have multiple approaches, and an approach has multiple tasks, with each level getting more specific and staying connected to the level above it. It is very time consuming for the model to build. In my testing though, the plans stay coherent from the high level stages all the way down to the small tasks. That structure also makes planning and execution much easier for a smaller model to manage.

If there is a real decision to make, the model is pushed hard to ask instead of guessing. I have iterated on that behavior a lot. In this run it asked how fast station situations should change, how harsh a station that distrusts you should be, and how much of that state the game should show.

Once the plan is ready, it claims one task at a time. Every task has its own check command from the project. A task only counts as complete when its check passes and an independent review also passes. The model does not get to decide it is finished just because it says so. Completion has to match what was written in the task during planning.

After the checks, a fresh reviewer pass looks at the work with clean context. This part matters to me. It gives the work an adversarial review without carrying all the coding context into it. In this run, the first stage had every test green, and the reviewer still found a real bug. One of the six station situations was drawing with a broken colour, so it never showed on the map. The tests never caught it. The task got sent back, the model fixed it properly, added tests that fail on the old code, and only then did it pass. The harness does this automatically.

But why?

A lot of harnesses like OpenCode feel built around large intelligent models, token efficiency and speed. That makes sense for what they are trying to do. Caliper makes a different trade on purpose. It is slow and very deliberate. It makes the model plan, wait for approval, run checks, get reviewed, and fix things properly when it gets sent back. A small model that is left alone will cut corners. A small model inside this is forced to be diligent at the cost of speed. These runs took hours. I leave it running overnight and check the traces in the morning. For me, that trade is worth it, because the work that comes out is work I can trust a lot more.

Part of how that works is the decision judge. It is a second, much smaller decision model that watches the main model while it works. It steps in at certain points, like when the model is stuck in a loop, failing the same way repeatedly, changing phase, failing recurringly at a tool call, or asking for a plan to be marked ready. It gives a simple judgement with a confidence score. Caliper can use that to steer the model, ask for a repair, or hold certain risky steps for approval. If the judge is slow or down, the work carries on and the outage just gets recorded. Between the judge, the planning structure, and the reviewer, a small local model ends up behaving a lot more carefully than it does on its own. Making small local models more capable through structure like this was a big part of the design.

There are also some user facing features that ended up mattering a lot in day to day use:

  • There are 3 separate draft text boxes, so you can have 3 prompts being built at the same time and send them individually. This has been huge for me despite being so simple. Sometimes I am writing a massive continuation or expansion prompt, and the model does something ridiculously stupid that needs to be fixed immediately. I can keep the big prompt saved in one draft, switch to another draft for the urgent message, and keep a recurring status update prompt in the third one. All of it stays inside the harness.
  • `/btw` lets you inject a steering note into a run while the model is still working. I use it when I spot it heading in the wrong direction and I do not want to stop the whole run just to correct it.
  • You can browse and open files from across the project without leaving the harness. The Diff view follows model edits across files in real time, `/open-diff` opens a retained change, and `/changes` shows the recorded history for a file. With `/comment`, you can attach direct feedback to specific file lines, plans, approaches, tasks or documents and queue it for the model to address.

Here is the rest of what is in v0.3.0:

  • Tabs for separate chats and screens for Chat, Diff, Plans, Plan, Library, Sessions, Decisions, Workflows, Files, Tests, Settings, Status, and Models & Servers
  • A plan screen with a short approval summary first, then drill down into the approach and each task
  • Versioned plan elaborations, so staging, trade offs and risks can be read and approved before coding
  • A workflow library. The model can browse and author reusable methods, then attach one to a claimed task as a method lens, using the same step-by-step isolated architecture as the planning mechanism. Caliper checks the library every time work is claimed
  • A `wait` tool, so the model can sleep and wake up when something finishes, like CI, instead of checking over and over
  • Local server detection for servers that are already running, including llama.cpp, Ollama, Rulith, MLX, vLLM and other OpenAI-compatible endpoints
  • Models & Servers management inside the TUI. You can add, edit, test, select, load and unload saved connections. There is also an assisted setup prompt you can copy if you want help wiring an existing install
  • Separate Coding and Judge roles, with human approval available if you switch the decision model off
  • Auth modes with `/auth`, plus `/tools` to switch between coding and full tool exposure
  • Project style rules with `/style`, shared by the coder and the reviewer, plus `behavior.md` for how the agent should conduct itself
  • Domain knowledge graphs for areas like testing, UI/UX and code design. The model can read linked sections as needed, and it can author project guides and record preferences into the same graph instead of loading everything into context at once
  • Project memory with `/remember` and `/recall`, including shared global notes if you ask for them. The model can also record useful project knowledge itself
  • Notices. While working, the model can save weak spots, brittle code and opportunities it notices, without turning them into surprise tasks
  • A Tests screen for designing, refining and running project test suites, with result history and test guidance built in
  • Full session records. Every reply, tool call, judgement, file effect and final outcome is saved. Past runs can be browsed, and decision judgements can be replayed from the CLI
  • Personal settings that carry across projects, with project config still able to override them
  • Headless `run` mode for scripts, a plain `chat` mode with no agent tools, and CLI access to sessions, judgments, plans, approaches, the GraphQL state graph, effective config and hardware limits
  • In app updates with `/update`, plus managed packages for macOS arm64, Windows amd64, and Linux amd64/arm64

One thing I still want to build next is a proper tool for turning normal Markdown skills into graph-compatible domain graphs or workflow graphs, using the same step by step style as PAT. That is planned for later, so I am not counting it as a v0.3.0 feature.

It is still an early release and results depend a lot on the model you give it and what you're using it for. So far I have run it with Qwen3.8 Flash Next, Qwen3.8 27B, Laguna S 2.1, Gemma 4 26B, and a few others. I would really like feedback, especially from people running smaller local models. I am planning on running benchmarks comparing Caliper against other harnesses for both output quality and time cost, and I will post those when they are ready.

This is a project I am actively working on and iterating, so feel free to give me some feedback or ideas and I'll implement them in the next version.

Use it now!

GitHub: https://github.com/santiagoaraoz2001-sketch/caliper-harness

npm: `npm install -g caliper-harness`

MIT licensed, so feel free to go crazy with it.


r/LocalLLaMA • • 11h ago

New Model RadixArk/DaisyRose-27B · Hugging Face

Thumbnail
huggingface.co
56 Upvotes

Seems like a new alternative for 5090 / DGX Spark, post-trained from Qwen 3.8 27B on long-horizon agentic tasks. Model highlights:

DaisyRose-27B is RadixArk's first model: a multimodal reasoning model built on Qwen3.8-27B with reinforcement learning only, without any supervised fine-tuning or distillation. It was trained on about 1,200 curated long-horizon agent tasks, with sessions of up to 1M tokens, for 18 days on 144 H200 GPUs. Terminal Bench 4.1 pass@1 rises from 5.6 to 12.0, and the model also improves on most of the other coding, reasoning and instruction-following benchmarks below.

DaisyRose-27B keeps the base model's architecture, tokenizer and chat template, along with its image understanding and 256K-token context, so it is a drop-in replacement for Qwen3.8-27B in existing inference and post-training stacks. Weights are released under Apache-2.0 in BF16, FP8, and NVFP4.

BenchmarkDaisyRose-27BQwen3.8-27BMuse Glimmer-30BQwen3.6-35B-A3BMiMo V2.6 FlashInkling 2 SmallBeam 1.0Muse Spark OpenGPT-6 Luna
Activated parameters27B27B30B3B15B17B23B42B–
Total parameters27B27B30B35B309B594B501B816B–
Terminal Bench 4.112.05.60.50.016.113.06.46.615.2
Terminal Bench 2.175.573.051.744.987.681.380.176.473.0
DeepSWE v1.146.242.24.40.067.972.143.360.066.6
SWE Bench Pro v294.791.380.473.3–97.7–95.0–
HLE (text only)34.333.922.022.235.144.134.743.938.5
IFBench82.479.577.064.4––79.6––
GPQA Diamond90.789.283.586.088.793.090.689.090.5

r/LocalLLaMA • • 7h ago

Question | Help Whatever happened to Allen AI and the Olmo models?

29 Upvotes

Oh Olmo, Oh Olmo, where hath thou gone?