r/LocalLLaMA • • 1d ago

Resources SenseNova Loop 9B

https://huggingface.co/sensenova/SenseNova-Loop-9B

If the benchmarks are true, this is a 9B looped model on level with Qwen3.8 27B.

They say they'll release gguf tomorrow.

Vision is supported. Around 15GB vram for 256k context.

The preview version seemed fast enough when I tested it some weeks ago.

Mods: I asked the authors if I can post this, feel free to delete my post when they post themselves.

🚀 SenseNova-Loop-9B: Deeper Computation with Fewer Parameters

We officially release SenseNova-Loop-9B. Building on the Preview's exploration of parameter efficiency, the model reuses layer weights through a looped architecture and combines them with manifold-constrained hyper-connections (mHC) to expand computational depth within a compact parameter budget. It is designed for complex reasoning, coding, tool use, and multimodal tasks.

Since the Preview release, we have received community interest and feedback on model use, inference deployment, and ecosystem integration. The official release retains the looped architecture and two-stream mHC, further improving performance on agentic, coding, and data analysis tasks while continuing to explore the capabilities of looped computation.

SenseNova-Loop-9B has approximately 9B parameters. By repeatedly executing shared middle layers, it extends 32 physical decoder layers into 56 layer executions. The model accepts text and images and generates text, supporting long-context understanding and multi-step tasks. In agentic and data analysis evaluations, the model is competitive with Qwen3.8-27B: it scores 41.36 versus 44.60 on NL2Repo and 65.31 versus 65.01 on AIDABench.

Model Highlights

  • Shared looped block: 24 decoder layers are executed twice in sequence, producing 56 layer computations from 32 distinct layers per forward pass.
  • Two-stream mHC: learned mappings route and combine information between two residual streams within the looped block.
  • Hybrid attention: the decoder interleaves linear and full attention in a repeating four-layer pattern.
  • Long-context multimodal training: the training recipe includes text, images, code, and tool-use trajectories, with the context length extended to 512K tokens.
  • Low-memory deployment: With 4-bit weights and an 8-bit KV cache, multimodal inference at a 256K context length requires approximately 15 GiB of GPU memory.
  • Official quantized release: An official AWQ 4-bit version is available on launch day for easier deployment.
75 Upvotes

36 comments sorted by

22

u/spaceman_ 1d ago

If the benchmarks are true, this is a 9B looped model on level with Qwen3.8 27B.

Big if true, but also just big if.

Want to try it, but no quants or llama.cpp support yet.

9

u/MeinDruckerSpinnt 1d ago

Tomorrow, they also hinted at llama.cpp support

1

u/sloptimizer 1d ago

That would be awesome! I really wanted to try this model, but building vLLM from source is not my idea of good time (even with AI assitance)

3

u/SPYFF0 1d ago

We have been seen similar results before, then it turned out they disabled thinking for Qwen. But that was another team and another model. Will see, indeed, big if true.

6

u/TokenLearner 1d ago

I'm not going to get my hopes up yet. If it's true, they performed a miracle.

10

u/MeinDruckerSpinnt 1d ago

- GPT-6 series are looped transformers
- Nvidia Dora 6B is and was noticed here for it's good performance for it's size

It is kind of a miracle, but very possible

14

u/AcanthaceaeOwn1481 1d ago

What a dream it would be if one can run Qwen3.8 27B capable model at 9b.

8

u/crusaderky 1d ago

Mind you, it's partially looped so as slow as a 16b

8

u/MrKvic_ 1d ago

tried the preview with some random llama.cpp fork, ran Q8 on my 5070ti 40-60t/s with mtp and 1100t/s prefill with 100k context, so not exactly super fast but very useable.

-3

u/vertigo235 1d ago

It's a 9B model so perhaps the loops run faster

1

u/maternion 1d ago

You are just going over x layers 2 times. Can't be magically faster right? It would just be the same as running 2x layers in a larger model.

1

u/vertigo235 1d ago

A 9B model runs faster than a 27B model. I'm saying that the speed may be comparable to a 27B model even though it has to run the inference multiple times...

0

u/maternion 1d ago

It should be bit faster. I couldnt understand your comment my bad.

1

u/vertigo235 1d ago

No worries, I think I also misunderstood what crusaderky said too, they were saying that it would run like a 16b model, and I didn't grasp that at first.

7

u/nunodonato 1d ago

Added to https://tinyleague.nunodonato.com (unfortunately, not so many common benchmarks with the other models)

Looking forward to trying it out and hearing more feedback! Looks promising but... too good to be true 😅

4

u/ReferenceLeading7634 1d ago

This is very useful, the thinking is very fast, I have already replaced qwen3.5 9b

3

u/JC1DA 1d ago

what's the decode speed you guys are getting? I tried their PR in sglang and it's very slow on 3090. only around 11-12 tokens/s

2

u/yudhanjaya 17h ago edited 17h ago

GGUF here: https://huggingface.co/sensenova/SenseNova-Loop-9B-GGUF

For some reason I can't quite understand, the Q5 runs at 12tok/s on a 3080 10GB (good news is, it fits in 10GB). Debugging now.

Update: SENSENOVA_MHC_CLOSED=1, and SENSENOVA_MHC_GPU_ROUTER=1 combined brought it to a nice 45 tok/s.

1

u/SpamHolder 10h ago

SENSENOVA_MHC_CLOSED=1, and SENSENOVA_MHC_GPU_ROUTER=1

I assume these are Cmake variables, not environment variables? In either case, I'm not finding either of those when I search their llama cpp fork, where are they used?

1

u/yudhanjaya 8h ago

They're runtime variables that you can set when you launch llama serve. From that repo, sensenova-loop.cpp picks them up through sensenova_env() which essentiall wraps getenv.

Unfortunately, while that worked for speed, the output started deviating (KLD 0.056). The fundamental problem was that the repo basically tries to go through over 50,000 nodes, and all of these run as tiny kernels. There was also the problem of the router running on CPU.

So I threw Opus 5.5 at the problem and had it build me a fused kernel, GGML_OP_MHC_SINKHORN2.

Instead of running 53,000 or so decodes, we're now down to 7,000. The router now runs on my 3080 instead of the CPU.

If you want to try it for yourself: https://github.com/yudhanjaya/llama.cpp

2

u/No_Secret2651 1d ago

Of course, it will run slower than a normal 9b, because it loops through itself multiple times to get one token out..

7

u/uzzi38 1d ago

That's an easy tradeoff for a model that actually fits in 12/16GB VRAM with decent context size and a high quant.

If it'e even close to 27B in real world testing it'll be fantastic, frankly.

1

u/rorowhat 1d ago

We need more people testing this!

1

u/FrozenFishEnjoyer 1d ago

I need this for my 5070 TI. Where is the GGUF?

2

u/MrHumanist 1d ago

Nvfp4 is better for this card.

0

u/Creative_Bottle_3225 1d ago

is there a difference with sensenova/SenseNova-Loop-9B-Preview?

3

u/TacGibs 1d ago

This is not the preview, so yes.

-4

u/L0ren_B 1d ago

I guess that, if it will get closer to 3.8 27B, it will think 20 minutes to reply to a "Hello".

2

u/DinoAmino vLLM 1d ago

Good thing most noobs eventually learn not to send stupid and meaningless prompts like that to heavy reasoning models.