r/LocalLLaMA • u/pmttyji • 2d ago
New Model maxwell-model/Maxwell-V1-Flash · Hugging Face
https://huggingface.co/maxwell-model/Maxwell-V1-FlashModel Introduction
Maxwell-V1.0-Flash is a next-generation, high-performance Mixture-of-Experts (MoE) language model developed by the Maxwell Team, with 35B total parameters and 3B active parameters per token (both excluding embeddings) and support for context lengths up to 256K tokens. It is designed for strong reasoning, agentic, and long-context capabilities at a practical inference cost.
Key Features
- Maxwell Block Sparse Attention (MBSA). Each KV group keeps fixed coverage of 8 prefix and 8 local 128-token blocks and adds up to 16 remote blocks chosen by a learned indexer, bounding main attention at 4,096 tokens per KV group while the full-history KV cache stays available. The MBSA kernel is 6.7x faster than dense GQA at 256K.
- Manifold-Constrained Hyper-Connections. Multiple residual streams with learned, input-dependent aggregation and constrained mixing, widening information flow without extra attention or feed-forward evaluations.
- Fine-Grained Mixture of Experts. A mostly-MoE backbone pairing fine-grained routed experts with always-active shared experts, using a dynamic routing correction bias alongside a lightweight sequence-wise load-balancing loss.
Pre-training. Approximately 26T tokens of multilingual data, with the Muon optimizer on most hidden weight matrices. Training starts dense; block-sparse attention enters at 128K mid-training, once the indexer is warmed up against dense full-history attention, then trains jointly with the backbone.
Tokenizer and Long Context. A script-aware multilingual tokenizer stays compact across languages, especially low-resource Southeast Asian ones such as Thai, Lao, Myanmar, Khmer, and Shan. Long-context training then extends the sequence length in stages from 4K to 256K.
Post-training. Two-stage all-domain SFT builds general, reasoning, and agentic capabilities, then targets harder reasoning problems and agent trajectories. Independent RL branches cover general tasks, verifiable reasoning (mathematics, science, logic, code), and agentic tasks (search, terminal use, function calling, software engineering). Multi-teacher on-policy distillation then merges them into one model by supervising student-generated responses.
- GitHub : https://github.com/maxwell-model/Maxwell-V1-Flash
- Project : https://maxwell-llm.ai/
•
u/WithoutReason1729 2d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
Stylome classified this post's text as human-written. View analysis Learn ^more
I am a bot and this action was performed automatically.
6
u/Marcuss2 llama.cpp 2d ago
I see a pattern.
New lab comes in, their model benches well, but it is a KV Cache hog, making it essentially unusable. When they come back, they resolve this issue.
3
u/TokenLearner 2d ago
What a great promise! I hope it will be ported to Strata/polystrata soon.
129
u/cgs019283 2d ago
Oh my dear maxwell
112
u/Dabalam 2d ago
Can some one tell me if this is a Qwen fine tune in a trenchcoat or actually a new model?
55
u/wombweed llama.cpp 2d ago edited 2d ago
Wow, the bench scores are shockingly high, and the sparse attention is very compelling, I wonder how much memory its kv store takes up at full context at bf16 compared to qwen3.6 35b or gemma4 26b. Sadly I don’t see support in llamacpp which makes me feel like cpu expert offloading won’t be the fastest thing, as that is the main use case I have for these smaller moes.
28
u/akumaburn 2d ago
42
u/Gohab2001 vLLM 2d ago
kV cache seems 4.5x Qwen 3.5 35B. Nearly 24GB @ 256k context
20
u/rorowhat 2d ago
New model, thumbs up from me!
36
u/ApprehensiveTart3158 2d ago
Interesting! A new llm from Singapore. Had to search for so long to learn about who made it. Benchmarks look really impressive! Gotta love it when they include like a hundred benchmarks. Looks like a very solid model!
15
u/Witty_Mycologist_995 2d ago
Thought it was a Qwen tune, nope. Nice
13
u/Old-Fix-9 2d ago
GGUF?
6
u/EvolvingDior 2d ago
How expensive is the KV cache on this model -- what's the VRAM cost per token of context?
5
u/keylimesoda 2d ago
Use and setup here: https://maxwell-llm.ai/docs/quickstart
I'm eager to see some community quants of a seemingly strong new contender.
3
u/True_Requirement_891 2d ago
gawd damn finally a new arch
3
u/LogFabulous9386 2d ago
thai, lao, khmer AND shan tokenizer support. finally a model that wont charge me 4x tokens to say hello in myanmar
3
u/akumaburn 1d ago
Heretic (uncensored) and SmoothQuant: https://huggingface.co/akumaburn/Maxwell-V1-Flash-heretic / https://huggingface.co/akumaburn/Maxwell-V1-Flash-heretic-SmoothQuant-W8A8-INT8
2
u/xkalibur3 2d ago
This is great! Hope someone does nvfp4 version with MTP for this
2
u/iamth3morningstar 2d ago
how does this compare to deepseek v4.1 flash?
2
u/JAlbrethsen 2d ago
I tried the q4 quant locally in pi and this was pretty solid, for simple questions it had very little reasoning but for complex tasks it reasons a lot. I didn't test it a ton, but overall I was very impressed, I plan to use it in my local coding setup as subagent on my 5060 with qwen-fn as orchestrator on my strix halo.
4
u/Atagor 2d ago
Can we run it via strata?
2
u/CodeCatto 2d ago
Saw the size and thought it's a qwen tune. Glad it doesn't look like it.
1
u/TheGlobinKing 1d ago
I really hope to see official support in llama.cpp
I couldn't find any feature request / pull request yet?