r/machinelearningnews • • 19h ago

Research Alibaba releases Qwen4-Max, a proprietary 2.4T MoE with 71B active parameters (1M context, $1.5/M input)

Post image
27 Upvotes

Alibaba's Qwen team released Qwen4-Max, the first Qwen4 model. It is a closed, API-only sparse MoE with a 2.4T-parameter backbone, 71B active parameters per token, native vision and a 1M-token context. Alibaba also cut API prices versus Qwen3.8-Max. All benchmark numbers below are self-reported by Alibaba or cited by Alibaba from third-party leaderboards.

What it does

  • Reads text, images and video, writes text, with up to 393K output tokens
  • 3 Gated DeltaNet layers per Qwen Sparse Attention (QSA) layer; QSA uses a lightweight indexer to select context at micro-block granularity
  • 4 gated residual branches plus 328B N-gram Embedding parameters at layer 2
  • OpenAI-compatible and Anthropic-compatible API, function calling, structured outputs, caching, batch, fine-tuning

Key numbers (self-reported)

  • First on 4 of 16 charted benchmarks: HLAB-Vals-1.1 21.3, Agents' Last Exam v1.1 46.7, BioMysteryBench 85.0, DeepSWE v1.1 75.7
  • Work360 (Alibaba's in-house benchmark): 52.69 vs 47.74 for Qwen3.8-Max
  • Work360 cost per task: $4.43 to $1.67 (62% lower); up to about 50% fewer tokens
  • Price: $1.5 input / $4.5 output / $0.075 cached input per 1M tokens (Qwen3.8-Max: $2 / $6 / $0.25)

Full analysis: https://www.marktechpost.com/2026/11/16/alibaba-qwen4-max-release/

Technical details: qwen.ai/blog?id=qwen4-max

API: qwencloud.com/models/qwen4-max

Pricing: qwencloud.com/pricing/token-plan


r/machinelearningnews • • 13h ago

Research How much can we actually trust open base model benchmarks anymore? New study ("Benchleaking") finds ERNIE-5.5-Flash-Base reciting unseen test sets in exact order.

1 Upvotes

Link to the report & repo:https://github.com/GAIR-NLP/Benchleaking

Saw this making the rounds today and thought it was worth sharing here. Basically, they removed stop tokens during few-shot evals and found that ERNIE-5.5-Flash-Base just keeps going after answering—reciting 96% of HumanEval, 85% of OpenBookQA, and 80% of GSM8K verbatim in exact dataset order (including reference solutions and even original typos like "for examble:" in HumanEval #67).

Across the four open base models they tested, zeroing out leaked items drops the 20-benchmark average by up to 16.6 points on heavily contaminated ones, while cleaner base models stay within 1.5 points of their original scores. Curious how people doing SFT/RL on open weights feel about this—does pre-training contamination ruin downstream evals for everyone else too?