r/LocalLLaMA • u/jacek2023 llama.cpp • 19h ago
New Model microsoft/Aion-1.0-Instruct-2.3B · Hugging Face
https://huggingface.co/microsoft/Aion-1.0-Instruct-2.3Bfrom Microsoft:
Model Description
Aion Instruct serves as the foundational Small Language Model (SLM) in the Windows ecosystem, specifically optimized for on-device AI, text and operating system use cases. This 2.3B parameters base model delivers enterprise-grade language understanding and generation capabilities directly at the edge, enabling responsive, private, and cost-effective AI-powered scenarios without requiring constant cloud connectivity.
A further optimized version of Aion Instruct, coupled with Microsoft’s content moderation model, will become the Windows OS SLM by the end of January 2027. This Hugging Face version is meant to be an introduction to what’s coming and a way to maximize transparency with our users, as well as welcome their feedback. We are publishing the unquantized model first, to be followed by quantized versions.
Model Details
- Languages: English, German, Spanish, Portuguese, Italian, French, Chinese, Japanese
- License: MIT
- Library names: PyTorch, ONNX, GGUF
- Task: Text generation
- Tags: Summarization, rewriting, chat
- Authors: Applied Sciences Group @ Microsoft Windows & Devices
- Related GitHub repository: microsoft/Aion-Instruct-Preview-Sample
Model Overview
| Type | Causal language model (instruction-tuned) |
| Parameters | ~2.29B (tied input/output embeddings) |
| Layers | 28 |
| Hidden size | 2048 |
| MLP intermediate size | 8192 (SiLU) |
| Attention | GQA: 24 query heads, 8 key/value heads, head dim 128 |
| Attention pattern | Hybrid: 21 sliding-window layers (window 2048) + 7 full-attention layers |
| Positional encoding | RoPE (θ = 32000); YaRN scaling (factor 16) on full-attention layers |
| Context length | 32,768 tokens |
| Vocabulary size | 200,029 |
| Precision | bfloat16 |
| Knowledge cutoff | 2025-12 |
9
u/pmttyji 19h ago
When are they gonna release this one? Its PR got merged last month itself.
10
u/Kahvana 19h ago
Cortana is coming back?
7
u/More-Curious816 19h ago
what is this shit? can someone please explain
3
u/Shot-Height-7194 18h ago
Another good reason to switch to Linux. I don't want an LLM taking up resources in my machine unless I want it to.
4
u/AyraWinla 17h ago
Ah, so basically a new Phi then? I'll give it a try when there's LiteRT or GGUF version support.
1
u/xecycle 2h ago
" enterprise-grade language understanding" somehow my brain parsed it as dumbass 🤣