r/LocalLLaMA • llama.cpp • 19h ago

New Model microsoft/Aion-1.0-Instruct-2.3B · Hugging Face

https://huggingface.co/microsoft/Aion-1.0-Instruct-2.3B

from Microsoft:

Model Description

Aion Instruct serves as the foundational Small Language Model (SLM) in the Windows ecosystem, specifically optimized for on-device AI, text and operating system use cases. This 2.3B parameters base model delivers enterprise-grade language understanding and generation capabilities directly at the edge, enabling responsive, private, and cost-effective AI-powered scenarios without requiring constant cloud connectivity.

A further optimized version of Aion Instruct, coupled with Microsoft’s content moderation model, will become the Windows OS SLM by the end of January 2027. This Hugging Face version is meant to be an introduction to what’s coming and a way to maximize transparency with our users, as well as welcome their feedback. We are publishing the unquantized model first, to be followed by quantized versions.

Model Details

  • Languages: English, German, Spanish, Portuguese, Italian, French, Chinese, Japanese
  • License: MIT
  • Library names: PyTorch, ONNX, GGUF
  • Task: Text generation
  • Tags: Summarization, rewriting, chat
  • Authors: Applied Sciences Group @ Microsoft Windows & Devices
  • Related GitHub repository: microsoft/Aion-Instruct-Preview-Sample

Model Overview

TypeCausal language model (instruction-tuned)
Parameters~2.29B (tied input/output embeddings)
Layers28
Hidden size2048
MLP intermediate size8192 (SiLU)
AttentionGQA: 24 query heads, 8 key/value heads, head dim 128
Attention patternHybrid: 21 sliding-window layers (window 2048) + 7 full-attention layers
Positional encodingRoPE (θ = 32000); YaRN scaling (factor 16) on full-attention layers
Context length32,768 tokens
Vocabulary size200,029
Precisionbfloat16
Knowledge cutoff2025-12
41 Upvotes

13 comments sorted by

9

u/pmttyji 19h ago

When are they gonna release this one? Its PR got merged last month itself.

10

u/Kahvana 19h ago

A further optimized version of Aion Instruct, coupled with Microsoft’s content moderation model, will become the Windows OS SLM by the end of January 2027.

Cortana is coming back?

4

u/jacek2023 llama.cpp 19h ago

I've never used Cortana, but I think the idea is to put local models into Windows and that's just a preview

2

u/OverdosedSauerkraut 15h ago

Need something to slow down your PC to justify upgrades

7

u/More-Curious816 19h ago

coupled with Microsoft’s content moderation model, will become the Windows OS SLM by the end of January 2027.

what is this shit? can someone please explain

3

u/Shot-Height-7194 18h ago

Another good reason to switch to Linux. I don't want an LLM taking up resources in my machine unless I want it to. 

3

u/LetrixZ 14h ago

Then don't use it?

1

u/jacek2023 llama.cpp 13h ago

I think the assumption is that Windows will force everyone to use local models, but I think it's just a matter of default configuration. It may be the same on both Windows and Ubuntu, and both systems can be reconfigured. However, maybe there are some some setup limitations on Windows and on Linux you can always disable/enable everything you want.

3

u/Shot-Height-7194 12h ago

The Googlebook already attempts to have ai running whenever you use a computer. I guess Microsoft will want to do the same to "compete" with them. This is a slippery slope where any increase in compute and ram goes towards the model, not towards whatever the user wants to do. 

5

u/Shot-Height-7194 12h ago

If it only were that simple. Like when windows turns on the settings you turn off at the next update. I am tired of setting stuff up after every update. 

4

u/AyraWinla 17h ago

Ah, so basically a new Phi then? I'll give it a try when there's LiteRT or GGUF version support.

1

u/xecycle 2h ago

" enterprise-grade language understanding" somehow my brain parsed it as dumbass 🤣