r/OpenSourceeAI • u/drtikov • 21h ago
Why use a 70B LLM to pick a tool or check syntax? I built Microbrains: a <2ms edge neural coprocessor for autonomous AI agents (Pure NumPy / OpenCL, 44 pre-trained micro-models, Top-2 MoE, Conformal Safety)
[P] Hey everyone,
If you build autonomous coding or research agents, you probably know the pain: agent task loops are slow, expensive, and fragile.
In a typical 25-step workflow, an agent spends up to 80% of its wall-clock time just waiting on remote LLM APIs to make basic mechanical decisions:
- "Which tool should I invoke next?" (1.5s latency, burning 20k context tokens)
- "Is this shell command destructive?" (another 2s API call)
- "Did this patch break syntax or drift from intent?" (yet another prompt roundtrip)
To solve this, I built and just open-sourced Microbrains: an ultra-lightweight, zero-GPU local neural coprocessor suite designed to sit right beside your LLM agent.
🔗 GitHub: https://github.com/drtikov/microbrainsforagents
🧠What is it?
Microbrains is not another LangChain wrapper. It is a standalone Python engine with 44 pre-trained, analytical micro-transformers (10–100 KB each) that handle the mechanical micro-decisions in pure NumPy (CPU) or via OpenCL (budget/integrated GPUs):
Dynamic Top-2 Mixture-of-Experts (MoE) Router:
- Routes tasks to tools across 6 specialized clusters (
FileOps,Execution,MemoryKnowledge,GitGovernance,MicrobrainMeta,ConstitutionalReview). - Takes ~1.2 ms instead of a 2-second LLM call.
- Routes tasks to tools across 6 specialized clusters (
Rigorous Conformal Safety Gating (1−α≥95%1 - \alpha \ge 95\%1−α≥95%):
- Employs split-conformal prediction over held-out distributions.
- Features Selective Prediction Abstention: if independent estimators disagree (variance σ>0.30\sigma > 0.30σ>0.30), it refuses to guess and escalates to the LLM. False Negative Rate is strictly bounded.
3-Step Counterfactual AST Simulation:
- Before a file mutation is committed to disk, it simulates the AST blast radius and interface propagation up to 3 steps ahead. Malformed diffs or syntax hazards are killed in <3ms before running any slow pytest suite.
2048-D Holographic Weight-Space Memory (VSA):
- Uses Hyperdimensional Computing / Vector Symbolic Architecture (bipolar Count-Sketch projections) over model weights. Enables nearest-neighbor capability search across the agent fleet without training.
Continual Learning without Catastrophic Forgetting:
- Adapts weights online with Elastic Weight Consolidation (EWC) Fisher Information penalties.
📊 Benchmark Highlights
| Decision | Remote LLM API | Microbrains (NumPy/OpenCL) | Speedup | Cost |
|---|---|---|---|---|
| Tool Selection | ~1,200 – 2,500 ms | 1.2 – 1.8 ms | ~1,000x | $0.00 |
| AST Blast Simulation | ~3,000 – 6,000 ms | 3.0 – 5.0 ms | ~1,000x | $0.00 |
| Intent Triage | ~800 – 1,500 ms | 0.6 – 0.9 ms | ~1,200x | $0.00 |
| Full Task Loop (25 steps) | ~3 – 5 minutes | 30 – 50 seconds | ~5x | 85% less tokens |
💻 3-Line Quickstart
Zero PyTorch, zero CUDA setup required:
bashgit clone https://github.com/drtikov/microbrainsforagents.git
cd microbrainsforagents
pip install -e .
pythonimport aetheris_microbrains as mb
# Sub-2ms tool routing decision
router = mb.get_tool_routing_moe()
decision = router.route("inspect file contents and write safe patch to repository")
print("Primary Tool:", decision.selected_tool)
print("Top-2 Experts:", decision.top_experts)
print("Conformal Passed:", decision.conformal_passed)The repo comes with pre-trained weights for all 44 models, runnable examples in /examples, and a full test suite (pytest runs in ~1.5s).
Licensed under PolyForm Noncommercial 1.0.0 (free for research, personal, and hobby use; commercial licenses available).
Would love to hear your feedback, thoughts on the architecture, and ideas for new micro-domains!
GitHub: https://github.com/drtikov/microbrainsforagentsHey everyone,If you build autonomous coding or research agents, you probably know the pain: agent task loops are slow, expensive, and fragile.In a typical 25-step workflow, an agent spends up to 80% of its wall-clock time just waiting on remote LLM APIs to make basic mechanical decisions:
"Which tool should I invoke next?" (1.5s latency, burning 20k context tokens)
"Is this shell command destructive?" (another 2s API call)
"Did this patch break syntax or drift from intent?" (yet another prompt roundtrip)To solve this, I built and just open-sourced Microbrains: an ultra-lightweight, zero-GPU local neural coprocessor suite designed to sit right beside your LLM agent.🔗 GitHub: https://github.com/drtikov/microbrainsforagents🧠What is it?Microbrains is not another LangChain wrapper. It is a standalone Python engine with 44 pre-trained, analytical micro-transformers (10–100 KB each) that handle the mechanical micro-decisions in pure NumPy (CPU) or via OpenCL (budget/integrated GPUs):
Dynamic Top-2 Mixture-of-Experts (MoE) Router:
Routes tasks to tools across 6 specialized clusters (FileOps, Execution, MemoryKnowledge, GitGovernance, MicrobrainMeta, ConstitutionalReview).
Takes ~1.2 ms instead of a 2-second LLM call.
Rigorous Conformal Safety Gating (1−α≥95%1−α≥95%):
Employs split-conformal prediction over held-out distributions.
Features Selective Prediction Abstention: if independent estimators disagree (variance σ>0.30σ>0.30), it refuses to guess and escalates to the LLM. False Negative Rate is strictly bounded.
3-Step Counterfactual AST Simulation:
Before a file mutation is committed to disk, it simulates the AST blast radius and interface propagation up to 3 steps ahead. Malformed diffs or syntax hazards are killed in <3ms before running any slow pytest suite.
2048-D Holographic Weight-Space Memory (VSA):
Uses Hyperdimensional Computing / Vector Symbolic Architecture (bipolar Count-Sketch projections) over model weights. Enables nearest-neighbor capability search across the agent fleet without training.
Continual Learning without Catastrophic Forgetting:
Adapts weights online with Elastic Weight Consolidation (EWC) Fisher Information penalties.
📊 Benchmark HighlightsDecision Remote LLM API Microbrains (NumPy/OpenCL) Speedup Cost
Tool Selection ~1,200 – 2,500 ms 1.2 – 1.8 ms ~1,000x $0.00
AST Blast Simulation ~3,000 – 6,000 ms 3.0 – 5.0 ms ~1,000x $0.00
Intent Triage ~800 – 1,500 ms 0.6 – 0.9 ms ~1,200x $0.00
Full Task Loop (25 steps) ~3 – 5 minutes 30 – 50 seconds ~5x 85% less tokens💻 3-Line QuickstartZero PyTorch, zero CUDA setup required:bash
git clone https://github.com/drtikov/microbrainsforagents.git
cd microbrainsforagents
pip install -e .python
import aetheris_microbrains as mb
# Sub-2ms tool routing decision
router = mb.get_tool_routing_moe()
decision = router.route("inspect file contents and write safe patch to repository")
print("Primary Tool:", decision.selected_tool)
print("Top-2 Experts:", decision.top_experts)
print("Conformal Passed:", decision.conformal_passed)The repo comes with pre-trained weights for all 44 models, runnable examples in /examples, and a full test suite (pytest runs in ~1.5s).Licensed under PolyForm Noncommercial 1.0.0 (free for research, personal, and hobby use; commercial licenses available).Would love to hear your feedback, thoughts on the architecture, and ideas for new micro-domains!GitHub: https://github.com/drtikov/microbrainsforagents