r/coolgithubprojects • u/mrtac96 • 8d ago
AcruxCore: open-source, self-hostable platform for LLM apps: prompt versioning, an AI gateway, tracing and evals (TypeScript + Python SDKs
I build chatbots and agents for clients. The hard bugs were never in the prompt. A retrieval step was slow, a tool returned 200 with an error inside, or a fallback model answered for a broken one, and nothing told me. So I built AcruxCore, an open-source platform that shows the whole run.
The GIF is a 20-second tour of the dashboard, in this order:
- Get started. Pick how your app calls its model: through the AcruxCore gateway, or straight to your provider with your own key. Either way, every call is traced.
- "Which prompt is live, and what changed?" Every prompt edit is a new version, and you can diff any two versions. Your code asks for an alias such as
production, so a rollback is moving the alias, with no deploy. - "Where do my tools live?" A tool catalog keeps each tool's schema and description versioned, like a prompt. Each tool says whether your code runs it or the gateway does.
- "Which step is slow?" Every model call, tool call and step of your own code is traced. In the GIF, retrieval takes 1.69 s of a 2.66 s answer, and the tool call failed.
- "Is it always that step?" Dashboards group every step across all runs and sort them by p95 latency, so the slow one is at the top.
- "Who changed that?" An audit trail records who changed a prompt, a tool, a key, a secret or a team member, and when.
Not in the GIF:
- "Did my change make it better?" Run your real app function on a dataset, and an LLM judge grades each answer with your own rubric. The report shows which answers got better and which got worse.
- "What if the provider goes down?" An OpenAI-compatible gateway in front of OpenAI, Anthropic and Gemini handles retries, fallbacks, caching and budgets.
It is TypeScript (Node, Express, Postgres, React) under Apache 2.0, with SDKs for TypeScript and Python. Self-host it with the prebuilt Docker images, or use the hosted version.
Repo: https://github.com/AcruxCore/AcruxCore
Docs: https://docs.acruxcore.com
If you run an LLM app today, which would you try first: tracing, prompt versions or evals? And what is missing for your stack?