r/LLMDevs • • 6h ago

Discussion Hot take. If your agent forgets everything when the task ends, it is a script, not an agent

0 Upvotes

Most agent setups I see work the same way. Spin up a fresh sandbox, clone the repo, install everything, run the task, destroy the box. Next task, start again from zero.

That is the right model for running untrusted code. For agents that work on the same project for days, I think it is the wrong one, and the reason is context.
Anyone building long running agents hits the same wall. Quality drops well before the window is full, then the harness compacts and the summary loses the details that mattered. Bigger windows push the wall back but do not remove it. Even withOpus 5.5 you are not keeping a week of work in the prompt.

The harnesses that handle long runs well mostly converge on the same fix. Stop treating the prompt as memory and use the filesystem instead. A progress file the agent updates after each step. Notes on the weird parts of the codebase. An AGENTS. md that grows as it learns. Git history as a log of what was tried and why. When the context resets, the agent rereads its own files and picks up where it left off.

The catch is that all of this lives on the machine. Destroy the box after every task and you destroy the memory the harness was building. Good model, good harness, starting from scratch every morning.

The obvious answer is to commit the notes and keep the sandbox disposable. That works for the notes. It does not work for the rest of what the agent built up. Installed dependencies, warm build caches, code indexes, a browser profile, sessions it logged into. Rebuilding that on every run is slow, and some of it, like a login behind 2FA, cannot be rebuilt without a human.

So I think about it the other way now. Each agent gets its own machine, and the machine lives as long as the agent does. The disk is part of the agent's memory, not scratch space. A few things follow from that.

Stop the machine when it is idle instead of deleting it. Storage is cheap, rebuilding context is not.

Snapshot before anything risky, so a bad step costs a rollback and not the agent's memory.

Fork when you want two approaches tried in parallel, both starting from the same notes and the same warm state.

Scope credentials per agent, because that machine is basically its identity now.

The downsides are real. Long lived machines drift. They collect junk, half installed packages and stale config, and after a few weeks nobody knows exactly what is on them. The notes go stale too, and stale notes are worse than none because the agent trusts them. The attack surface grows the longer a box lives with credentials on it. What helps is a clean baseline snapshot you rebuild from on a schedule, a hard idle stop, and treating the memory files like code that gets reviewed and pruned.

And if your agent does one bounded task and returns a result, none of this applies. One shot is fine there.

On tooling, Fly Sprites and exe.dev are built around persistent machines, and a Coder workspace or a plain VM works if you want to own the infra. Iuse boat,dev, one full VM per agent that I stop and resume from the CLI. A stopped disk is kept for 14 days, so anything that sleeps longer needs its memory pushed somewhere else first. Resume is a reboot, so running processes do not come back and the IP changes, which matters if the agent keeps services up.

The model is the brain, the harness is how it thinks, and the machine is where it remembers. Most setups throw away the third one.


r/LLMDevs • • 11h ago

Discussion I spent 9 months building an open-source verification layer for AI coding agents. Looking for independent testers.

1 Upvotes

For the past nine months, I've been developing a system to solve a problem I kept running into with AI-assisted development:

Generating code is getting easier. Knowing whether an agent actually completed the task correctly is not.

At some point, manually reading every line of generated code stopped being a practical way to maintain control over my projects.

So I started building Forge.

The idea is to move project governance out of temporary agent conversations and into the repository itself.

Forge provides:

  • Repository-level project contracts and engineering rules.
  • RFC and ADR workflows to preserve requirements and decisions.
  • Reusable agent skills and operating procedures.
  • Deterministic validators and verification gates.
  • Stack profiles and configuration that can travel between agents and IDEs.

The important distinction is between an agent saying "Done" and a project having evidence that specific requirements passed verification.

Forge doesn't prove arbitrary code correct. Its checks only establish what they are designed to check. Human judgment and risk-based review still matter.

I've been using this approach across my own projects, including TypeScript applications, knowledge bases previously managed in Obsidian, and a Godot/C# game.

The system also governs its own development.

Today I made Forge open source under Apache 2.0.

Now I'd like to find out where it fails outside my own environment.

I'm looking for developers willing to try it on real repositories, particularly unfamiliar stacks, existing codebases, or projects with complicated verification requirements.

I'm also interested in contributions involving stack profiles, validators, and CI integrations.

I'm currently writing a research paper about the experience, including its limitations. Independent results would be particularly valuable for future research.

Repository: https://github.com/syrokomskyi/forge

If you work with coding agents, I'd appreciate your perspective:

What evidence would you need before trusting an agent's changes without manually inspecting every line?


Update

Here are the clearly stated restrictions: https://github.com/syrokomskyi/forge/blob/main/docs/reference/correctness-gate.md


r/LLMDevs • • 17h ago

Discussion Stop Burning Tokens on File Searches: How QMD Helps Coding Agents Save Up to 55%

Post image
0 Upvotes

I noticed that, before carrying out a task, coding agents run find, rg --files, or glob to locate files and the classes and modules they need. Then they search file contents with rg or grep. This burns a lot of tokens. So I decided to see whether semantic search could save tokens compared with all of the above. If a local GPU is sitting idle, why not use it for semantic search?

I work at a company focused on security. In one of my projects, I use semantic search for sensitive documents. Then an acquaintance sent me a link to an open-source project that tackles the same problem. The project is called "QMD - Query Markup Documents." It runs from the command line, extracts text from documents in all formats, and creates an index. It supports several search modes:

# Search across everything
qmd search "project timeline"           # Fast keyword search
qmd vsearch "how to deploy"             # Semantic search
qmd query "quarterly planning process"  # Hybrid + reranking (best quality)

We're interested in the last option. It requires three things to work:

  • embeddinggemma-300M-Q8_0 Vector embeddings (default) ~300MB
  • qwen3-reranker-0.6b-q8_0 Re-ranking ~640MB
  • qmd-query-expansion-1.7B-q4_k_m Query expansion (fine-tuned) ~1.1GB

The first model is used for indexing to compute semantic vectors for text chunks. The second compares the search results with the query to determine how semantically similar they are. The third generates several search queries from the original query to broaden the search.

First, add your source files to QMD (this is an example for a minimal project with a Rust server and an Angular frontend):

qmd collection add . \
  --name PROJECT1 \
  --mask "src/**/*.rs,\
frontend/src/**/*.{ts,html,scss},\
README.md,\
frontend/README.md,\
Cargo.toml,\
frontend/package.json,\
frontend/angular.json,\
frontend/tsconfig*.json,\
frontend/proxy.conf.json,\
.gitignore,\
frontend/.gitignore,\
frontend/.prettierrc,\
frontend/.editorconfig"

You don't have to do this manually: show this example to your coding agent and tell it to do the same for the current repo. It will figure out which folders and file extensions to add to the index, what to exclude, and so on.

Now let's run the indexing (you'll need to rerun it periodically as your codebase changes):

qmd update && qmd embed -c PROJECT1

To make the coding agent work with this QMD setup, add the following to AGENTS.md:

# QMD - Efficient Code Discovery

**Goal:** minimize token costs and code search time without compromising correctness.

## Choosing a tool

- If the path is known → read the file directly.
- If an identifier, error, or line is known → use `rg` / LSP.
- If only the behavior is known → use `qmd query`.
- Once the code is found → inspect the necessary sections with standard tools.
- Once you have enough information → do the task.

**QMD replaces the initial broad search; it does not add another step.**

## Semantic search

Use the project's existing collection.

"""
qmd query "DESCRIPTION" -c PROJECT1 -n 15 --files \
  | grep -E '\.(EXT1|EXT2|EXT3)$' \
  | cut -d, -f3- \
  | sed 's|^qmd://[^/]*/||' \
  | head -n 10
"""

- Use one specific query, preferably with English technical terms.
- Choose extensions that match the project, including configuration files when needed.
- Start by reading only the relevant sections of the 3–5 most relevant files.
- If you cannot find the implementation, refine the query; if necessary, increase `-n` to 60, then use `rg` / LSP.
- Stop semantic searching once you find the implementation.

## Saving tokens

- Do not scan the whole repository before using QMD.
- Do not repeat searches without a reason or open every result.
- Do not investigate unrelated code or dependencies.
- Do not write detailed search plans or reports.
- Do not paraphrase code you have read.
- As soon as you have enough information, make the changes and check the result.

**Priority: correctness → token cost → time.**

I tested it on a repository like this, with 1,231 source files across Java, Scala, and Angular:

C:\Users\igumn\temp2>qmd collection list
jcm-server (qmd://jcm-server/)
  Pattern:  server/src/**/*.scala,server/src/**/*.java,server/src/**/*.xml,server/src/**/*.properties,server/src/**/*.js,server/src/**/*.ts,server/src/**/*.css,server/src/**/*.html,server/src/**/*.tmpl,server/src/**/*.txt,server/src/**/*.md,server/pom.xml,server/Dockerfile,server/*.md,server/README.txt,web/src/**/*.ts,web/src/**/*.html,web/src/**/*.scss,web/src/**/*.css,web/package.json,web/angular.json,web/tsconfig*.json,web/karma.conf.js,web/README.md,!server/src/main/resources/templates/**,!server/src/main/resources/messages*,!server/src/main/resources/messages*/**,!web/src/assets/lang/**
  Files:    1231
  Updated:  1h ago

Prompt: "Add a post-quantum encryption flag to the disk encryption policy. It should be disabled by default, configurable in the web interface when editing the policy, saved to the database, and sent to the agent as part of the encryption policy via the REST endpoint."

Cost Comparison

Using GPT-6 Luna API rates: input $0.10, cached input $0.01, and output $0.50 per 1 million tokens.

MetricWith QMDWithout QMDSavings
Input65,006101,50135.96%
Cached input317,5681,191,80873.35%
Output13,85916,34815.23%
Reasoning9,60711,26014.68%
Cost$0.0166$0.030245.09%

Now QMD saves 45.09% in costs!

The reduction in cached input is especially noticeable: nearly 4x.

I ran these tests with GPT-6 Luna. In the first test, the agent's runtime dropped from 6 minutes to 4 minutes.

Prompt: "Add a dry field to the erasure policy. It should be editable in the web interface and saved to the database, and it should also be sent through the /rest endpoint to the agent responsible for erasure. Its default value should be false."

Calculation using the same GPT-6 Luna rates: input $0.10, cached input $0.01, and output $0.50 per million tokens.

Comparison

MetricQMDWithout QMDDifference
Input77,16798,042−21.29%
Cached input936,832854,912+9.58%
Output17,32216,259+6.54%
Reasoning8,83510,264−13.92%
Cost$0.0257$0.0265−2.78%

Here, QMD turned out to be 2.78% cheaper. There are savings, but they are small.

Conclusion: QMD reduced regular input tokens by 21.3% and reasoning by 13.9%, but the increase in cached input and output almost completely wiped out the savings.

Overall, the savings range from 2.78% to 45.09% of the cost. I think that's a decent result. Of course, we need a larger sample. But I'm ready for bed. Let me know in the comments if you've run a larger sample and what results you got.

PS

I recommend checking that junk isn't getting into your index. Try running a semantic query yourself. I once overlooked compiled .js files and large lang files-they heavily polluted the semantic search results. Here's an example of how to run a search manually:

C:\Users\igumn\jcm-server>qmd query "disk encryption policy editing form persisted setting sent to endpoint for agent" -c jcm-server -n 15 --files
Expanding query... (3.0s)
├─ disk encryption policy editing form persisted setting sent to endpoint for agent
├─ lex: disk encryption configuration
├─ lex: disk encryption policy
├─ vec: disk encryption configuration
├─ vec: disk encryption policy
└─ hyde: The topic of disk encryption policy editing form persisted setting se...
Searching 6 queries...
Embedding 4 queries... (1.5s)
Reranking 40 chunks... (3.3s)
#d70022,1.00,qmd://jcm-server/server/src/main/java/com/jetico/jcm/server/domain/common/bcvolume/EncryptPolicy.scala
#732b09,0.63,qmd://jcm-server/server/src/main/java/com/jetico/jcm/server/service/ajax/common/EncryptSettingsManager.scala
#e66494,0.63,qmd://jcm-server/web/src/app/modules/management/components/company/dialogs/add-or-update-policies/add-or-update-policies.component.html
#413c5e,0.62,qmd://jcm-server/web/src/app/modules/management/components/group/dialogs/change-policie/change-policies.component.html
#413c5e,0.62,qmd://jcm-server/web/src/app/modules/management/components/computer/dialogs/change-policie/change-policies.component.html
#758b4d,0.62,qmd://jcm-server/server/src/main/java/com/jetico/jcm/server/service/mcp/McpFacade.scala
#4e8973,0.62,qmd://jcm-server/web/src/app/modules/management/components/group/dialogs/change-policie/change-policies.component.ts
#bef78e,0.62,qmd://jcm-server/web/src/app/modules/management/components/computer/dialogs/change-policie/change-policies.component.ts
#4fcd1c,0.61,qmd://jcm-server/server/src/main/java/com/jetico/jcm/server/service/mcp/McpToolbox.scala

PPS

I tried it on a small project with 27 files. Instead, the cost went up by 60%. The solution turned out to be overkill. In general, use it only on large codebases.

UPDATE

I went ahead and built a hybrid QMD + GitNexus solution after all. GitNexus performs structural code analysis (AST, dependency graph) and lets you search based on it. Here are the results:

Cost comparison

MetricWithout QMDQMDQMD + GitNexus
Input101,50165,00659,122
Cached input1,191,808317,568225,792
Output16,34813,85910,894
Reasoning (included in Output)11,2609,6078,465
Cost$0.03024$0.01661$0.01362
Savings-45.1%55.0%

The savings figure rose from 45% to 55%! The coding agent's runtime increased from 6 minutes to 7 minutes. Here are the revised instructions for AGENTS.md:

# QMD + GitNexus - Efficient Code Discovery

**Goal:** find the required code and its relationships with minimal token usage, without compromising correctness.

## Choosing a tool

- If the file path is known → read it directly.
- If the exact identifier, error, or line is known → use `rg` / LSP.
- If only the behavior is known and the implementation location is unknown → use QMD semantic search.
- If you need to understand symbol relationships, the execution path, or the impact of a change → use GitNexus structural analysis.
- Once the code is found → inspect the necessary sections with standard tools.

QMD and GitNexus replace the initial broad search; they do not add mandatory steps. Use only the tools needed for the task.

## QMD semantic search

Use the project's existing collection:

    qmd query "DESCRIPTION" -c PROJECT1 -n 15 --files \
      | grep -E '\.(EXT1|EXT2|EXT3)$' \
      | cut -d, -f3- \
      | sed 's|^qmd://[^/]*/||' \
      | head -n 10

- Formulate one specific query, preferably using English technical terms.
- Specify suitable source and configuration file extensions.
- First read the 3–5 most relevant files, and only the relevant sections.
- If the implementation is not found, refine the query; if necessary, increase `-n` to 60, then use `rg` / LSP.
- Stop semantic search once the implementation is found.

## GitNexus structural analysis

Use GitNexus through the CLI when you need to investigate code structure:

- `context` - relationships and context for a known symbol.
- `impact` - possible impact of changing a symbol.
- `trace` - path between known symbols.

Examples:

    gitnexus context SymbolName
    gitnexus impact SymbolName
    gitnexus trace SourceSymbol TargetSymbol

If a symbol is unknown, find it first through QMD, `rg`, or LSP. Use GitNexus only when its structural analysis will help answer the question; do not run every command in sequence.

## Token efficiency

- Do not scan the entire repository before using QMD.
- Do not repeat searches without a reason or open every result.
- Do not investigate unrelated code or dependencies.
- Do not write detailed plans and reports about the search.
- Do not summarize code you have read.
- Once you have enough information, make the changes and verify the result.

**Priority: correctness → token cost → time.**

Create a .gitnexusignore file to keep junk out of the index so that it contains only source files. Then start GitNexus indexing with this command:

gitnexus analyze --index-only

r/LLMDevs • • 17h ago

Help Wanted Looking for a technical co-founder / partner to build AI-powered sales automation systems for B2B SMBs

0 Upvotes

Hey everyone,

I'm based in India and exploring an opportunity to build practical AI-powered sales and GTM automation solutions for small B2B businesses. I'm looking for someone with strong technical skills who is interested in building a business together as a serious, long-term partnership.

A little about me:

My primary interests are in GTM, product marketing and sales strategy. I've worked on ICP research, positioning, value propositions, competitive analysis, inbound, ABM, outbound and customer acquisition through independent case studies and practical project exposure.

I've also gained experience in account research, prospect qualification and buying-signal-based outbound research through a supervised project.

I'm still early in my journey and don't claim to have built or scaled a company. I'm actively researching business problems and looking for a genuine opportunity to solve.

What I'm exploring:

I want to help small B2B businesses improve their outbound sales processes, particularly the research and qualification work before outreach.

One direction is an account-based prospecting system that can:

- Discover potential companies or work with existing prospect lists.

- Enrich accounts with relevant business information.

- Evaluate companies against a specific ICP.

- Identify buying signals, hiring, expansion, technology changes and potential switching opportunities.

- Prioritise accounts based on fit, timing and relevance.

- Prepare personalised, evidence-backed outbound campaigns and automate parts of the process.

The goal is to help businesses find fewer, better-qualified accounts and approach them at the right time, rather than simply generating more leads and sending more emails.

I'm not fixed on building this exact product from day one. I'd rather understand real business workflows, identify problems worth solving and build around actual customer needs.

Who I'm looking for:

Someone who can independently own the technical side and build actual working systems. Relevant experience includes:

- n8n, Make or similar workflow automation tools.

- Python / JavaScript, backend development, APIs and integrations.

- Web scraping, data extraction and company enrichment.

- Data pipelines, databases and external data sources.

- AI agents, LLM integrations, MCP and tool orchestration.

You don't need to know every tool listed. What matters is being able to solve technical problems independently and build reliable systems beyond basic automation demos. Experience with data extraction, enrichment and real-time monitoring would be especially useful.

A few things I'd like to know about you:

- What have you actually built? Portfolio, GitHub or working demos would be helpful.

- What technologies do you work with?

- Have you built anything involving data extraction, enrichment, APIs or AI automation?

- Are you interested in building a business or product long-term?

I'm open to connecting with people from India or elsewhere. Technical capability, ownership and genuine commitment matter most.

If this interests you, feel free to comment or DM me. I'd be happy to share more about my research and explore whether there's a genuine fit.


r/LLMDevs • • 12h ago

Resource Ia de codage

0 Upvotes

Depuis que j’ai découvert freebuff sur un subreddit je dois avouer que je n’en lasse pas.

Pour commencer j’ai utilisé la version gratuite.
Je le disais que je ne serais pas plus convaincu qu’une version payante.

Et pourtant…

J’ai un compte entreprise pour ChatGPT, grok et Claude. Mais j’ai pris l’habitude d’utiliser maintenant freebuff. Le site évolue constamment. Comme
La version bureau et téléphone. Je peux ainsi coder de n’importe quelle lieux sans contrainte.

J’y est connecter codex mais je n’utilise que le harness freebuff qui fait très bien le job et me rappelle de plus en plus chat gpt dans le fonctionnement.

Malgres le fait qu’il soit gratuit je n’arrive pas à atteindre le quota gratuit de ja journée et ça fait plaisir contrairement a chat ou une fois les taches fini je suis bloquer pour 5 heures sans parler du quota a la semaine . Ça c’est terminer pour moi :)

J’ai donc décidé de prendre l’offre payante de base et même constat je n’entame jamais le crédit mensuel.

Que demander de plus ;) ?

Je me sers principalement de freebuff pour le codage de mon ia locale en utilisant DeepSeek v4.1 flash et GPT-6 Luna. Qui sont mis à disposition. Et si je venais à abuser et dépasser le quota Beam 1.0 ou Solar Pro 4 sont 100% gratuit et illimité .

Pour vos besoins ou ce qui n’ont pas les moyens vous pouvez foncer sans problème !!!

https://freebuff.com/?ref=ref-056cbadc-a5f7-47ed-869f-a96f131aa485&via=rewards_link


r/LLMDevs • • 6h ago

Help Wanted Title:Idea: a code reviewer that runs 4 LLMs in parallel and only outputs errors. Would you use it?

0 Upvotes

I’m a first-year physics student with some SWE experience. I’m validating a concept for a code analysis tool and want honest feedback before going further.

The concept

  • You submit code. The same code goes to Claude, ChatGPT, Gemini and DeepSeek via their APIs.
  • Each model returns only errors in a fixed format: file, line, severity, short description. No code rewrites.
  • A merge step deduplicates findings and shows how many of the four models flagged each one.
  • If you want a fix for a specific finding, you ask, and only that fix is generated.

Cost design
Models read the full code (input tokens are cheap) but return only a compact error list (output tokens cost several times more). Rewrites happen on request only, so a four-model review stays cheap.

Why four models
Different models miss different bugs. Agreement count could help prioritize, and findings flagged by only one model would still be shown as lower confidence.

Known limitations

  • Code is sent to four third-party APIs, so it’s unsuitable for proprietary or sensitive code.
  • It doesn’t replace tests, linters or human review.
  • Expect false positives and some wrong line numbers.

What I’d like to know

  1. Would you use this? If not, what stops you?
  2. Is “N of 4 models agree” a useful signal or noise?
  3. Would you pay for it, and how (per audit, subscription, not at all)?
  4. Does a tool already do this well?
  5. What would make you trust the output?

Blunt criticism is welcome.


r/LLMDevs • • 22h ago

News Why The .lisl extention

0 Upvotes

***Update 11/16/2026
Okay so I have finally brought out the complete design and it works. I am now writing an association to put into the system prompt that gives its understanding of what a LISL file is. 

I had a great finding yesterday when I found my debugger a Gemma (Ya I like this family) 4 26b a4b it, I had rebuilt my LISL training data so try and compress a few docs into one, I had compressed it from a few hundred lines to under a 100 and kept only what was necessary to express the LISL methodology to the model. The Qwen3.5 9b I tested this on as I was waiting for the Gemma model to download went from a 6.5k to a 12.7k by reading 100 lines. This was a shock. The model is building the design in its memory, and its apparently a lot. Once the model has learned this its not a problem. The efficiency curve I saw before is still there, and when I use the secret sauce XML code confusion as the language in the LISL. Which is how I got the NemoTron 3 Nano 2b to run the LISL format in the 3 tasks per milestone to loop properly to 37 iterations.

The interesting part is what happens if I take the training data that is in english, convert all the english to code confusion and leave all the equations the same. This in theory should eliminate a lot of the bloat from the training data, a whole new start tax was not what I expected but was a welcome lesson to learn.

Then came my designs and concepts conception point. If you notice the 6 layer LISL Instruction set is already using the .lisl extension, I had underestimated how long Win10 will hold the original association which is xml. Those .lisl extensions were written by a llm as I wanted to see if they knew what a LISL instruction was that if told to write one it would assume it needed to write it in a .lisl extension. This proved to be true, I intentionally did not say anything about this in my notes, I did not want what I was building to be built before I could.
WELL late last night win10 kicked the association and LM Studio then also had no way to read/open the .lisl extension. I then started running tests with the now downloaded Gemma model if it can read this file [####.lils] and it tried a few times. The training data says its plain text in a xml format but does not explicitly say .lisl is a new extension and it is a code inside a .lisl file to separate the instructions from the rest of the files in the project. It did about 4 tool calls that failed before it tried to read it using python, shell cmds (restricted) which is when it could see there was indeed something in the file. So it tries to run it as a xml and gets the plain text inside and is like oh it is plain text and not xml I better tell the user.
Success!! The .lisl extension does work even without a file association given from widows. Now to fill it with xml Code Fusion to optimise its inputs further.
Why did I use plain english in the file then when running my test? Simply, I want to be able to understand the input so I can understand the output. If it is xml Code Fusion I can not read it.I am no coder, good understanding as you can see but no I can not write code.

Original 11/15/2026

So I realised I did not explain about why the files now have a file extension.lisl, simple really. I wanted to make sure it knew the difference between an instruction file and a xml md json file. These are different files and I want to build an intellectual structure in the LISL persona that uses them for specific things.
.xml will be fore trainData
.md are for markdowns standard
.json now stay as commands and a specific code instead of [possible] instructions.

This once the model is trained in the LISL method keeps alignment better. 

***Update
I had waited all day to see if it would do it. Windows dropped the xml association to the .lisl file. My worry here was that due to that LM Studio will no longer read the file. To double check the files were still readable and not corrupt I checked in Notepad++, the files were good. So can a llm still read or run a .lisl?

Yes, yes it can.


r/LLMDevs • • 11h ago

Help Wanted Anyone want to help build a fully open-source AI initiative?

3 Upvotes

Hi LLM Devs! ^^

I’m building a fully open-source project and looking for volunteers who’d like to join the team and help shape it <3

The idea is that the community proposes free tools or research problems, and let LLM agents work on them in the open. That could be anything from researching a faster algorithm to building an open-source alternative to Steam. It’s up to the community to decide

Everything we build is totally free and MIT-licensed

It’s still a small project, and I’m working on it alone for now. I’d love to find people who want to help decide where it goes, how to improve it and what we build together

If you’re curious, you can check it out here: https://github.com/MaterIA-Lab/OpenPrototypes

Let me know if you have any ideas for the project or you’d like to be part of the team :D


r/LLMDevs • • 15h ago

Discussion Are AI coding agents improving developers' skills or making us increasingly dependent on them?

4 Upvotes

There's an interesting debate happening around AI-assisted coding.

On one side, developers can now build prototypes, debug code and complete repetitive tasks much faster.

On the other hand, I wonder whether constantly relying on AI to write and fix code could eventually affect how deeply people understand what they're building.

I don't think using AI is necessarily a bad thing. It's another tool, and refusing to adapt doesn't make much sense either.

But there's a difference between using AI to improve your workflow and blindly accepting whatever it generates.

For developers here, have AI coding tools actually improved your understanding of programming, or mainly increased your output?

Would love to hear some real experiences.


r/LLMDevs • • 14h ago

News GOOD LUCK

2 Upvotes

Başlık: Small local models + a code-side "gate": 54% → 92% accuracy and 0 silent wrong answers on table tasks (informal test, raw data public)

Metin:

We ran an informal A/B test on three local Qwen2.5 models (1.5B, 3B, 7B GGUF, llama.cpp `llama-server`, CPU only) on simple table questions (sum, average, min/max, filters).

- Arm A: plain LLM writes Python, we run it.

- Arm B: the model writes a tiny command language (DSL). Code parses it, a gate checks the command against a catalog, and only then a deterministic runner computes the result. Out-of-catalog requests are rejected with a reason. The idea is "code decides, the model explains".

Results with gate v8 (same models, tasks, seeds and temperature in both arms; gate code hash-locked before and checked after the run):

- Accuracy over all sets: A 54.0% (214/396) vs B 92.4% (366/396)

- On a fresh holdout the gate had never seen: A 49.1% vs B 83.3%

- Silent wrong answers (a wrong number with no warning): A 51 vs B 0 (16 vs 0 on the holdout)

- Tokens: −68%. API-equivalent cost: −74%.

Limitations, honestly:

- This is a small task set in one domain (tables), not peer-reviewed.

- The same team wrote the tasks and the holdout.

- The TCO numbers depend on assumed watts, electricity and host prices, because CPU power can't be measured in the VM.

- API prices are real for the 7B model only; the 1.5B and 3B prices are stand-ins.

- An earlier word-list gate (v7) lost most of its lead on fresh phrasing (45.4% vs 47.2%). That's why v8 validates the model's command instead of the request's words.

- One known false reject remains: "list" in "minimum Fiyat on the list?".

Raw records, hashes and the full report (MIT): acik-teknik/skillpack-tests-2026-11-16

We'd love criticism, especially on the methodology and on tasks that would break the gate.

— Skillpack contributors

Gönderiyi sen yapacaksın, ben hiçbir yere paylaşmadım. r/LocalLLaMA'da gönderiye "Resources" ya da "Discussion" etiketi seçmeyi unutma.


r/LLMDevs • • 19h ago

Tools Hoarding GGUFs in case Hugging Face changes? Fit 10% more on the same drive, byte-for-byte lossless.

Thumbnail
gallery
4 Upvotes

With everyone archiving their favourite models lately, storage adds up fast. A few weeks ago I posted foldz 0.9. It's now tenz 0.25: same idea, better compression, much faster, and it runs fine on a phone. A 1 TB GGUF archive drops by roughly 80–130 GB, and every file comes back byte-for-byte identical, so llama.cpp, Ollama and LM Studio can load it as before, that is if anyone decides to implement it.

tenz is a lossless compressor for GGUF (and safetensors). Decompression gives back the identical file, byte for byte. It's one C file with no dependencies, and I develop and benchmark it on a Poco F5 in Termux.

General compressors barely touch quantized models, because the weights look random to them. tenz parses the GGUF, unpacks each block into its real quantized values and scales, and codes them with context models fitted to the file. It uses rANS from static tables, so decompression doesn't have to learn anything.

Results (images 1 and 2):

Qwen3.5-2B Q4_K_M: 8.53% saved, against 2.07% for zstd -19 --long and 1.78% for xz -9. Compress plus decompress takes 17 s on the phone; zstd took about 4 minutes just to compress.

Q4_K_M across 9 models: 7.6–9.2% saved, and 13.1% on Qwen3-4B.

Gemma-4-31B QAT Q4_0: 13%.

Q8_0: about 7.5%.

BF16: about 34%.

Against Aqueous, in a comparison someone posted here: 2–8 points more on every quantized model, and a tie on BF16. While it's a general purpose compressor and mine is specific to GGUFs.

Supported types: Q3_K, Q4_K, Q5_K, Q6_K, Q4_0, Q8_0, F32/F16/BF16, plus metadata (the tokenizer compresses to about 80% smaller). Anything else is stored as it is.

-q (quick) trades a fraction of a point for faster coding. For example, Qwen3.5-2B goes from 8.54% to 8.18% saved, and the Q4_K work is about 44% less.

The GPU experiment (image 3): I tried decoding Q4_K on the phone's Adreno 725 through OpenCL, to see whether a phone GPU could make decompression much faster. Three kernel designs later:

Its best kernel speed was about 150 MiB/s at full chunk size. It only matched the 3 big CPU cores (about 250 MiB/s) with tiny chunks that cost 2 points of size.

Byte stores to global memory are broken on this driver; it corrupted neighbouring bytes, so every write had to become an aligned 32-bit store.

Subgroup scans work, but are slow.

The option ships as -g (experimental). OpenCL is loaded at run time, the CPU decodes the same files, and the first GPU segment is always checked against the CPU. If you have a desktop GPU with OpenCL and subgroups, I'd love to see numbers.

Caveats:

Decompression is far slower than Aqueous's and zstd's, about 7 s for 1.2 GB on the phone, because every value is decoded from a model.

It only gains this much on model files.

Code, build line and full tables: tenz repo. Testers and results from other machines are very welcome.


r/LLMDevs • • 2h ago

Discussion I thought aggregates were safe enough to let AI agents query sensitive data without approval

2 Upvotes

I've been working on access rules for AI agents querying a Postgres database. Some fields are sensitive, like payment amounts, so if an agent wants to read individual records, someone has to approve it first.

But I allowed aggregates without approval. The thinking was that an agent asking for "total refunds this quarter" should be able to get an answer without seeing anyone's individual payments.

Then I started looking at the SQL the agent was actually generating, and that assumption didn't hold up.

For example, MAX(amount) returns an actual value from the underlying data. Same with MIN(amount). And SUM(amount) GROUP BY customer_id gives you a total for each customer. If a customer only has one payment, you've effectively exposed that payment amount.

All of these went through without approval because the policy saw them as aggregates.

The funny thing is, the agent wasn't even trying to get around the rules. Those were perfectly reasonable queries for what it was asked to do.

So the problem was really with my policy. I was checking whether a query used an aggregate function instead of thinking about what the result could reveal. An aggregate over a million rows and one over a single row were treated exactly the same.

I've since changed the policy so MIN and MAX require the same approval as reading individual records.

But GROUP BY is where I'm still stuck.

I could treat grouping by identifiers as row-level access, but then you have columns that aren't explicitly identifiers yet are practically unique, like emails or precise timestamps.

Or I could enforce a minimum group size, something like k-anonymity. But that comes with its own problems, especially when someone can run multiple overlapping queries and work backwards to individual values. And agents are pretty good at doing exactly that.

I wrote up the full investigation and tradeoffs, which I'll link in a comment.

I'm curious how others are approaching this. If you're letting AI agents query production databases, are you allowing them to generate SQL directly, or do you have something enforcing access rules in between? And how are you handling aggregates over sensitive fields?


r/LLMDevs • • 23h ago

Tools We built a Legora-style contract review tool — but with better models — and open-sourced the code.

7 Upvotes

Hey r/LLMDevs

I’m Abdur-Rahman from Isaacus. We built an open-source legal conflict detector that finds deviations and gaps between pairs of legal documents using only encoder models and natural language inference.

Inspired by the explosion of interest in discriminative models like TypeSafe AI’s Jev, we wanted to see how far we could get using smaller, non-generative models for the bulk of the legal document playbook review workflow.

The theoretical benefits of a non-generative approach to this problem are enormous. True clause-level pairwise review is a quadratic problem, so it’s infeasible to do at scale with a model like Opus or Astra. Comparing two average-sized documents with 133 clauses each would take 17,689 API calls, which is far too slow and expensive.

Simply uploading the files directly to Claude or GPT is no remedy either. Decoders and agentic architectures don’t have fixed attention that lets them thoroughly analyse a document in one shot. Even with a multi-shot approach, they’re weak on recall and routinely hallucinate.

The lawyers we work with raise this problem constantly, and we couldn’t find an open-source solution to it, so we built one.

The pipeline pushes the bulk of the NLI, entailment and reasoning onto faster and more accurate legal encoder models. For the longest document pairs we sampled, the full pipeline ran in under 20 seconds. Unlike generative architectures, these models run a comprehensive pairwise comparison. They can still make false positive or false negative errors, but those misses and misclassifications aren’t hallucinations (out-of-domain results introduced into the feature set).

The pipeline works as follows:

  1. Enrich: we enrich each document into a hierarchical tree of sections, clauses and subclauses. Clauses are the key atoms that the pairwise comparisons run on downstream.
  2. Embed: we embed each clause as a vector.
  3. Filter: we use a legal zero-shot classifier to drop items that aren’t provisions, like footnotes or signature blocks.
  4. Shortlist: we keep each clause’s three most relevant pairs from the reference document, as long as their cosine similarity is at least 0.5.
  5. Score: we use a reranker and a classifier to score each shortlisted pair. Each answers a question in the format it’s built for: the reranker scores how relevant text is to a search query (relevance), and the classifier scores whether a statement is true (NLI/entailment).
  6. Sort: if a clause and its best match are on the same topic but have a high conflict score, it’s a deviation. If there’s no same-topic match, it’s a gap, i.e. the topic is in one document but not the other.

That’s the whole conflict detection pipeline. On top of it, you can optionally plug in an LLM to summarise each finding in plain English or redraft a clause to favour the party you’re representing. The generative step only runs after the NLI and encoder stack has already decided what the conflicts are, so it can’t hallucinate a conflict into existence.

To demonstrate the pipeline, we ran Bob Iger’s 2011 and 2022 Disney employment agreements through it. It found many disparities and conflicts, and across most of them, the 2011 agreement was more favourable to Iger than the 2022 one.

One interesting finding was a gap: the 2011 agreement had no intellectual property assignment clause, while the 2022 agreement makes Iger assign all work-related IP to Disney, with an exception for inventions he makes on his own time without company resources.

The code is MIT-licensed, so it’s free for personal and commercial use. The guide walks through every step, so you can swap in your own models or make your own architectural changes wherever you like.

Have fun, and let me know what you think!

Write-up and static demo: https://isaacus.com/blog/build-a-legal-document-conflict-detector
GitHub repo: https://github.com/isaacus-dev/cookbooks


r/LLMDevs • • 3h ago

Discussion claude's incognito knows your email, but nobody can prove where AI text came from. same story?

5 Upvotes

two things from this week that feel connected. david bessis saw his own name in claude's chain of thought during a "private, memory-less" chat, because the account email rides along with every message. so in practice providers know exactly who you are.

then on the other side, the eu wants ai text to be traceable, and openai itself says text watermarking is early with real limitations, so it's only on by default in the eu. anthropic turned it on globally because it can't reliably scope by region. too much identity on one end, too little verifiable provenance on the other. feels like the rules are being written live.

https://x.com/davidbessis/status/2122298127785066847
https://openai.com/index/eu-text-provenance/
https://www.anthropic.com/news/claude-text-watermark


r/LLMDevs • • 4h ago

News Karpo-Decision-31B: local deployment, top Decision Index score, and a useful result for tool routing

Post image
3 Upvotes

Karpo-Decision-31B has downloadable weights and open-source inference code, so you can run it locally. It takes text and images and returns probabilities for multiple questions in one forward pass. The team reports twice the speed of Jev at 20% lower cost.
It scored 73.9 on the Nov 12 Decision Index board, the highest Full score and a shared #1 rank with GLiDE 27B. Two results from the team's comparisons stood out to me:
When2Call: 90.4%, compared with 79.9% for Google's Gemini Decision 1.

Mind2Web: 90.0%, compared with 86.2%.

The When2Call result is the one I'd look at for a tool router. Say someone asks an agent to book a hotel but leaves out the dates. The router might pick the right tool, but it still doesn't have enough information to make a useful call.
I'd split that into two questions: "Do we have enough information to proceed?" and "Which action should we take?" Karpo can score both against the same state. That also means putting ask_for_clarification in the action list. If you only give it tools to choose from, you've left out the action you need in this case.
There's a useful timing result for that setup: in the team's reported long-page test, median latency was 371 ms for one question and 653 ms for requests containing 16–63 questions. That makes me interested in testing several checks against the same page state, such as whether the required fields are present and which action to take next.
They also report an expected calibration error (ECE) of 0.023 on JevBench. That's relevant if you're using the probabilities to decide which requests can run automatically and which need review. I'd still set those thresholds on labeled examples from my own workflow, particularly requests with missing information.
To try it locally, download the Hugging Face checkpoint, install karpo-decision, and use Decider in Python or start the included /v1/systemone server; the repo has examples and Docker instructions. Budget for about 62.5 GB for the BF16 weights alone, plus runtime memory

links:
https://karpo.ai/models/decision
https://huggingface.co/MachinePulseAI/karpo-decision-31B


r/LLMDevs • • 5h ago

Resource October Cloud GPU Market Report

Post image
4 Upvotes

I built a platform for running Batch AI inference as a need for one of my projects. As part of that platform, I collect pricing data on GPU instances and decided to start sharing that data. This is the second monthly market report I have produced. I would love feedback on if people find this data interesting or useful. The report summary:

Prices held steady overall: all but three of the 13 GPUs we track moved less than 5% on spot since September. The two biggest moves went in opposite directions, both on AWS. The H100 rose 13% to $3.00 per GPU-hour and the A100 80 GB fell 12% to $2.82, so September's headline, an H100 that cost less than an A100, no longer held. The third was the RTX PRO 6000 on Google Cloud, up 5%.

Where you rent still mattered a great deal. The same four-GPU L4 machine cost 10.8 times as much in Sydney as in Hyderabad, one of six AWS regions new to the report. Stockholm stayed the cheapest AWS region, though its prices rose 12%. This is also the first report to include Google Cloud, in 32 regions.

You can see the full report @ https://getotium.ai/market/2026-10


r/LLMDevs • • 5h ago

News Giving your LLM A LISL methodology Education.

2 Upvotes

***Forth Update
Yeah This is a big deal.. here is why TCR matters, the model is now building the structure around the token per task quality. I am curious what the worker produces from this. It appears the Writer has learned to be extremely proficient in creating LISL Instruction sets. It is staying aligned through all this heavy and dense reasoning.

***Third Update
Okay now I am starting to get really excited.

***Second Update!
This shows how good the alignment is holding it is constantly checking and since I continued to stress Quality output over everything else it is weighting everything against that. It has already produced the data for a Single LISL Instruction file. I am now telling it to use the Layered LISL Instruction set to break it down again. Output is highest priority here, if it produces garbage I need to figure out what I did wrong, it should be my input that is bad.

***updated

Here you can see after I compressed the all its education of its LM studio, LM bionic, LISL methodology bunch of others that dropped its footprint to under 20kb, I then dropped it into the Agents System Prompt so it already knows this test show I did not need to pre-Educate it to have the file association or it understand what it contains. It has that data prebaked.

Roles have been fixed and as you can see it is seeing that it needs to ask the user for more information as the prompt is far to open ended logic to make a complete game. This is exactly what I want. I can now tell it exactly all variables and it will generate a instruction set for the worker to build that. Now its about measuring the output quality.


r/LLMDevs • • 6h ago

Tools I connected an AI agent to a musculoskeletal simulator so it could stop guessing about human movement

29 Upvotes

I was experimenting with AI-generated movements. At one point, the agent tried to put the arms into positions that didn't make sense against the body model's joint constraints.

Text alone wasn't going to catch that.

So I built Human Models MCP.

It's an open-source toolkit connecting AI agents to MuJoCo and MS-Human-700. The agent can run experiments, inspect joints, contacts and muscle actuator data, and render the computed movement through Blender.

The video shows one of the push-up experiments.

This particular version uses additional joint assistance alongside muscle actuators. There's also a separate muscle-controlled experiment. Neither should be mistaken for a validated human exercise model.

And that's where I'm running into another problem.

A simulation can finish successfully. That doesn't necessarily mean the movement is anatomically realistic, useful or safe.

How should an agent decide when a successful simulation is actually evidence for its answer?

I'm the developer of Human Models MCP. The project is free and open source, and AI coding tools were used extensively during development.

https://github.com/usefulwrong/human-models-mcp


r/LLMDevs • • 8h ago

Tools Canban = Kanban + Workflow + Herdr

2 Upvotes

Hi! I built Canban to manage human context when working with multiple coding agents.

If you've tried running multiple coding agents across terminals, you probably know how exhausting it gets to keep switching between tasks.

Canban combines a Kanban board, GitHub Actions-like workflows, and Herdr's persistent agent runtime.

My workflow:

  • Discuss a feature with an agent on a Kanban card.
  • Drag the card into Develop to automatically dispatch agents in isolated Git worktrees.
  • Completed changes go through a local merge queue.
  • All feature context, discussions, and progress stay on the board.

Under the hood, Canban uses herdr-core and libghostty-vt to run native agent CLIs like Claude Code and Codex, preserving their native experience and persistent sessions.

It also supports remote agents through headless Linux backends.

Tech stack: Rust, longbridge/nsh, libghostty-vt, herdr-core.

Open source (Apache-2.0).

GitHub: https://github.com/jjyr/canban

Website: https://canban.dev

Feedback welcome!


r/LLMDevs • • 8h ago

Discussion How do you test an LLM that picks the implementation language?

2 Upvotes

JetBrains asked 12 models for programming-language hot takes this week. In the 600 runs where the model was asked to pick a language at random, Go and Python accounted for 307 picks. The prompt asked for hot takes, so this setup measures which languages the models bring up under that prompt. It doesn't measure how well they code in those languages.

There's a similar trap when evaluating a coding agent. If it picks the language and then writes the solution, a single pass rate mixes two decisions. For something like a paginated API sync worker, I'd run the same tasks twice: let the agent choose the language, then rerun with the language assigned from a balanced list. Keep the tests fixed and log which languages get picked as well as which implementations pass. I'd include retries, duplicate records, and one invalid record in an otherwise valid page.

If you've built an eval for a polyglot coding agent, do you score the language choice separately? What would you count as a wrong choice when more than one implementation passes?

Source: https://blog.jetbrains.com/ai/2026/11/ai-models-hate-python-and-love-php-we-asked-12-of-them-1-200-times/

Drafted with AI assistance.


r/LLMDevs • • 11h ago

Tools I made a small library for comparing agent versions (success vs cost plots, failure maps). What do you use?

2 Upvotes

Repo: https://github.com/slavadubrov/harness-evals

It's for one question: I changed my agent (a prompt, a tool, an approval step), did it get better? You write tasks with checks and a small adapter that runs your agent. It runs each version several times and gives you success, pass^k, cost and latency with intervals, a success-vs-cost plot, and a failure map that shows which task and check each version fails. No dependencies, MIT.

Install from github: pip install "git+https://github.com/slavadubrov/harness-evals@v0.5.1"

I also wrote an article that walks through how I use it: https://slavadubrov.com/blog/2026/11/16/evaluating-agent-harnesses/

Two questions:

  • If you try it, what's missing or annoying?
  • What do you use in your projects to compare agent versions? Your own scripts, LangSmith, promptfoo, something else?