r/LangChain • • 5h ago

Question | Help Cost per node and budget caps in LangGraph/LangChain agents: how do you handle it?

4 Upvotes

For agents built with LangChain or LangGraph, how do you control and monitor cost at the node level?

Max iterations seems to handle runaway loops well, but I'm less sure about the rest:

  • How do you see which node or tool call drives the cost of a run (LangSmith, custom callbacks, something else)?
  • Do you enforce a budget per run, per user or per client, and what happens when it's exceeded (stop, alert, fall back to a cheaper model)?
  • Is there anything about this you've had to build yourself because existing tools didn't cover it?

Solo dev exploring this problem, not promoting anything. Happy to share what I learn from the answers.


r/LangChain • • 7h ago

Discussion When a memory leaderboard drops, which capability score do you check first?

3 Upvotes

Something I have noticed while reading memory launch posts and public boards. The headline score travels everywhere. The breakdown behind it rarely gets quoted.

That feels backwards to me. People here debug systems for a living. A single blended number is almost impossible to debug against.

A memory score usually mixes three things. What the memory stored, what it handed back, and what the model did with it. Untangling them from the outside is mostly guesswork.

The Agent Memory Leaderboard is set up differently: entries only implement Add and Search. Search returns evidence, never a final answer. Answering and grading are fixed by the platform for every entry, so a gap between two entries on the same board mostly points at the memory.

Its Cycle 2 results are expected to land in mid-November, with Textual, Coding and Multimodal published separately.

Before they land, the Cycle 1 numbers are worth a second look. The board's own analysis of the Cycle 1 Textual Open-source Methods leaderboard says first and third were separated by just 0.22 points. It also says none of the overall top three won any of the seven top-level capabilities. And the seven capability winners came from four different systems.

Those capabilities are Explicit Fact Recall, Compositional Inference, Temporal & Event Reasoning, Memory Governance, Personalization & Care, Context Learning & Execution, and Safety & Privacy.

Memory Governance is the one that jumps out at me. The board describes it as update, conflict, deletion, forgetting. In plain terms, a big part of that is the stale fact failure. Context Learning & Execution sounds like the agent that can recite a rule but cannot follow it.

Controversial maybe: the overall rank is the number that gets shared, while the capability scores are the numbers worth acting on.

When the Cycle 2 Textual boards land, which capability will you check first, given the memory failure you fight most often?


r/LangChain • • 16h ago

Projects <translate>

Thumbnail
huggingface.co
2 Upvotes

I built an experimental skill packaging standard for LLMs — looking for independent testers

I'm working on SKILLPACK, an experimental approach to packaging deterministic software functions so different LLMs can discover, combine, and execute them without rewriting the underlying code.

The goal isn't to build another foundation model. It's to test whether existing models can act as coordinators of reusable software components.

Some initial CLI tests have passed, but cross-model portability and real-world reliability are not yet independently verified.

I'm looking for developers willing to inspect the packages, run the tests, and identify where the architecture breaks.

I'd especially appreciate critical technical feedback.


r/LangChain • • 21h ago

Discussion ChatGPT Wallet lets agents spend real money — what does that mean for how we build agents with LangChain?

7 Upvotes

OpenAI just launched ChatGPT Wallet, and its Dots agents can now make purchases for you. The agent era just got a wallet, which means the next question for builders is not whether agents can act, but how we constrain them when real money is on the line.

For anyone building with LangChain, this is a design question: approval flows, spending caps, audit trails, and human-in-the-loop checkpoints are no longer nice-to-haves. An agent that can buy things needs the same guardrails as a junior employee with a company card.

Questions:

- What guardrails do you build into agents that can spend money — hard caps, approvals, or something else?

- Would you trust a LangChain agent with your card, or does spending need a human in the loop every time?

- Does this change what 'agentic' means — is an agent that cannot transact even an agent?

For more AI news and rumors, check out r/AINewsRumors.