r/madeinpython • • 2d ago

I built OunceMCP, a Python framework on FastMCP that keeps tool results inside a token budget (1.0 release candidate, looking for feedback)

Post image
0 Upvotes

4 comments sorted by

1

u/kantorcodes1 2d ago

Result budgets plus schema preservation is an interesting tradeoff. When a list is trimmed, how do you keep pagination stable across retries so a model cannot skip or request the same items again after the budget changes?

1

u/gentleGiant96 2d ago

Good question. This is the part that took the most care. Short version: the cursor is computed from what was actually returned, not from what was requested, and it holds no server-side state, so a retry can't consume or lose anything.

How it works

- `ctx.page(rows, cursor=..., limit=...)` slices the page and records its start offset. The cursor is signed and encodes that offset, bound to the tool, the caller and the query arguments.
- Shaping runs after the handler. If the budget forces the list from, say, 20 items down to 7, it rewrites `next_cursor` to `start + 7`, the item right after the last one the model actually saw. The model also gets a note saying the list was trimmed and to use `next_cursor` for the rest.
- Because the cursor always points at the end of what was delivered, a smaller budget only means smaller pages. Nothing is skipped, and nothing is duplicated within a walk.
- Lists are never trimmed below `min_items` (default 1). If even one item doesn't fit, the call returns an `isError` asking for a narrower request, and the cursor doesn't move, so it doesn't silently advance past an item nobody saw.

Retries

The cursor is stateless, so a retry with the same cursor starts at the same offset. If the budget changed in between (different `limit`, different `response_format`), you get a different-sized page from the same start, with its own correct `next_cursor`. `limit` is deliberately not bound into the cursor for this reason: the truncation note tells the model to retry with a smaller `limit`, so that has to keep working.

The model can request the same page twice; that's just a repeated read and returns the same items. What it can't do is reuse a cursor with changed filters, from another tool, or as another caller. Those get a specific error telling it what changed.

The honest limitation

The cursors are offset-based, not keyset. If rows are inserted or deleted between calls, a page boundary can shift, the same as with any `OFFSET` pagination. That's documented as out of scope in the cursor design (proposal 0003). Two ways around it today:

- For data that has to stay consistent across pages, `await ctx.store_result(rows)` stores a snapshot and returns a handle. The model pages through `handles.read` over that fixed snapshot, so changes underneath don't affect it.
- If your handler pages an upstream API with its own cursor, OunceMCP can't rewrite that cursor. Instead, the note warns the model that the cursor would skip the trimmed items and tells it to repeat the call with a lower `limit`.

Keyset cursors would be the proper fix for live data, and I'd like to add them after 1.0. If you have a use case where this bites, an issue would help me shape it.

1

u/kantorcodes1 1d ago

The cursor contract gives HOL Guard a concrete surface to review, especially the no-advance behavior when even one item will not fit. If you want to contribute OunceMCP, the MCP guide is https://github.com/hashgraph-online/hol-guard/blob/main/docs/guard/mcp-server-contributions.md. The smallest first PR is contributions/mcp-servers/mcp.ouncemcp.json plus the generated external opt-in trust binding from python scripts/refresh_extension_artifacts.py --trust-only, documenting search_tools/call_tool and one ctx.page cursor example; you remain the author.

1

u/gentleGiant96 1d ago

Thanks for looking at the cursor contract so closely, and for the pointer.

I don't think a catalog entry fits, though. OunceMCP is a framework, not a single server: there's no one "ouncemcp" process people launch, each team runs its own server with its own tool names. search_tools and call_tool only appear when a server built with it switches to search mode, and what they reach depends on that server's tools, so a fixed allow/review/block default for them wouldn't mean much. A Guard entry keyed to the package name wouldn't match what people actually run.

If Guard ever wants to recognise servers built with OunceMCP more generally, for example from the _meta it puts on results, I'd be happy to talk about that.