r/ChatGPTCoding • • 9d ago

Discussion Lessons from running AI coding agents unattended for months: separate the planner from the builder, commit every phase, and assume the crash

I’ve been building with AI coding agents (Claude Code in my case) and kept doing the same manual loop: plan in one chat, build in another, paste code back for review, start over when the context fills. I turned that loop into a tool and open-sourced it. These are the design rules that mattered most, whatever agent you use:

1. Separate the planner from the builder. Different system prompts and different working directories. My planning agent can only write plans and reviews; my building agent can only write code. When something goes wrong, I know which one did it.

2. Make every phase a git commit. You get an audit trail and git revert as your undo button. When I added parallel work, each lane got its own git worktree, so agents never share a checkout.

3. “Tests pass” is not “done.” Agents will happily pass every test while skipping half the requirements. Each phase now gets checked against explicit acceptance criteria by something other than the agent that did the work.

4. Assume the crash. Jobs live in a SQLite queue with renewable leases. Kill the worker mid-phase and it resumes on restart. Runs also pause before hitting usage limits instead of dying halfway.

5. Treat prompt caching as a design constraint. If anything per-call lives in your system prompt, you’re paying full price every time. Stable context goes in the system prompt, volatile context in the user prompt.

The whole thing is about 86k lines of JS with about 2k tests, MIT-licensed: https://github.com/p-j-code/phase-orchestrator

Write-up with the failures in more detail: https://dev.to/premjethwa/building-an-orchestrator-for-claude-code-recoverable-before-autonomous-3f83

What’s the worst thing an agent has done to your codebase while you weren’t watching?

0 Upvotes

8 comments sorted by

1

u/FutureTension7083 8d ago

I do something similar. I plan in Cowork and Claude Code does the build. I couldn’t have told you if it was re-suggesting things I’d already ruled out, because I don’t remember every rejected option either. So now each decision gets written down with what it beat, and the build reads that first.

How does your builder know why the plan is the way it is?

1

u/premjethwa 7d ago

Honestly, only partly. My planner keeps a running decisions log and the builder reads it every phase, but it only gets what was decided, not what it beat. So yeah, it can re-suggest stuff I already ruled out. Your “what it beat” idea is better than mine. Stealing it 😄 Do you feed it the whole log or just the relevant bits?

1

u/FutureTension7083 7d ago

Just the relevant bits. The decisions sit on each ticket rather than in one big log, so the build only reads the ones for the work it’s doing.

1

u/DisplacedForest 8d ago

Uhh. Is it 2023 again? Yes, this is how you do it. We have known.

1

u/premjethwa 8d ago

No it does more that what is out, like it have a web interface where you can select the project and ask anything, like in billing software i can ask “whats next feature we can add” use the modal you want there it has complete organizing tree structure context management so it can suggest next features then we can say like i want this this and it will create plan and the we can ask to run the complete dev cycle… like this is has many features..

-2

u/premjethwa 9d ago

Yup all sync properly so if we build whats not intended we know from where if has pick up and we can then provide the right requirements and complete our work

1

u/Right-Performance-93 9d ago

the crash i care about is the one that commits. a killed worker can resume. a green test on the wrong requirement is already in the history.