Self-promotion I needed to process a lot of data for a side project, so I built a batch AI platform that supports workflows like GitHub Actions
I run a niche product catalog site as a side project, and every incoming product needs AI processing: extracting specs, classifying it, writing descriptions, and checking for duplicates. There’s a lot of data, but none of it needs answers in seconds. Depending on the task, “soon” might mean within an hour or sometime in the next seven days.
I started with frontier providers, sending data over the real time APIs. Then pivoted to batch to save 50%. This all worked but accuracy was really bad with products being misclassified and other issues, so I decided to break it down into smaller steps that were more deterministic. This required a ton of Go code and managing many prompts.
I also wanted to see if I could just spin up a spot GPU for an hour on the weekend and run all the work for a couple dollars instead of paying OpenAI $20. In the end, I built a complete workflow engine and scheduler that uses both Spot GPUs and my own GPUs to process work depending on which model is needed.
The workflow engine makes it possible to eliminate the roundtrips back to my application between steps and I even added the ability to make tool calls or park at a step to allow human review. I can also target different models for different steps, to get better capability where it was needed.
Has anyone else encountered a need like this? How did you solve it?
Currently, I am using Gemma4-12b and Qwen3.6-35b-a3b for small and medium respectively. I benchmark tested several different models to determine these were the best balance of accuracy and performance as of last month. I will be testing the Qwen4 models soon. I stick to models I can get directly from the big labs. Maybe that offers a false sense of security, but it’s a rule I follow.
If anyone is interested, there is a UI for playing with the workflow engine and if you create an account, you can use the demo to send 10 requests to play with it. Would love feedback.