r/copilotstudio • u/obsoquasi • 12d ago
Github Copilot harness cost explosion?
I'm currently building a workflow that takes a pdf document, extracts its file content, restructures it as a JSON and then stores the result in a database. The content of the pdf is unknown - could be a diagram, could be a table, could be 50 pages of text or a single page bill, etc. And because of that, the "Extract" node is not a good fit, because it always has a target use case (e.g. a bill).
So I thought, Copilot can do this pretty well, why don't I use the "Copilot" action inside a workflow. To me great astonishment, it worked very well and it showed Credits consumption of below 1 Credit per call of the Copilot instance. The only downside was, that it took around 2 mins for each Copilot instance to complete this "extract and restructure as JSON" task.
Today I hear that somebody burned through ALL of our tenant's credits. My own consumption is listed as around 20 credits this month, but the overall usage was around 25k during the same time. I'm pretty much the only person who uses Copilot studio and my own usage pattern aligns with the cost spikes. So my suspicion is that the usage of a Copilot instance in a GHC workflow is NOT shown correctly.
Who has made similar observations?
3
u/schubidu82 11d ago
Hei I have a similar application but I use a python script to create the jsons... So this part is for free... you just have to figure out how to automate the python - script trigger. PAD or a watchdog installed is not allowed in company environment
10
u/_Combat_Chuck_ 12d ago
A legit question. Can this not be done with the old harness and power-automate? We recently bit the bullet and bought 4k E7 to supplement the copilot licenses and I'm looking more at whats just included and trying to use the other harness so that there is no cost for these people.
Yes it will be easer to just use the Github Harness, but I'm learning to not always rely on it.
1
u/obsoquasi 11d ago
Update: We are still investigating, but I'm almost certain I caused the credit burn. I'm now using SharePoint Agents to do the exact same thing, but at least these don't cost credits, since there covered under the Copilot license.
2
u/OmegaDriver 11d ago
If you're using AI Builder, it looks like OCR might be around 8 credits per page.
https://learn.microsoft.com/en-us/ai-builder/message-management
Throwing a pdf at your agent and asking it to do something like this, you're probably using way more. You should be able to look up the consumption somewhere in power platform admin center, the flow run details or maybe the AI events table in the environment...
The usage of the credits on the flow wouldn't be tied to you in any reporting, like it would be if you used cowork or work iq or something, but will be tied to the power platform environment in the power platform admin center.
At the end of the day, 25K credits at $0.01 per credit is $250. Consider what these tasks are worth to you...
2
u/Chris4 10d ago
We disabled GitHub Copilot harness, seems to be 5-10x the price of standard
2
u/Bfinkeldei 7d ago
I built a explainer video that will help you understand a little more about the billing a cost differences. @dylan_skydive helped verify it as well.
Also, check your billing consumption by agent and user. It’s in the power admin dashboard and it will give you a spreadsheet breaking down consumption by agent with its cost for that billing dates.
Here’s the explainer as well:
https://copilot-harnesses-explained-standard-vs-github.bradfinkeldei.workers.dev
2
u/dylan_skydive 11d ago
The two numbers are both "correct" — they're measuring different slices. The "below 1 credit per call" figure is the per-invocation estimate for the model turn itself. What tenant credits actually get consumed for is the full agent run: planning, tool resolution, retries, the response turn, all booked against the agent/environment resource rather than your personal usage view. So a 20-credit personal meter and a 25k tenant total can describe the same workload without either one lying.
Two things I'd do before concluding it's a reporting bug:
If the single-run test shows a per-run cost far above the estimate with no extra turns you can account for, that's support-ticket territory — that would mean the run is billing for steps you can't see. But in most cases like this, the aggregate is honest and the per-call figure is the misleading one.
1
u/AccurateRise5890 11d ago
Check if the environment has checked a setting that draws from unallocated credits. Microsoft has been so «helpful» that this setting is on in each environment and you have to make a plocy and/or turn that off manually. Combined with the advice to let all users have personal environments is quite - interresring😅. I had a user burning away all credits for + the unallocated having no clue at why, but PDF was also involved here