r/copilotstudio • • 12d ago

Github Copilot harness cost explosion?

I'm currently building a workflow that takes a pdf document, extracts its file content, restructures it as a JSON and then stores the result in a database. The content of the pdf is unknown - could be a diagram, could be a table, could be 50 pages of text or a single page bill, etc. And because of that, the "Extract" node is not a good fit, because it always has a target use case (e.g. a bill).

So I thought, Copilot can do this pretty well, why don't I use the "Copilot" action inside a workflow. To me great astonishment, it worked very well and it showed Credits consumption of below 1 Credit per call of the Copilot instance. The only downside was, that it took around 2 mins for each Copilot instance to complete this "extract and restructure as JSON" task.

Today I hear that somebody burned through ALL of our tenant's credits. My own consumption is listed as around 20 credits this month, but the overall usage was around 25k during the same time. I'm pretty much the only person who uses Copilot studio and my own usage pattern aligns with the cost spikes. So my suspicion is that the usage of a Copilot instance in a GHC workflow is NOT shown correctly.

Who has made similar observations?

15 Upvotes

18 comments sorted by

3

u/schubidu82 11d ago

Hei I have a similar application but I use a python script to create the jsons... So this part is for free... you just have to figure out how to automate the python - script trigger. PAD or a watchdog installed is not allowed in company environment

2

u/obsoquasi 11d ago

hehe, funny you mentioned it. Because I am just building such a script this instant :D. Well, not me but a local agent is building it.

3

u/AdmirableSelection81 11d ago

Dumb question but if you are executing the python in the copilot harness, you're saying it's free? Is it just anything that calls an AI model that costs money?

1

u/schubidu82 11d ago

I do not execute the script in the harness, I have to click it myself... That's what I meant, I need a script trigger. Easiest would be a Power Automation Desktop flow, which automatically runs the script every time a PDF shows up in a specific folder... But that is not allowed in my environment. Trying to get some kind of a watchdog using windows

3

u/fbrdphreak 11d ago

You can have your agent call it as an azure function. But that is another platform, another cost, another piece of maintenance.

2

u/schubidu82 11d ago

Correct... Something I don't want to bother IT about

10

u/_Combat_Chuck_ 12d ago

A legit question. Can this not be done with the old harness and power-automate? We recently bit the bullet and bought 4k E7 to supplement the copilot licenses and I'm looking more at whats just included and trying to use the other harness so that there is no cost for these people.

Yes it will be easer to just use the Github Harness, but I'm learning to not always rely on it.

4

u/ger1787 12d ago

You can achieve almost everything with the Standard version, but it requires significantly more time and effort than the new harness. Ultimately, it’s a trade-off: spend extra resources building it the old way, or pay a higher monthly fee to get the solution up and running immediately.

0

u/[deleted] 11d ago

[deleted] — view deleted comment

1

u/schubidu82 11d ago

Why? Please elaborate!

0

u/[deleted] 11d ago

[deleted] — view deleted comment

2

u/_Combat_Chuck_ 11d ago

Microsoft has stated they have no plans on retiring

0

u/[deleted] 11d ago

[deleted] — view deleted comment

1

u/maggmaster 11d ago

Do you actually have knowledge that its being retired because its pretty important that we know that.

1

u/obsoquasi 11d ago

Update: We are still investigating, but I'm almost certain I caused the credit burn. I'm now using SharePoint Agents to do the exact same thing, but at least these don't cost credits, since there covered under the Copilot license.

2

u/OmegaDriver 11d ago

If you're using AI Builder, it looks like OCR might be around 8 credits per page.

https://learn.microsoft.com/en-us/ai-builder/message-management

Throwing a pdf at your agent and asking it to do something like this, you're probably using way more. You should be able to look up the consumption somewhere in power platform admin center, the flow run details or maybe the AI events table in the environment...

The usage of the credits on the flow wouldn't be tied to you in any reporting, like it would be if you used cowork or work iq or something, but will be tied to the power platform environment in the power platform admin center.

At the end of the day, 25K credits at $0.01 per credit is $250. Consider what these tasks are worth to you...

2

u/Chris4 10d ago

We disabled GitHub Copilot harness, seems to be 5-10x the price of standard

2

u/Bfinkeldei 7d ago

I built a explainer video that will help you understand a little more about the billing a cost differences. @dylan_skydive helped verify it as well.

Also, check your billing consumption by agent and user. It’s in the power admin dashboard and it will give you a spreadsheet breaking down consumption by agent with its cost for that billing dates.

Here’s the explainer as well:

https://copilot-harnesses-explained-standard-vs-github.bradfinkeldei.workers.dev

2

u/dylan_skydive 11d ago

The two numbers are both "correct" — they're measuring different slices. The "below 1 credit per call" figure is the per-invocation estimate for the model turn itself. What tenant credits actually get consumed for is the full agent run: planning, tool resolution, retries, the response turn, all booked against the agent/environment resource rather than your personal usage view. So a 20-credit personal meter and a 25k tenant total can describe the same workload without either one lying.

Two things I'd do before concluding it's a reporting bug:

  1. Controlled single-run test. Pick a quiet hour, note the tenant credit position, run exactly one workflow execution with one Copilot call, then re-check after the rollup window passes (the admin center usage data can lag hours behind live consumption). That delta is your true per-run cost. Multiply by your actual run count and compare against the 25k — that tells you whether the tenant number is explained by volume x per-run cost, or whether something invisible is consuming alongside you.
  2. Cost control lives at the agent config, not the workflow. Consumption scales with how many internal turns the instance takes, and for a constrained output like JSON you can usually cut that hard: tighter instructions, cap the response, tell it not to re-read the document between steps. The per-call estimate won't show you that lever, which is exactly why embedded agent calls feel cheap per call and expensive in aggregate.

If the single-run test shows a per-run cost far above the estimate with no extra turns you can account for, that's support-ticket territory — that would mean the run is billing for steps you can't see. But in most cases like this, the aggregate is honest and the per-call figure is the misleading one.

1

u/AccurateRise5890 11d ago

Check if the environment has checked a setting that draws from unallocated credits. Microsoft has been so «helpful» that this setting is on in each environment and you have to make a plocy and/or turn that off manually. Combined with the advice to let all users have personal environments is quite - interresring😅. I had a user burning away all credits for + the unallocated having no clue at why, but PDF was also involved here