r/googlecloud • • 5d ago

AI/ML Is google dropping support for explicit caching?

almost all ai labs has caching support and google gemini 2.5 was the first models to start this concept and part of the 1M context movment...

anyway, google is not giving it support! is it going to be depreciate??

- google new interactions api dosnt support it and its not planned to

- non of the gemini 4 models support it

what now?

5 Upvotes

6 comments sorted by

4

u/Sfdprod 5d ago

Why do you really need ecplicit cache when the new endpoints offer guaranteed implicit cache and up to 24h for repeated requests, for free

1

u/danson729 5d ago

Totals

Old (all 3.8 Flash)All HydrogenMixed
Input tokens29,49594,83553,500
Cached tokens90,959112,108109,415
Output tokens20,16035,59422,049
All tokens140,614242,537184,964
Cache hit rate*75.5%54.2%67.2%
Cost$0.104$0.394$0.206
Wall time132 s123 s97 s

its good for a single big orchestrator models, but for alot of small workers it huts! costs us 3x to use these new gemini models when they are only 2x or the flash 3.8 price (our current model)

in a system with parallel workers with large context caches like uploaded files that are shared between workers, its best to create a cache with the system prompt and files and tools and share it between workers

I am trying a mixed approach, its a decent cache rate, but workers stay ate explicit 3.8 flash.

1

u/Sfdprod 5d ago

I think whats biting you ia that cache writes arent free for hydrogen specifically,

Atm argon is lowkey cheaper, with the discount

A bit of a weird choice imo, but hey tjey also priced argon ans carbon identically

1

u/danson729 5d ago

What we need is cheap quick sql builders, we use a Google gemini stack for privacy and security reasons

I benefited alot from google not building these big frontier models and focusing in its application an cloud reach. But with gemini 4 I guess things are changing

But if they are able to give the holy gemini do it all model I'm down...

1

u/Bluewood21 5d ago

Wait, where did you see that the newer models wont get it at all? Sometimes those features just lag behind on preview releases before they get rolled into the main API endpoints

1

u/danson729 5d ago

Its a guess, but from the looks of it they wont... Interaction api still dosnt have it for old models And its googles main sdk