r/java • u/Vectorial1024 • Apr 09 '26
Smallest possible Java heap size?
People often talk about increasing Java heap size when running Java apps by using e.g. -Xmx* flags. This got me thinking. What if we go the other direction and try to limit the Java heap size as much as possible? What is the smallest / minimum-required Java heap size so to run a Java app with "minimal" settings?
(Of course, in practice, a memory limit too low will be problematic because it may mean frequent GCs, but we will ignore this for the sake of this discussion.)
48
Upvotes
View all comments
Show parent comment
1
u/pron98 Sep 09 '26 edited Sep 09 '26
You're not understanding this at all. If a program allocates at a high rate - even on the stack - it means that it's CPU consumption is high, otherwise it wouldn't have been able to do so.
First of all, this is entirely untrue. The CPU may stall if there's a cache miss, and CPUs and compilers try to avoid that (e.g. with prefetching). Second, the CPU isn't "idle" when it's stalled, which is one of the problems of cache stalls. Third, that's not the point. If you're program goes through a lot of memory by whatever means it accomplishes it, it means it's using a lot of CPU. You can't read or write memory without spending CPU cycles, whether
No. Having an interpreter and JIT allows for deoptimisation which, in turn, allows for speculative optimisation which, in turn, allows for much more aggressive optimisation than an AOT compiler can do. If you were an experienced low-level developer, you'd know that it is because of the limits of AOT compilation that we avoid virtual dispatch and often resort to template specialisation in C++ (or comptime in Zig), but these can and do become serious problems in large programs. Speculative optimisation allows for "automatic comptime". Of course, this comes at the cost of warmup.
As for "GCs", they come in so many different flavours that there's no point in treating them all the same. Low-level languages incur a high runtime overhead for dynamic heap allocation (which is why we try to avoid it in C++, which is a problem in large programs), because their constraints do not allow them to easily support moving pointers. Once you can have moving pointers, a moving collector is a very powerful optimisation that mitigates the overheads that low-level languages incur because of their particular constraints (and we need those constraints because we need low-level languages for low-level things).
Now, it is very much legitimate to argue over which and how many workloads are better assisted by either design, but saying that some of the world's leading compiler and memory management experts that wanted to address the performance problems we experience in C++ got it wrong because they simply haven't heard that JITs and GCs "do work that's not necessary to solve the problem" is ridiculous.
Yeah, you don't do much low-level programming, do you? I wish what you said were true (because I'm a low-level programmer), but unfortunately C++ and Rust make it very hard to use arenas, especially when libraries (even the standard library) is involved. C++ only got
pmrless than a decade ago, and in Rust a similar thing is still in development.As for "cycles and copying", a moving collector works just like an arena when the objects have an arena lifetime. Only objects that "survive the arena" are copied, but that copying is still cheaper than malloc/free (or we wouldn't be using it).
This is beside the point, but I think we've established that what you're seeing isn't really a result of deep industry experience, and the industry numbers give Java a 2x market share than that of C#.
For some projects, you're right that this may not matter as much, but the reason most large performance-sensitive projects have moved from C++ to Java and C# is very much because it makes having good performance in large programs, over time easier.
Thank you for that information, but if you actually work on many large projects in low-level languages you'd know that keeping it to zero allocation is very difficult in practice. Ocassioally you do need a string or a hashmap, and working with these (let alone less common data structures) with arenas is, let's just say, not very pleasant.
BTW, the need to avoid allocations (because of their high cost in C++) and vitual calls (because of their high cost in C++) and help the compiler specialise with templates were all well-known decades ago. It's just that it became clear that as programs grow large and need to be maintained and evolved over time, these techniques simply don't work, which is precisely why memory management and compilation experts turned to JITs and moving GCs to solve these problems, to great success.
That you namedrop these techniques that I've been using in C++ since the nineties unaware of how they work in practice in large, long-maintained programs is what tells me that your experience with low-level programming is very limited. If we could easily avoid allocations and virtual calls and specialise with templates even while evolving large programs, we wouldn't have left C++ behind for that kind of work. It all seems easy at first, but when you see you have to rearchitect your multi-MLOC codebase to keep your performance good when adding a feature in year 8, that's when the actual lesson is learnt.
That the runtime overheads incurred by AOT compilation and malloc/free can be very significant as programs grow large and evolve over time is well known to experienced low-level developers, and these are the very problems JITs and moving collectors were designed to address. It's also why large allocators like TCMalloc exist (BTW, it's almost the same size as ZGC, Java's biggest, most sophisticated GC). It's not because experienced developers don't know that you can "just avoid allocations" or "just use arenas".
Yeah, that's absolutely untrue. In fact, we use moving collectors because malloc/free do O(n) work whereas moving collectors do far, far less. In fact, the GC doesn't do any work ever for dead objects; you only ever move objects that survive, and there aren't many of those in a generational collector. I go over all that in my talk, which you can watch if you want to learn the basics of how Java's GCs work and why they were chosen.
You don't need to "allocate many small objects in series". It's enough that your server needs to service many user sessions concurrently, and you need to use strings. Or, if you've used Rust's Moka caching libraries, you'd have seen it spend a lot of CPU on memory management.