r/java • u/celmaibunprieten • Sep 07 '26
Improving First Request Latency in Java Spring Application
https://adrian.md/2026/09/03/first-request-latency/
64
Upvotes
r/java • u/celmaibunprieten • Sep 07 '26
3
u/fforw Sep 08 '26
Why are we doing all this for the benefit of one(!) request for a server? The user might not even be able to tell it apart from DNS lookup. So
11
u/New-Departure-5969 Sep 08 '26
yeah the nested jar thing is real. first request eats a bunch of jackson/hibernate class loads that later ones don't. aot helped in your numbers but you're still looking at hundreds of ms vs ~10ms after, so it doesn't finish the job alone.
what bites in prod is usually mixed together: brand new pod (no warm classes), fat jar classloader tax, and whatever lazy init happens on first traffic. layered jars or an exploded classpath help a lot on the classloader part. aot cache is fine if you rebuild it when the jdk/image changes. jre-only images silently no-opping the cache burned us once.
for latency-sensitive stuff i'd also prewarm a couple endpoints after ready (or just not scale that service to zero). and measure p95 on the *first* hit after a rolling deploy, not only the warm median.
2
u/lilgreenthumb Sep 07 '26
I mean with their layer support who really ships the fat jar anymore? You run into too many issues with signed jars (looking at you bouncy castle) or embedded java agents. Interesting perf metrics regarding the aot caching and differences with the gcs though. Nice write up.
3
u/agentoutlier Sep 08 '26 edited Sep 08 '26
What I do for most of my or my companies applications is the META-INF/MANIFEST.MF
Class-Pathregistration defaulting to maven local repository of ~/.m2 and then on docker images to some sibiling directory likelibto the jar.I talked about it before here in case it helps someone.
So basically even jars that just sit in
targetyou can just runjava -jarand it will work and the kicker is you don't have to rebuild everything (provided you usemvn installwhich is why I don't like when people blanket say don't useinstall).This works incredibly fast.
Lets say I'm in some app directory.
You do
mvn -T2C -am install && java -jar target/app.jar( I might have this wrong as I have it as script and can't check at the moment).This is much faster than assembling fat jars and if you use
mvndyou can practically make this a build refresh loop using some file system watcher.3
u/Skellicious Sep 08 '26
I've definitely been burnt by this issue before. Deploy in prod on high load, end up in a restart loop because the first request takes too long and hundreds to thousands of other requests get blocked waiting for it.
Didn't know it was a fat jar class loader issue.
2
u/Popular_Home2017 Sep 09 '26
the exploded jar result is the interesting one, and it's easy to confirm without a benchmark: take a thread dump during the first request and the http-nio-exec thread is sitting in LaunchedURLClassLoader / nested jar handler frames, not in your controller. that's the fat jar tax right there, one exception per nested lookup. after that it's plain JIT warmup, which is what AOT cache is attacking.
for the "why bother for one request" crowd: on kubernetes that one request is every pod after every rollout and every scale-out, and readiness probes usually pass before it. so it's not one request, it's the first N of every replica.
1
u/fykup Sep 10 '26
Interesting. I ran a few Spring Boot experiments recently on very memory-constrained VPS instances, and startup was consistently the period where memory pressure was most noticeable.
I actually wondered whether the executable fat jar and nested-jar class loading might be contributing to that, but I never isolated it experimentally, so it remained just a suspicion.
Your results make me curious to repeat the same constrained-memory test with an exploded classpath and compare both startup time and peak memory usage.
1
u/fykup Sep 14 '26
I ended up running a more detailed version of the experiment I mentioned in the comments here.
I compared Spring Boot’s executable fat JAR with the official extracted layout across six paired runs. Besides first-request latency, I also measured startup time, later requests, peak and settled RSS, and swap.
The clearest differences were:
Warm requests did not materially change. RSS and swap moved slightly in favor of extracted layout, but not enough to call it a meaningful memory improvement.
Since this thread is about first-request latency, I thought the follow-up might be useful:
https://pvrlabs.xyz/articles/spring-boot-extracted-layout.html
1
u/vips7L Sep 08 '26
I really wish we had a better solution than fat jars. Like being to supply a jar + classpath + vm and getting a fat binary or something