Gonchik in touch.
As we know Garbage-First is quite a popular garbage collector nowadays. At least more and more articles, product documentation mention G1GC.
Today, I’d like to share a story about leaked Jira Service Desk DC (3 nodes) with more than 20 apps from the marketplace, quite a lot of groovy scripts (405 files), 6 in-house apps. Also, it's continue of previous article.
One of the instances every night do import files into the “Insight” app, those asset management includes too many assets around 1 mln, as it’s daily import history of objects can grow as well.
Let’s focus only on importing procedure, we had 1 dedicated instance with 32GB heap for jdk8u282-b08 and G1GC. During import find out the next behaviour:
# cat atlassian-jira-gc.0.current| grep -i "full"
2021-03-30T21:13:19.462+0300: 934.828: [Full GC (Allocation Failure) 31309M->23165M(32768M), 58.7721842 secs]
2021-03-30T21:14:31.997+0300: 1007.363: [Full GC (Allocation Failure) 28965M->23085M(32768M), 53.9423610 secs]
2021-03-30T21:15:58.527+0300: 1093.893: [Full GC (Allocation Failure) 29277M->23233M(32768M), 55.0332227 secs]
2021-03-30T21:17:18.147+0300: 1173.513: [Full GC (Allocation Failure) 29249M->23469M(32768M), 56.4150710 secs]
2021-03-30T21:18:28.114+0300: 1243.480: [Full GC (Allocation Failure) 29253M->23540M(32768M), 56.5262454 secs]
After just updating the jdk to jdk-11.0.10+9, we can find improvement only for the full GC activity.
# cat atlassian-jira-gc.1.log.0 | grep -i "full "
[2021-04-07T07:58:29.710+0300] GC(3745) Using 11 workers of 11 for full compaction
[2021-04-07T07:58:29.710+0300] GC(3745) Pause Full (G1 Evacuation Pause)
[2021-04-07T07:58:39.163+0300] GC(3745) Pause Full (G1 Evacuation Pause) 32768M->24074M(32768M) 9452.818ms
[2021-04-07T08:54:55.518+0300] GC(6005) Using 11 workers of 11 for full compaction
[2021-04-07T08:54:55.518+0300] GC(6005) Pause Full (G1 Evacuation Pause)
[2021-04-07T08:55:05.032+0300] GC(6005) Pause Full (G1 Evacuation Pause) 32734M->25724M(32768M) 9513.629ms
[2021-04-07T08:55:32.812+0300] GC(6070) Using 11 workers of 11 for full compaction
[2021-04-07T08:55:32.812+0300] GC(6070) Pause Full (G1 Evacuation Pause)
[2021-04-07T08:55:41.549+0300] GC(6070) Pause Full (G1 Evacuation Pause) 32714M->25595M(32768M) 8736.933ms
So especially did not change the parameters in that article, to make GC improvements with next parameters: InitiatingHeapOccupancyPercent, UseCompressedClassPointers, UseCompressedOops, MaxGCPauseMillis, OmitStackTraceInFastThrow, UseStringDeduplication.
What’s happened, why does Full GC on JDK8 work so slow? If you look on htop, you can see only 1 CPU is busy.
After looking at the picture, the next JEP-307 (ParallelGC for FullGC), https://openjdk.java.net/jeps/307 is a motivating upgrade to Java 11.
And, please, don’t forget to checkout compatibility on the Supported platforms page.
In conclusion, if you’re using G1GC in JRE 8, you can meet with small problems once Full GC will be called and your system can be freezed for a long time instead of JRE 11.
Hope it helps to make a decision.
UPD: Don't forget to test on your setup the JDK11.
Gonchik TsymzhitovCommunity Leader
Hello folks! To the member of organizations who are still running their Atlassian applications on the server, we are on the side of the bridge, and if we need to sail the boat with confidence either...
Connect with like-minded Atlassian users at free events near you!Find an event
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no Community Events near you at the moment.Host an event
You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events