Confluence memory usage

Question

Dear all,

I have a weird issue that I do not understand clearly.
When I use the TOP of Linux to see our process usage, I can see the confluence java process is reaching 83% of memory.

What I noticed is that after restarting confluence, memory using drop to 50 to 60% but after around 1 hour it grows up to 72 percent and keep on growing.

When I check the cluster resources not in confluence admin portal shows 50%, so roughly a 20% difference between Linux process list

What is correct and what is the reasonable memory using of confluence in run time ?

what to do if memory reach over 85%, will it be safe or a tuning need to be made ?

Thanks for clarification

urawal · Answer

Thanks Daniel, for your time and effort in writing the above which was very useful.

Daniel Eads · Answer

P.S. if you found this information interesting and have time to learn more, Atlassian's own Denise Unterwurzacher has done two amazing talks on memory management in Confluence:

* Trash Talk! How to Reduce Downtime by Tuning Garbage Collection - Atlassian Summit 2016
* Expecto Performa! The Magic and Reality of Performance Tuning - Atlassian Summit U.S. 2017

Daniel Eads · Answer

Hey, great question!

Some of the discrepancy you're seeing stems from how Java allocates memory. There are two major things you're going to want to know about:

1. Xms vs Xmx (initial vs maximum heap). The Java virtual machine acquires memory from the system in big chunks and doesn't really release it back to the operating system if it's not using it. There are flags in the startup string for Confluence that tell the Java process how much memory to start the application with, and how much heap it can use total. We've got some information about changing these parameters here. When you start Confluence and see only 50% memory usage in top, but then 80% memory usage later, that's generally going to be Java ramping up from its minimum/initial RAM usage (the xms parameter) to its maximum heap (the xmx parameter). For this reason, a lot of administrators will set the minimum/maximum heap usage to the same value to ensure the JVM uses a predictable amount of RAM and has it all available at startup. As an example of that, these are the flags you might use to give Confluence a set value of 4 GB:

-Xms4096m -Xmx4096m

2. Heap and total RAM usage are different. The Xms and Xmx values set the JVM's heap size, which is where Java is storing the objects it needs to work with. Separately, the JVM needs to keep track of where those objects are in the heap, what's in them, and how many times they've been called by the application. In Java 8 and above, the area where all this extra information (metadata about memory objects) is stored is called metaspace.

I like to think of metaspace and heap in terms of how libraries worked before electronic catalogs. Back in the day, you'd have a library. It would be a room of a certain size, and you could put books in there until you ran out of space. This is like the Java heap.

Separately from that, you'd have a card catalog that told you what books were in the library, which shelves they were on, and if anyone had checked them out. The card catalog was generally in a different room than where the shelves of books were.

A librarian sorting through a card catalog, no shelves of books are present

So even if you've set Xms and Xmx to 4GB, you may find that Java is using 5GB of RAM when looking at the output of top. There are flags you can set to limit the amount of metaspace that Java will use, but it's generally considered good practice to let Java figure out how much metaspace it needs without putting a hard upper limit on it. The metaspace will be released back to the operating system if the JVM finds it can remove some items from memory. In fact, Atlassian recommends not setting a maximum metaspace value so that you don't accidentally run into out-of-memory errors. Java 8 and above generally do a good job managing metaspace automatically.

You can see how much metaspace the JVM is using with tools that monitor the JVM over JMX, such as JConsole. Jira has some of this information in its web interface. But in general, you could consider a 2/3 rule to account for metaspace and the normal jobs your OS needs memory for. If you've determined you're going to set Confluence at 5GB, make sure your server has at least 8GB. Whatever your Confluence heap needs to be, size your server's RAM up so that Confluence's heap is only using ~2/3 of the total system memory.

The amount of metaspace that Confluence will need varies based on how many active users you have, how many plugins are installed (and the types of things they do), and some other environment-specific factors. As long as you are aware of the differences between heap and metaspace, and keep in mind that your OS needs some free memory even after Confluence is completely scaled up, I think you'll be in good shape!

Cheers,
Daniel