This post depicts the back end performance improvements found after the Jira and Confluence upgrade of Sept 8 2018. With this upgrade we:
Aside from upgrading to keep the applications current along with getting improvements and bug fixes, the expectation/hope was that we would resolve issues we were seeing with Jira. Jira was exhibiting a continuous growth in container memory utilization that, when it hit ~75%, would become sluggish or even unresponsive enough that it was unusable. Restarting Jira would always clear this. Worked but a bit of a PITA. After reviewing all the Atlassian release notes between the then current versions of Jira and Confluence and the target versions, there were several bug fixes that didn't speak exactly to what we were experiencing but suggested that they might be related to them.
As noted in the introduction, my expectation/hope was that we'd at least solve the problems with the Jira instance. What I did not expect was just how big the improvement would be. After running for ~1 working week on the new versions to capture meaningful empirical evidence , this is what we're seeing on the back end...
In all images below, what we're looking at is a 2 week span from the AWS CloudWatch console. The upgrade was started at 23:00 on Sept 8 and, with initially updating a bunch of plugins in both Jira and Confluence along with a couple of other maintenance items completed "officially" on Sept 9. This is where the delineation can be seen.
The before and after for both CPU utilization and network I/O is startling. A very clear and large drop. These are the graphs for the single m4.2xlarge EC2 instance running both Jira and Confluence.
As with the EC2 instance, we see both CPU and I/O having a clear drop at the Sept 09 delineation point. What is interesting is that the CPU drop for Confluence was greater than for Jira but Jira had a far more significant I/O drop
Another interesting statistic is that the connections also changed rather a lot. In our environment, Jira is configured for a minimum of 20 connections and can span to 60. Conversely, Confluence is configured for a minimum of 0 and can span to 60. As such we see a clear drop in the number of connections consumed. I suspect, were Jira configured the same, we'd see the same with it.
The discrete container memory and CPU utilization also have exhibited these same significant changes.
While Jira had only been up for about a week before the upgrade, we can see the gradual growth in memory utilization that would have eventually have led to needing to restart the instance. After the upgrade, it's been a nice, steady, flat utilization lower than where Jira usually started out with.
Confluence changes were a pleasantly unexpected change. While both CPU and memory utilization were flat, they both had very significant drops after the upgrade.
Hey @Timothy,
That was actually the part that was most startling...
The images are identical other than the versions of Jira and Confluence in them. I have my image build put together such that I specify Jira/Jira Service Desk and Confluence versions as a variable which then goes out to pick up the associated tarball and plop it in on top of my base image.
The base image did not change at all; still the same Debian base and JDK that was under the previous versions. The container task definition in ECS is identical as well other than merely changing the task name (for visibility) and the image it picks up.
As such, I was rather (pleasantly) surprised to see the dramatic change.
That said, I will look through my dockerfile specs to see if there is anything in there that exposes something I don't want exposed and post them here. I don't think there is as I was rigorous in making them stateless and location agnostic. If nothing else, I have a great set of dockerfiles that builds rock solid production images.
I meant to say that the images are broken for me. They seem to link to a Confluence page that requires authentication.
@Timothy see if you can see those now... my stoopid; copy/pasted this directly from my conf instance and i guess it put in links to the images rather than the images
Nope. This is how it looks for me.
@Timothy How bout now? I did the strict "insert photos" thing this time...