Unable to create max heap size bigger than 4gb on remote agent

[Ted Thompson]

Hello,

We have a maven build that is failing on a remote agent with java heap space errors.

The remote agent is a virtual machine and I've been experimenting with combinations of increasing the ram on the box and increasing/decreasing the heap size of the Bamboo agent to try and resolve our problem.

However, I have noticed that if I specify a heap size greater than 4gb the agent doesn't seem to reflect the max size I set, it almost seems to be limited to 4gb.

[Mon Jul 13 18:05:19]# cat /opt/bamboo/bamboo-agent-home/conf/wrapper.conf
...
# Initial Java Heap Size (in MB)
wrapper.java.initmemory=256

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=8192

[Mon Jul 13 18:10:35]# top
...
3736 bamboo 20 0 9709340 769112 34456 S 0.3 4.7 0:40.07 java -Dbamboo.home=/opt/bamboo/bamboo-agent-home -Dbamboo.agent.ignoreServerCertName=false -Xms256m -Xmx4096m

I can specify max heap size lower than 4gb and the heap size will change OK, but it won't go higher than 4gb.

The agent is version 5.9.1

Thanks
Ted

[Jeyanthan I]

Hi @Ted Thompson ,

This sounds like you're using a 32 bit JDK.The maximum theoretical heap limit for the 32-bit JVM is 4G as explained in Oracle FAQ page. You may check it by running java -version.

Cheers, Jey.

[Ted Thompson]

Thanks Jey, unfortunately that doesn't seem to be the problem.

[Tue Jul 14 09:50:10]# java -version
java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

[Tue Jul 14 09:50:16]# uname -r
3.10.0-1127.13.1.el7.x86_64

[Tue Jul 14 09:50:27]# cat /etc/centos-release
CentOS Linux release 7.8.2003 (Core)

I'll keep digging.

Thanks,
Ted

[Ted Thompson]

Interestingly, if I put the heap sizes as additional options in the wrapper and comment out the predefined heap option it works?

[Tue Jul 14 10:13:03]# cat /opt/bamboo/bamboo-agent-home/conf/wrapper.conf
# Java Application
wrapper.java.command=java

# Java Main class.
wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp

# Java Classpath (include wrapper.jar)
wrapper.java.classpath.1=../lib/wrapper.jar
wrapper.java.classpath.2=../lib/bamboo-agent-bootstrap.jar

# Java Library Path (location of Wrapper.DLL or libwrapper.so)
wrapper.java.library.path.1=../lib

# The Bamboo Agent home configuration file
wrapper.java.additional.1=-Dbamboo.home=/opt/bamboo/bamboo-agent-home
wrapper.java.additional.2=-Dbamboo.agent.ignoreServerCertName=false

wrapper.java.additional.3=-Xms256m
wrapper.java.additional.4=-Xmx8192m

...

# Initial Java Heap Size (in MB)
#wrapper.java.initmemory=256

# Maximum Java Heap Size (in MB)
#wrapper.java.maxmemory=8192

[Tue Jul 14 10:16:43]# top
...
-Dbamboo.agent.ignoreServerCertName=false -Xms256m -Xmx8192m -Djava.library.path=../lib -classpath ../lib/wrapper.jar:../lib/bamboo-a

The predefined heap areas need to be commented out otherwise you end up with two lots of heap options being appended to the command.

[Jeyanthan I]

Hi Ted,

Could it be that you have multiple Java installed on your server? The following command should list all the available Java versions:

alternatives --config java

I usually find the JVM parameters and the Java path of Bamboo agent process by running this:

ps -ef | grep bamboo-agent

Also, what happens when you try to start Bamboo without a service wrapper?

You can start using the direct agent jar from <baseURL>/admin/agent/addRemoteAgent.action as described in here.

java -jar bamboo-agent-2.2.2.jar http://<host>:8085/agentServer/ <tokenKey>

Cheers,
Jey

[Ted Thompson]

Hi Jey,

We we have a few versions of java as these are also JDK capabilities.

[Tue Jul 14 21:44:06]# alternatives --config java

There are 4 programs which provide 'java'.

Selection Command
-----------------------------------------------
+ 1 /usr/java/jre1.8.0_25/jre/bin/java
2 /usr/lib/jvm/zulu-8/bin/java
3 /usr/java/jdk1.8.0_144/jre/bin/java
* 4 /usr/lib/jvm/zulu-11/bin/java

Enter to keep the current selection[+], or type selection number:

Do you mean lib/bamboo-agent-bootstrap.jar ?

Thanks,
Ted

[Jeyanthan I]

Hi Ted,

Thanks for sharing the output. It looks like your server is pointing to a JRE binary as its default to start the agent. Could you please use one of the supported variants of JDK as described in our supported platforms page?

Do you mean lib/bamboo-agent-bootstrap.jar ?

Do you find a jar file under the section Running the agents without the service wrapper  in the page Bamboo administration -> Agents -> Install remote agent? I meant this file.

Cheers,
Jey