Why does bamboo agent stop spontaneously

Question

I have installed Bamboo and started learning it. I've started getting accustomed to bamboo notions like agents, plans, builds, etc. However, I have a problem with one of my remote agents stopping after having been up for some time. I have two remote agents: One of them is my own Windows 10 machine and the other one is Linux. In both hosts I've run the wrapper jar like this:

java -jar atlassian-bamboo-agent-installer-6.8.1.jar http://bamboo.mycompany.com:8085/agentServer/

I've started the wrapper again and again several times. Then I found out that mismatch in time between the bamboo and remote agent hosts can cause this. There indeed was a mismatch between times, I fixed it. But the problem won't go away. I couldn't find anything useful in the logs. Here are the last several lines of the log file before the wrapper stopped:

INFO | jvm 1 | 2019/05/23 18:04:43 | 2019-05-23 17:04:43,712 INFO [0-BAM::kubemaster.mycompany.com::Agent:pool-3-thread-1] [BuildAgentControllerImpl] Agent 1114114 checking build queue for executables...
INFO | jvm 1 | 2019/05/23 18:09:43 | 2019-05-23 17:09:43,725 INFO [0-BAM::kubemaster.mycompany.com::Agent:pool-3-thread-1] [BuildAgentControllerImpl] Agent 1114114 checking build queue for executables...
INFO | jvm 1 | 2019/05/23 18:14:43 | 2019-05-23 17:14:43,741 INFO [0-BAM::kubemaster.mycompany.com::Agent:pool-3-thread-1] [BuildAgentControllerImpl] Agent 1114114 checking build queue for executables...
INFO | jvm 1 | 2019/05/23 18:19:43 | 2019-05-23 17:19:43,753 INFO [0-BAM::kubemaster.mycompany.com::Agent:pool-3-thread-1] [BuildAgentControllerImpl] Agent 1114114 checking build queue for executables...
INFO | jvm 1 | 2019/05/23 18:24:43 | 2019-05-23 17:24:43,767 INFO [0-BAM::kubemaster.mycompany.com::Agent:pool-3-thread-1] [BuildAgentControllerImpl] Agent 1114114 checking build queue for executables...
INFO | jvm 1 | 2019/05/23 18:29:43 | 2019-05-23 17:29:43,790 INFO [0-BAM::kubemaster.mycompany.com::Agent:pool-3-thread-1] [BuildAgentControllerImpl] Agent 1114114 checking build queue for executables...
INFO | jvm 1 | 2019/05/23 18:34:43 | 2019-05-23 17:34:43,804 INFO [0-BAM::kubemaster.mycompany.com::Agent:pool-3-thread-1] [BuildAgentControllerImpl] Agent 1114114 checking build queue for executables...
INFO | jvm 1 | 2019/05/23 18:39:43 | 2019-05-23 17:39:43,814 INFO [0-BAM::kubemaster.mycompany.com::Agent:pool-3-thread-1] [BuildAgentControllerImpl] Agent 1114114 checking build queue for executables...
INFO | jvm 1 | 2019/05/23 18:44:43 | 2019-05-23 17:44:43,825 INFO [0-BAM::kubemaster.mycompany.com::Agent:pool-3-thread-1] [BuildAgentControllerImpl] Agent 1114114 checking build queue for executables...
INFO | jvm 1 | 2019/05/23 18:49:43 | 2019-05-23 17:49:43,840 INFO [0-BAM::kubemaster.mycompany.com::Agent:pool-3-thread-1] [BuildAgentControllerImpl] Agent 1114114 checking build queue for executables...
INFO | jvm 1 | 2019/05/23 18:54:43 | 2019-05-23 17:54:43,872 INFO [0-BAM::kubemaster.mycompany.com::Agent:pool-3-thread-1] [BuildAgentControllerImpl] Agent 1114114 checking build queue for executables...
STATUS | wrapper | 2019/05/23 18:55:16 | <-- Wrapper Stopped
Waiting for the agent process to finish...
Underlying agent process has terminated cleanly,
Bamboo Agent Installer is shutting down, the underlying agent process has already been shut down.

As you can see, there's nothing significant in the logs that could indicate the root of the problem. Has anybody had this problem?

UPDATE

I've noticed one weird thing in the remote agent System properties. In the System time it shows 16:20, when in fact it's 17:20. I confirm that by logging into the remote agent host and issuing the date command. And for the Up time it shows 1 hour, 52 minutes, 28 seconds (since Mon Jun 03 15:28:13 AZT 2019), which is weird because if we add 1 hour 52 minutes to 15:28, it makes 17:20, which the actual time. You can see that in the image attached

mikayil-abdullah · Answer

Hi, Marcin. Thanks for the polite reply.

I'd contacted the Atlassian Support at the same time and have already received the support. Here's what they said:

What we usually recommend is that after you run the installer, you stop it, and then start the remote agent using either the service (as in Windows) or using the command:

<Bamboo remote agent home directory>/bin/bamboo-agent.sh start

which will start the remote agent in the background and exit.

That seems to solve the problem. However, the one hour difference is weird because both the Bamboo server machine and the agent machine have the same, Asia/Baku timezone.

Marcin Gardias · Answer

Clock differences/inconsistencies can definitely affect agents, although I would expect some message about heartbeats timing out in such case.

Still worth looking into. I would recommend that Bamboo server and agent have the same timezone.