Bamboo - After failed deployment SSH tasks throw permission errors

Todd Ellner April 27, 2015

We have a deployment task for moving .wars and service configuration from Bamboo builds to tomcat test servers.

  1. Clean working directory
  2. Artifact download
  3. SCP Task- Copy war to server
  4. SSH - Clean next folder
  5. SSH - Explode war
  6. SSH - Explode serverconfig
  7. SSH - restart server

If the process is interrupted or times out the next deployment fails on step 4 ( rm -rf /opt/acton/tomcat/releases/theapp-next ) with a flood of "permission denied".

The user and ssh key in those steps have access to the directories. When I log in with them I can clear out theapp-next by hand and subsequent deployments succeed.

Does anyone know what causes this behavior and how to work around it?

3 answers

1 accepted

0 votes
Answer accepted
rsperafico
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 27, 2015

Hello Todd,

Thank you for your question.

At this stage is difficult to say what is going on as it would be better having the build log to verify the tasks you are running and their output. Would be possible to provide those information, please?

Also, I would suggest you on running the same commands from Terminal using the same user running Bamboo just to narrow this issue down and determine if the issue is related to Bamboo or the tasks you are running, please.

Kind regards,
Rafael P. Sperafico
Atlassian Support

0 votes
Todd Ellner April 28, 2015

Note: this is from a stored set of logs. When I cleaned out the acton-next directory by hand it had the same ownership.

 

Sadly, when I tried to do a fresh example of the issue I started a deployment and stopped it. The bamboo agent has now been deploying for half an hour; deployments typically take 3-4 minutes. Hitting the "stop deployment" button does nothing. Several Atlassian JIRA tickets have noted this problem. Every time they've been closed as "fixed" with subsequent comments that the issue is not fixed and advice that a "workaround" is to restart the bamboo server - not a good option in a high-use production CI server. This isn't your fault, obviously. Just refreshing an ongoing pain-point to send up the queue

0 votes
Todd Ellner April 28, 2015

I'll do a run and provide logs when I get a few free minutes. The reason I'm quick to lay this at Bamboo's door, or at least Bamboo running ssh tasks is that the command I run by hand is exactly the one that fails from Bamboo after (and only after) a build is run after a build fails or is aborted.

${bamboo.deploy.environment}
Username: installer
Key without passphrase: <installer key>
rm -rf /opt/acton/tomcat/releases/acton-next

 

From the build log:

simple 23-Apr-2015 13:30:15 Build Deployment of 'dbr-segmentation-9' on 'qa-tomcat12' started building on agent Deploy QA Agent 3
simple 23-Apr-2015 13:30:15 Build working directory is /opt/bamboo/bamboo/bamboo-home/xml-data/build-dir/11927553-12025862
simple 23-Apr-2015 13:30:15 Executing build Deployment of 'dbr-segmentation-9' on 'qa-tomcat12'
simple 23-Apr-2015 13:30:15 Starting task 'Clean working directory task' of type 'com.atlassian.bamboo.plugins.bamboo-artifact-downloader-plugin:cleanWorkingDirectoryTask'
simple 23-Apr-2015 13:30:15 Cleaning working directory '/opt/bamboo/bamboo/bamboo-home/xml-data/build-dir/11927553-12025862'
simple 23-Apr-2015 13:30:16 Finished task 'Clean working directory task' with result: Success
simple 23-Apr-2015 13:30:16 Starting task 'Download release contents' of type 'com.atlassian.bamboo.plugins.bamboo-artifact-downloader-plugin:artifactdownloadertask'
simple 23-Apr-2015 13:30:16 Preparing to download plan result ACTON-AC1986-9 artifact: Shared artifact: [acton.war], pattern: [acton*.war] anchored at: [acton]
simple 23-Apr-2015 13:30:19 Artifact [acton.war] downloaded successfully in 3s to working directory
simple 23-Apr-2015 13:30:19 Preparing to download plan result ACTON-AC1986-9 artifact: Shared artifact: [serverconfig.tar.gz], pattern: [serverconfig.tar.gz] anchored at: [acton]
simple 23-Apr-2015 13:30:19 Artifact [serverconfig.tar.gz] downloaded successfully in 0s to working directory
simple 23-Apr-2015 13:30:19 Finished task 'Download release contents' with result: Success
command 23-Apr-2015 13:30:19 Substituting variable: ${bamboo.deploy.environment} with qa-tomcat12
simple 23-Apr-2015 13:30:19 Starting task 'Copy acton.war to server' of type 'com.atlassian.bamboo.plugins.bamboo-scp-plugin:scptask'
simple 23-Apr-2015 13:30:19 Connecting to qa-tomcat12 on port: 22
simple 23-Apr-2015 13:30:19 Uploading '/opt/bamboo/bamboo/bamboo-home/xml-data/build-dir/11927553-12025862/serverconfig.tar.gz' to '/opt/acton/tomcat/releases'...
simple 23-Apr-2015 13:30:20 '/opt/bamboo/bamboo/bamboo-home/xml-data/build-dir/11927553-12025862/serverconfig.tar.gz' was uploaded successfully.
simple 23-Apr-2015 13:30:20 Uploading '/opt/bamboo/bamboo/bamboo-home/xml-data/build-dir/11927553-12025862/acton.war' to '/opt/acton/tomcat/releases'...
simple 23-Apr-2015 13:30:39 '/opt/bamboo/bamboo/bamboo-home/xml-data/build-dir/11927553-12025862/acton.war' was uploaded successfully.
simple 23-Apr-2015 13:30:39 Finished task 'Copy acton.war to server' with result: Success
command 23-Apr-2015 13:30:39 Substituting variable: ${bamboo.deploy.environment} with qa-tomcat12
simple 23-Apr-2015 13:30:39 Starting task 'Clean -next folder' of type 'com.atlassian.bamboo.plugins.bamboo-scp-plugin:sshtask'
simple 23-Apr-2015 13:30:39 Connecting to qa-tomcat12 on port: 22
simple 23-Apr-2015 13:30:39 Executing [rm -rf /opt/acton/tomcat/releases/acton-next]
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/selfserve/signupForm.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/selfserve/index.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/selfserve/signupFormSubmitAjax.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/selfserve/signupConfirm.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/selfserve/signupConfirmAjax.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/chooserlib/messageDialog_campaign.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/chooserlib/messageDialog_webinar.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/chooserlib/listDialog.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/chooserlib/messageDialog_sent.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/chooserlib/messageDialog.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/chooserlib/messageDialog_draft.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/chooserlib/messageDialog_template.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/chooserlib/messageDialog_program.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/chooserlib/messageDialog_form.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/chooserlib/messageDialog_scheduled.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/_segment/by-query.jsp': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/_segment/stage-expressions-metadata.json': Permission denied
error 23-Apr-2015 13:30:40 rm: cannot remove `/opt/acton/tomcat/releases/acton-next/_segment/images/accept.png': Permission denied

rsperafico
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 28, 2015

Hello Todd, Please, connect to qa-tomcat12 on port: 22 manually and check the owner to "/opt/acton/tomcat/releases/acton-next" and its subdirectories/files. — Kind regards, Rafael P. Sperafico Atlassian Support

Todd Ellner April 28, 2015

Owner: installer Group: tomcat

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events