How to scp from remote host to bamboo EC2 instance?

After googling around, I stumbled upon this:

Which led me to experiment for the past several hours with decent success. However, I seem to have run into a snag.

The story so far:

I have a build in bamboo that needs to scp a file from a remote server to the Amazon EC2 instance that bamboo is running on. 

I can ssh into the EC2 isntance as root and, using an ssh key, successfully scp the desired file from the remote server to the EC2 instance.

I can ssh into the EC2 as the ec2-user provided by bamboo and, after appending "sudo" am again successful.

However, when I paste this command into a script task in bamboo, it fails.

If I leave off the "sudo" I get: "Host key verification failed.".

If I append the "sudo" I get: "sudo: no tty present and no askpass program specified".

By all accounts this should be working, so there must be something different about the user/permissions/something to do with bamboo running the command as opposed to myself, but I simply don't know what and I can't find any documentation that states this difference. 

Any insight would be greatly appreciated.

Thank you.

2 answers

1 accepted

1 vote
Accepted answer

While Przemek's answer will work, security wise it is not the best answer. If your bamboo instance is compromised through a fault in bamboo's web services, the attacker would have total control over your bamboo machine. And since bamboo usually has privileged control over webservers for deployment, those could be at risk too.

When bamboo runs scripts, its running as the bamboo user account on the bamboo machine. What you'll want to do is either handle these things using options to ssh (like ignore host key verification), or you can su to the bamboo user (su - bamboo), and run your ssh command and answer yes to adding the host key to the known hosts (this will live in /home/bamboo/.ssh).

I've seen @Przemyslaw Bruski's answer but held off on that particular fix as it was touted as insecure practice. David, Thank you for the response. I've have actually already done this. I took the public key and put it in /home/bamboo/.ssh/authorized_keys as was instructed in the link in my post. This didn't change anything. I also am already using the "-i <path to privatekey>" and tried "-o StrictHostKeyChecking=no". The former is necessary for this to work, but the latter did not help my cause. Any other ideas?

Just a note, I'm back at it again today after some sleep and I've notice that while using "-i <pathtokey>" and "-o StrictHostKeyChecking=no" I'm getting the following in the bamboo logs: "Warning: Permanently added '<server>' (RSA) to the list of known hosts. Permission denied, please try again. Permission denied, please try again. Permission denied (publickey,password)." So even though I'm specifying a private key, and the command works fine when I run it manually, it's still asking for a password when bamboo tries to run it.

I solved this.  Thank you very much for the responses. While David's response was correct, I was encountering an issue: The bamboo user did not have permissions to use the ssh key where I had it saved. (On the EBS) The bamboo user also did not have write permissions to save the file to the location I specified (Also on the EBS) To solve: I switch user (su) to bamboo and attempted to run the scp command from there. (This is what I was missing from my last attempts.) This allowed me to see more screen messages that led me to discover that the bamboo user was unable to read the ssh key from its current location. It also informed me that it was unable to copy to the specified file. (Both bits of info that I was not getting from the bamboo job logs) I moved the private ssh key into the bamboo home directory and also changed the destination of the copied file to bamboo's home directory so that both were accessible to the bamboo user. The command now runs manually and from the bamboo job task. 
 Thanks again for your help. Sometimes just having a bit of confirmation of previous steps is necessary. Cheers.

Glad it worked for you. One thing to note (for others who may read this), since you're storing a private key that has access to other machines, you should make sure its for an unprivileged user account on those remote machines. If you use root, and bamboo becomes compromised, you're remote machines are essentially rooted by the attacker.

0 votes
Add the following to your instance setup script (accessible via image configuration)
echo "bamboo ALL=(ALL) NOPASSWD:ALL" >$sudoersFile
chmod 0440 $sudoersFile

Suggest an answer

Log in or Sign up to answer
Community showcase
Asked Thursday in Jira Ops

I'm John Allspaw, Ask Me Anything about incident analysis and postmortems

I'm John Allspaw, co-founder of   Adaptive Capacity Labs, where we help teams use their incidents to learn and improve. We bring research-driven methods and approaches to drive effective inciden...

4,339 views 20 13
View question

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you