Bitbucket pipline integration with Ansible tower ./tower.bash: Permission denied

[Ian Williamson]

I am trying to follow the instructions for setting up pipline to Ansible tower as documented at https://bitbucket.org/AnsibleByRedHat/ansible-tower-pipelines/src/master/

When I commit the pipeling launches the bitbucket-pipelines.yml and successfully loaded the "pip install ansible-tower-cli"

However the next step failed with the error

./tower.bash bash: ./tower.bash: Permission denied

I seams to me that the pipline execution is not able to access the tower.bash file.

If I delete tower.bash the error changes to file not found. I change the batch call to an ssh command and that did executed.  is there some file permission setting in the repository or a sudo command in the YAML that I need to tweak some how?

bitbucket-pipelines.yml

# This is a sample build configuration for the Ansible Tower integration.
# Do not use tabs to indent.
# You can specify a custom docker image from Dockerhub
# for your build environment.
image: python:2.7

pipelines:
default:
- step:
script: # Modify the commands below to build your repository.
- pip install ansible-tower-cli
- ./tower.bash

tower.bash

#!/bin/bash -ex

echo "Configuring Tower Settings"
hostval=$(tower-cli config host $host)
userval=$(tower-cli config username $username)
passwordval=$(tower-cli config password $password)

if [[ $userval == "username: " ]] || [[ $passwordval == "password: " ]]
then
echo "WARNING: Configuration has not been fully set";
echo " You will want to run the $ tower-cli config ";
echo " command for host, username, and password ";
fi

echo " current configuration settings:"
echo $hostval
echo $userval
echo $passwordval

tower-cli "config verify_ssl false"

# Let's run a tower-cli job
tower-cli job launch --job-template $ID --monitor

[Ian Williamson]

Yay! I did it! W00t FTW

Hello To whom it may concern at Ansible / Red Hat / Atlassian

The following documentation does not demonstrate a working configuration and is misleading:

https://www.ansible.com/integrations/devops-tools/atlassian-bitbucket-pipelines

https://www.ansible.com/integrations/devops-tools/atlassian

https://bitbucket.org/AnsibleByRedHat/ansible-tower-pipelines/src/master/

Below is a working configuration. Please review this and update your documentation accordingly.

With this configuration when a commit happens to the Bitbucket repository, pipeline will call a job from Ansible Tower named  “Tower_Test”

Set variables in Bitbucket Pipeline

ID Tower_Test

host 10.10.10.10

username tower_user

password password

Note: select the lock icon when creating the password variable so that it is hidden

In the repo create the following files  

bitbucket-pipelines.yml

# This is a sample build configuration for the Ansible Tower integration.

# You can specify a custom docker image from Dockerhub

# for your build environment.

image: python:2.7

pipelines:

 default:

   - step:

       script: # Modify the commands below to build your repository.

           - pip install ansible-tower-cli

           - wget http://kr.archive.ubuntu.com/ubuntu/pool/main/t/tcl8.6/libtcl8.6_8.6.8+dfsg-4_amd64.deb

           - wget http://kr.archive.ubuntu.com/ubuntu/pool/universe/e/expect/tcl-expect_5.45.4-2_amd64.deb

           - wget http://kr.archive.ubuntu.com/ubuntu/pool/universe/e/expect/expect_5.45.4-2_amd64.deb

           - apt-get install ./libtcl8.6_8.6.8+dfsg-4_amd64.deb -y

           - apt-get install ./tcl-expect_5.45.4-2_amd64.deb -y

           - apt-get install ./expect_5.45.4-2_amd64.deb -y

#            - apt-get install -y gdebi && sudo gdebi ./expect_5.45.4-2_amd64.deb -y

           - chmod 755 tower.bash

           - chmod 755 tower.exp

           - chmod 755 qtower

           - ./tower.bash

Note: repositories available to the python:2.7 image do not include expect. expect and its dependencies are sourced from an asia/pacific Ubuntu mirror. This is less than optimal, please suggest how to make this better? Like a better Docker image.

tower.bash

#!/bin/bash -ex

#configur tower-cli settings

echo "Configuring Tower Settings"

tower-cli config host $host

tower-cli config username $username

tower-cli config verify_ssl false

# lets use expect to enter the pasword without echoing and get the Oath2 token from tower

./tower.exp $password

echo " current configuration settings:"

cat ~/.tower_cli.cfg

# Let's run a tower-cli job

tower-cli job launch --job-template $ID --monitor

tower.exp

#!/usr/bin/expect -f

set MYVAR [lindex $argv 0]

set send_slow {1 .1}

set timeout -1

spawn ./qtower

match_max 100000

expect -exact "Password: "

sleep .1

send -- $MYVAR

send -s -- "\r"

sleep .1

send -s -- "\r"

expect eof

qtower

tower-cli login  $username

Turn on Bitbucket Pipeline

The resulting pipeline log looks like this

Build setup

pip install ansible-tower-cli

wget http://kr.archive.ubuntu.com/ubuntu/pool/main/t/tcl8.6/libtcl8.6_8.6.8+dfsg-4_amd64.deb2s

wget http://kr.archive.ubuntu.com/ubuntu/pool/universe/e/expect/tcl-expect_5.45.4-2_amd64.deb1s

wget http://kr.archive.ubuntu.com/ubuntu/pool/universe/e/expect/expect_5.45.4-2_amd64.deb1s

apt-get install ./libtcl8.6_8.6.8+dfsg-4_amd64.deb -y9s

apt-get install ./tcl-expect_5.45.4-2_amd64.deb -y<1s

apt-get install ./expect_5.45.4-2_amd64.deb -y<1s

chmod 755 tower.bash<1s

chmod 755 tower.exp<1s

chmod 755 qtower

./tower.bash

+ ./tower.bash

+ echo 'Configuring Tower Settings'

+ tower-cli config host 10.10.10.10

Configuring Tower Settings

Configuration updated successfully.

+ tower-cli config username tower_user

Configuration updated successfully.

+ tower-cli config verify_ssl false

Configuration updated successfully.

+ ./tower.exp $password

spawn ./qtower

Password:

{

"id": 36,

"type": "o_auth2_access_token",

"url": "/api/v2/tokens/36/",

"created": "2018-11-02T02:11:59.819468Z",

"modified": "2018-11-02T02:11:59.837189Z",

"description": "Tower CLI",

"user": 1,

"refresh_token": null,

"application": null,

"expires": "3018-03-05T02:11:59.812174Z",

"scope": "write"

}

Configuration updated successfully.

+ echo ' current configuration settings:'

+ cat /root/.tower_cli.cfg

current configuration settings:

[general]

host = 10.10.10.10

username = tower_user

verify_ssl = false

oauth_token = xxxxxxxxxxxxxxxxxxxxxx

+ tower-cli job launch --job-template Tower_Test --monitor

Current status: running

------Starting Standard Out Stream------

SSH password:

PLAY [all] *********************************************************************

TASK [Check the date on the server.] *******************************************

changed: [ansible]

changed: [node-1]

...

[Ian Williamson]

This line also runs the script

- source tower.bash

There was something in the ansible-tower-cli documentation about invoking inventory scripts with this, so I think it is most likely being picked up by the ansible-tower-cli process.

[Ian Williamson]

OK so I got the script to run now, but the ssh channel is having problems with the password. I have tried setting the password variable as open and locked.

I tried invoking "pip install passlib" but made no difference. 

I'll  STFW for getpass and have another go tomorrow.

source tower.bash

source tower.bash

Configuring Tower Settings current configuration settings:

Configuration updated successfully.

/usr/local/lib/python2.7/getpass.py:83:

GetPassWarning: Can not control echo on the terminal.

passwd = fallback_getpass(prompt, stream)

Warning: Password input may be echoed.

Password for ssh_password:

Aborted!

[Ian Williamson]

Another great suggestion from Os Demo, so obvious (facepalm)

- chmod 755 tower.bash
When the batch file permission are set in the YAML It seams that the bash script now runs as intended, but something very weird is happening to the variable declarations. The environment variable set in Bitbucket settings are being passed into the script but the script is resting all variable to equal 'Configuration updated successfully.'
Is this some sort of standard in / standard error mix up ?

I think this could be the problem... The instructions say "Add tower.bash to your build configuration." but I don't know what that is, do I need to place the tower.bash file somewhere other than in the project?

./tower.bash
./tower.bash
+ echo 'Configuring Tower Settings'
Configuring Tower Settings
++ tower-cli config host x.x.x.x (this is the correct address for the tower)
+ hostval='Configuration updated successfully.'
++ tower-cli config username xxxxxxx (this was the correct user)
+ userval='Configuration updated successfully.'
++ tower-cli config password $password
+ passwordval='Configuration updated successfully.'
+ [[ Configuration updated successfully. == \u\s\e\r\n\a\m\e\:\ ]]
+ [[ Configuration updated successfully. == \p\a\s\s\w\o\r\d\:\ ]]
+ echo ' current configuration settings:'
+ echo Configuration updated successfully.
+ echo Configuration updated successfully.
+ echo Configuration updated successfully.
+ tower-cli config verify_ssl false
current configuration settings:
Configuration updated successfully.
Configuration updated successfully.
Configuration updated successfully.
Configuration updated successfully.
+ tower-cli job launch --job-template Ansible_tower_job --monitor

[Jeroen De Raedt]

Hi @Ian Williamson,

in your tower.bash script I see that you configure for example $hostval like this: 

hostval=$(tower-cli config host $host)

This probably doesn't return the value that is used for the host as you think it does. It just results in capturing the log output of the `tower-cli config host $host` command - which is `Configuration updated successfully.`

If you do want to print the host value that is used, just print $host

[Ian Williamson]

Hello Jeron

Thank you for confirming that. I kind of thought so. You know I got the tower.bash script as the official integration of Bitbucket and Ansible Tower from

https://www.ansible.com/integrations/devops-tools/atlassian

As a noob, well out of his depth, it is nice to know that it's not a mistake I made in flowing the instructions.

I have installed tower-cli on a VM and calling tower jobs is super easy in interactive.
To get non interactive automation Bitbucked sets variables for a bash script. The password variable in an obfuscated one. I thought the script was then trying to manipulate the variable into the tower-cli setup commands.

Playing with the variable, I now realize that the original script set up $hostval, $userval, $passwordval but did not actual use them to run the commands to setup the connection to the tower. So I set these variable in Bitbucket and ran them as commands in the script. I could set the host and the username but not the password. I guess I need some expert knowledge on how the underlying python does password management, things like getpass and passlib.

From the pipeline log
+ cat /root/.tower_cli.cfg
[general]
host = (this was the corect IP addr)
username = (this was the corect username)
password = $password
verify_ssl = false

So the pipeline ends with the errors:

Error: Could not get job_template. Invalid Tower authentication credentials (HTTP 401).

or

/usr/local/lib/python2.7/getpass.py:83: GetPassWarning: Can not control echo on the terminal.
passwd = fallback_getpass(prompt, stream)
Warning: Password input may be echoed.
Password for ssh_password:
Aborted!

[Jeroen De Raedt]

Hi @Ian Williamson,

it looks like you disabled Pipelines on your repository, so I can not check the output of your tests anymore? 

Googling a bit for your first error brought me to https://github.com/ansible/tower-cli/issues/579 - which suggests that you need to login with the tower cli first. However, I don't see this being part of the original script that tower cli provided for the integration. 

My guess is that the tower-cli has been updates since they published this integration, and possibly that update broke the integration. 

It might be worth raising a bug for Ansible so they can look into it and fix it. You can raise a bug here: https://github.com/ansible/ansible/issues (make sure you link to the integration page that you were using). 

[Ian Williamson]

Thanks Jeroen

I will get on it now

Ian

[Ian Williamson]

FYI - I have raised a bug over at Ansible

Ansible Tower integration with Atlassian Bitbucket. documented method is no longer functional #47033

[Ian Williamson]

The Bug was closed with a request to post to AWX. I have done this now

https://groups.google.com/forum/#!topic/awx-project/ZaiC5aoYJ38

Hello AWX community

I am trying to CI/CD from JIRA -> Bitbicket -> Ansible Tower. I think the documentation on how to pipeline from Bitbicket is out of date now tower-cli uses Oauth2.Can anyone please guide me on how to script the setup of the tower-cli connection to tower. 
There is official documentation here:https://bitbucket.org/AnsibleByRedHat/ansible-tower-pipelines/src/master/And it provides an example "tower.bash"
This example references the pre tower 3.3 Oauth2 login commands:$ tower-cli config host tower.example.com$ tower-cli config username user$ tower-cli config password pass
To connect to the current tower version you need to use $ tower-cli config host tower.example.com$ tower-cli login usernamePassword:
I am not very experienced at this stuff but it looks to me like, In the legacy mode a variable can be used to set the password in to the ~/.tower-cli.cfg file. I had a play with the "tower-cli login" command. I don't know of a way to answer the prompt for the password. I think this can not be handled in the script. I think it produces the error “/usr/local/lib/python2.7/getpass.py:83: GetPassWarning: Can not control echo on the terminal.  passwd = fallback_getpass(prompt, stream)Warning: Password input may be echoed.”
Can someone with better coding than me please confirm that I am correct or not. Please suggest a way forward. Is there such thing as an Oath2 command in bash that can take a variable for the password? 
Best RegardsIan

[Ian Williamson]

I managed an expect script that logs in without a prompt. But expect is not available in the Docker container and it was not available to install. 

[Ian Williamson]

I think I may have made a step forward with this, but I am not certain. The script will run if you change the line in the bitbucket-pipelines.yml from "- ./tower.bash" to "- sh tower.bash" 

Maybe all I have achieved is is running the script in sh instead of ansible-tower-cli. I am now reading the ansible-tower-cli documentation. will come back if I find anything.

Thanks to Os Demo for pointing me in the right direction.

[Osvaldo Demo]

Glad it worked Ian. :)