Waiting for dependent services, like databases Edited

Hi there,

I'm pretty new to Bitbucket pipelines, but have worked extensively with Jenkins Pipelines before.

For my current project (Angular4, Spring-Boot, MariaDB), I have set up a pipeline with a custom Docker container, linked to MariaDB via linking (database service in pipelines).

I encountered problems with randomly failing builds, because the MariaDB container wasn't ready at the time the actual maven build started. Therefore, I was looking to find a suitable solution and wrote a script that I'd like to share. I hope it comes in handy for you :)

File: wait-for-mysql.sh

#!/bin/bash -e

statusFile=/tmp/mysql-status
while [[ true ]]; do
telnet 127.0.0.1 3306 &> ${statusFile}
status=$(grep "Connection refused" ${statusFile} | wc -l)
echo "Status: $status"

if [[ "${status}" -eq 1 ]]; then
echo "MySQL not running, waiting."
sleep 1
else
rm ${statusFile}
echo "MySQL running, ready to proceed."
break;
fi
done

 File: bitbucket-pipelines.yml

image: alphabrik/maven-angular-bootstrap:latest

pipelines:
default:
- step:
caches:
- maven
- node
- node-app
script:
- bash wait-for-mysql.sh
- mvn install
services:
- database

definitions:
caches:
node-app: my-app/node_modules
services:
database:
image: mariadb:10.1
environment:
MYSQL_DATABASE: 'awesomeDBName'
MYSQL_USER: 'youruser'
MYSQL_PASSWORD: 'yourpassword'
MYSQL_ALLOW_EMPTY_PASSWORD: 'true'
volumes: path/to/your/init-dump.sql:/docker-entrypoint-initdb.d

 Now, how does it work?

My custom docker image includes everyhting needed for testing Spring and Angular apps. If you're using some different image, make sure it contains the 'telnet' program.

The pipeline steps will call the above mentioned script, before maven is executed. The script attempts a connection to the mysql-server via telnet in a loop. Once the connection is successful, the build can continue.

 

Since bitbucket pipelines currently doesn't offer a method to wait for services, I thought I'd start this discussion and see what kind of solutions the community can find for other dependent services.

Feel free to ask any questions!

 

Cheers,

Daniel

2 comments

Yes, it would be nice if Pipelines could have something like a serviceready feature instead of having to write custom scripts to poll if the service has started.

services:
database:
image: mariadb:10.1
environment:
...
serviceready:
script:
sh test_my_sql.sh

Hi all. Thanks for the script Daniel, that was almost exactly what I needed. I found that despite the "&" backgrounding, telnet would keep the script running even when succeeding in its connection, so I amended the telnet line to exit immediately upon success.

Additionally, through some trial and error I confirmed my suspicion that the approach of calling it a success when "Connection refused" doesn't appear is flawed, since it passes at system startup - probably due to a message like "No route to host". As a result, I have amended your script to the following:

#!/bin/bash -e

statusFile=/tmp/mysql-status
while [[ true ]]; do
echo "" | telnet db2.novarumcloud.com 5432 &> ${statusFile}
status=$(grep "Connected to" ${statusFile} | wc -l)
echo "Status: $status"

if [[ "${status}" -ne 1 ]]; then
echo "MySQL not running, waiting."
sleep 1
else
rm ${statusFile}
break;
fi
done

Apart from that, I'd like to mention where I had to insert this holder script on my local JIRA installation, in case anyone else needs to figure out the same:

/etc/init.d/jira1

# JIRA Linux service controller script
cd "/opt/atlassian/jira-core/bin"

case "$1" in
start)
bash wait-for-mysql.sh
./start-jira.sh
;;
stop)
./stop-jira.sh
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
;;
esac

Hope that expands the value of an already useful answer to a wider audience.

Hey @Mario Staykov, shouldn't be "${status} -ne 0" ?

Comment

Log in or Sign up to comment
Community showcase
Published Nov 06, 2018 in Bitbucket

Upgrade Best Practices

Hello! My name is Mark Askew and I am a Premier Support Engineer for products Bitbucket Server/Data Center, Fisheye & Crucible. Today, I want to bring the discussion that Jennifer, Matt, and ...

210 views 3 7
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you