How do you deploy from Pipelines to a remote Docker Swarm cluster ?

Sandeep Srinivasa December 21, 2017

hi,

I'm trying to get a Bitbucket Pipelines CD setup with a Docker Swarm cluster.

I messaged Geoff on Twitter and he suggested I post here - https://twitter.com/realfakegeoff/status/943729735703576577

You guys have deployment documentation for Kubernetes - https://confluence.atlassian.com/bitbucket/deploy-to-kubernetes-892623297.html

But there is no equivalent documentation for Docker Swarm. Please add documentation for deploys to docker swarm.

4 answers

2 votes
Pavel G September 24, 2020


Here are more details how to use Portainer webhooks for updating Docker swarm service from Bitbucket pipeline:

1.  get docker service webhook url: https://appsaloon.be/blog/service-webhooks-in-portainer/
 url will be in https://portainer.url.com/api/webhooks/TOKEN format

2. store TOKEN to a PORTAINER_WEBHOOK_TOKEN secret variable: https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/

3.
- step:
name: Publish app to Docker Hub
trigger: automatic
script:
- docker login -u $DOCKER_HUB_USER -p $DOCKER_HUB_PWD
- docker build -t XXX:$BUILD_VERSION_APP .
- docker push XXX:$BUILD_VERSION_APP
- step:
name: Deploy to staging
deployment: staging
trigger: manual
script:
- wget --server-response --method POST "https://portainer.url.com/api/webhooks/$PORTAINER_WEBHOOK_TOKEN?tag=$BUILD_VERSION_APP"

 

1 vote
Phagedo August 7, 2019

Hi! We are using webhooks as well. As last step in our bitbucket pipeline I trigger a webhook provided from  swarm management tool portainer which is deployed in our swarm cluster with a simple ‚curl -C WEBHOOKURL‘. Portainer fetch the images from a configured registry and make a rolling restart of the images within swarm. Works very smooth. Cheers, phillip

0 votes
Sandeep Srinivasa January 19, 2018

@wiesson that is superb! Thanks for taking the effort. Your recipe is something on the lines that we want to follow. 

However SSH is not an option - we want this to run on every commit/merge. 

Looking forward to see this working on Swarm!

Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
January 21, 2018

Hi @Sandeep Srinivasa,

Why is SSH not an option? Is it acceptable if SSH is run on every commit/merge as part of the pipeline script?

We have some documentation on setting up SSH commands inside of you pipeline here: https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html

Does that help at all?

Thanks,

Phil

Sandeep Srinivasa January 21, 2018

Thanks for replying.

For the same reason why kubectl tries not to use it..and every platform provides a SSL/TLS API.

Being able to physically log in to a server is a different security envelope with different considerations. For example, once you are on the server - you can attempt Spectre/Meltdown. You cannot do that with remotely issued docker commands over TLS.

In general, we would strongly request for a transport level security over being able to SSH.

IMHO both k8s and swarm have it - it's just a question of how to set it up with your infrastructure.

Trent Murray June 27, 2018

Did this ever get anywhere, @Sandeep Srinivasa - did you manage to find a solution?

Sandeep Srinivasa June 27, 2018

Nope.

If you do - please do post here.

peksipatongeis October 25, 2018

Any updates on this? Would be really nice to have a proper way of doing Docker deployments to remote hosts. 

Like Olivier Deprez likes this
tomasras November 14, 2018

+1

Like Jens Riisom Schultz likes this
peksipatongeis November 15, 2018

I was hoping I could execute the `docker stack deploy` remotely from Pipelines but I guess it's not going to happen due to how the remote host connection is set up with Docker. 

Ended up setting up a pipeline where docker-compose.yml is copied to remote host and running commands on the remote host as well. 

tomasras November 15, 2018
0 votes
Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
January 3, 2018

Hi Sandeep,

I'm not fully sure what Geoff had in mind as a workaround here. He's on leave until next week so I can't ask him until then. As far as I know this isn't possible as Swarm commands aren't available at the moment. But, I'll check with Geoff when he's back.

Thanks,

Phil

Sandeep Srinivasa January 7, 2018

> Swarm commands aren't available at the moment.

well I guess that was the point for this ! Swarm is the second biggest deployment infra after kubernetes and morever is bundled inside Docker itself.

We would appreciate Bitbucket Pipelines team to tell us how to do this effectively. Swarm already comes with TLS security - https://docs.docker.com/engine/security/https/ 

but unfair to only support kubernetes  :(

Like # people like this
Philip Hodder
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
January 9, 2018

Hi Sandeep,

You can follow this guide to deploy to your Swarm cluster: https://docs.docker.com/swarm/swarm_at_scale/deploy-app/

You will also need to update the DOCKER_CERT_PATH to point at the API keys of your Swarm cluster, mentioned here: https://docs.docker.com/engine/reference/commandline/cli/#description

I don't have an ETA on when we may have a similar guide on Swarm in our own documentation. However, those links should suffice for you by translating the appropriate commands into commands in your deploy step script.

Thanks,

Phil

Sandeep Srinivasa January 9, 2018

couldnt figure out how to do it with pipelines. maybe im making a stupid mistake.

I pray that you guys do get around to it /hopeful

wiesson January 18, 2018
stage: deploy
variables:
DOCKER_HOST: tcp://XXXX:2376
DOCKER_TLS_VERIFY: 1
SERVICE_NAME: barfbarf
image: docker:latest
script:
- mkdir -p ~/.docker
- echo "$TLSCACERT" > ~/.docker/ca.pem
- echo "$TLSCERT" > ~/.docker/cert.pem
- echo "$TLSKEY" > ~/.docker/key.pem
- docker login -u gitlab-ci-token -p "$CI_BUILD_TOKEN" "$CI_REGISTRY"
- docker service update --detach=false --with-registry-auth --image registry.gitlab.com/barfbarf/$CI_PROJECT_NAME:$CI_BUILD_REF barfbarf
environment:
name: production
url: https://barfbarf.com
only:
- master

Basically it works like this code snipped that I posted above (gitlab pipeline). I'm currently trying to "translate" it to bitbucket.

wiesson January 19, 2018

What you can do - SSH to one of your managers and run those commands

 

docker service update ...
Like Branislav Ballay likes this

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events