Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Problems to deploy kubernetes configuration using bitbucket pipelines

Edited

Hello Community,

I'm having a problem with bitbucket pipelines, I need to build and deploy a docker image to my kubernetes cluster at DigitalOcean, but I can't get success with that because the pipe atlassian/kubectl-run:1.1.2 is not working properly. I read the documentation of this pipe and followed the exemples described, but I can't get success with that because the pipe throws an error in line 100 of pipe.py. Considering that I do not use python in my application, and the error showing in the logs don't tell me what is wrong I can't get the deploy working and I don't know why... I need some help with this.

 

My pipeline config (edited of course):

image: atlassian/default-image:2

pipelines:
default:
- step:
name: Build fresh image and push to Amazon ECR
script:
# build the image
- docker build -t my-app --build-arg APP_ENV=$APP_ENV .

# use the pipe to push the image to AWS ECR
- pipe: atlassian/aws-ecr-push-image:1.1.0
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
IMAGE_NAME: my-app
- step:
name: Deployment in mokyun-kubernetes-nyc3-digitalocean
deployment: production
script:
- pipe: atlassian/kubectl-run:1.1.2
variables:
KUBE_CONFIG: $MOKYUN_KUBERNETES
KUBECTL_COMMAND: 'apply'
RESOURCE_PATH: '.kube-deployment/'

(I think the yaml will break because of code block, in original file, it's a valid yaml)

 

In the config, I have two steps, one for build and test the docker image and push to Amazon ECR, and another for deploying in DigitalOcean's Kubernetes Cluster. The docker image builds and push normally as expected, but the step to deploy on kubernetes do not work. The error occurs after pipe clone the atlassian/kubectl-run:1.1.2 and try to run pipe.py.

Here the log of the error:

Status: Downloaded newer image for bitbucketpipelines/kubectl-run:1.1.2
Traceback (most recent call last):
  File "/pipe.py", line 121, in <module>
    pipe.run()
  File "/pipe.py", line 107, in run
    self.configure()
  File "/pipe.py", line 100, in configure
    with open('config', 'w+') as kube_config:
IsADirectoryError: [Errno 21] Is a directory: 'config'
Searching for files matching artifact pattern .bitbucket/pipelines/generated/pipeline/pipes/**

Searching for test report files in directories named [test-results, failsafe-reports, test-reports, surefire-reports] down to a depth of 4
Finished scanning for test reports. Found 0 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.

 

I read the pipe source at bitbucket, but I can't tell what is wrong here, I need some help with this!

2 answers

0 votes

Hi @Natan @c_lingle @Patrick Hynes @judezhu 

Apologies for the late reply and thanks a lot for your feedback.
For some reason, this question was not tagged with the tag "pipes", so we missed it.

We fixed this issue and released a new version of the kubectl-run pipe 

script:
- pipe: atlassian/kubectl-run:1.3.3
variables:
KUBE_CONFIG: $KUBE_CONFIG
KUBECTL_COMMAND: 'apply'
RESOURCE_PATH: 'nginx.yml'

It would be nice if you provide us with more details of what improvement is expected for the kubectl-run pipe to make it better.

We're really appreciate and happy for any your contribution PR to the  kubectl-run pipe.


Thank you for your feedback!

Best regards,
Oleksandr Kyrdan

I just ran in to the same issue. I'm attempting to deploy a Laravel project which has a config folder present in the root of the repository. Since the pipe attempts to create the kube config file in the same directory as the repo is cloned into it can't write to config as a file since it's already a directory.

Yea, I also trying to deploy a laravel app, but this config problem at line 100 is really annoying.. I'm considering to move to gitlab CI/CD

Like # people like this

I realized there's a simple workaround to this that shouldn't cause any issues as long as you don't attempt to do anything with the repo that might need the config folder to be intact while in the same step definition as the kubectl pipe.

- step:
name: Deployment in mokyun-kubernetes-nyc3-digitalocean
deployment: production
script:
# Remove anything at the path $(pwd)/config to avoid naming
# conflicts with the pipe. Ideally the pipe would allow the kube
# config path to be configured to circumvent this.
# Removing the folder isn't an issue since it's applying the
# pushed image that was created in the previous step.
- rm -rf config
# Execute the pipe as normal.
- pipe: atlassian/kubectl-run:1.1.2
variables:
KUBE_CONFIG: $MOKYUN_KUBERNETES
KUBECTL_COMMAND: 'apply'
RESOURCE_PATH: '.kube-deployment/'

 An alternative if you still need the config folder later in the same step:

- step:
name: Deployment in mokyun-kubernetes-nyc3-digitalocean
deployment: production
script:
# Rename the config folder to prevent naming collisions.
- mv config config.bak
# Execute the pipe as normal.
- pipe: atlassian/kubectl-run:1.1.2
variables:
KUBE_CONFIG: $MOKYUN_KUBERNETES
KUBECTL_COMMAND: 'apply'
RESOURCE_PATH: '.kube-deployment/'
# Move the config folder to its original location.
- rm config && mv config.bak config
# Do whatever else.
- ls config
Like # people like this

@c_lingle I tested and It works well, thanks for your help... But I see this still an issue with this pipe and I will try to contact bitbucket pipelines team to report that

@Natan 

I am encountering the same issue. Has an issue been raised with the bitbucket pipelines team?


No, unfortunately I changed to Gitlab, I faced multiple issues with kubernetes integration in bitbucket and the support sucks so much

Like Guilherme Pacheco likes this

This image atlassian/kubectl-run:1.1.2 sucks. This how I made it work:

You encode your kubeconfig as a base64 string and decode it during deployment.

- step:
name: Deploy
deployment: staging
image: atlassian/pipelines-kubectl
script:
- export BITBUCKET_COMMIT_SHORT="${BITBUCKET_COMMIT::7}"
- echo $BITBUCKET_COMMIT_SHORT
- echo $KUBE_CONFIG_BASE64 | base64 -d > kubeconfig.yml
- cat kubeconfig.yml
- kubectl --kubeconfig=kubeconfig.yml apply -f k8s/svc.yaml
- kubectl --kubeconfig=kubeconfig.yml apply -f k8s/deploy.yam  

 

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket

New improvements to user management in Bitbucket Cloud 👥

Hey Community! We’re willing to wager that quite a few of you not only use Bitbucket, but administer it too. Our team is excited to share that we’ll be releasing improvements throughout this month of...

3,887 views 10 16
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you