Problems to deploy kubernetes configuration using bitbucket pipelines

[Natan]

Hello Community,

I'm having a problem with bitbucket pipelines, I need to build and deploy a docker image to my kubernetes cluster at DigitalOcean, but I can't get success with that because the pipe atlassian/kubectl-run:1.1.2 is not working properly. I read the documentation of this pipe and followed the exemples described, but I can't get success with that because the pipe throws an error in line 100 of pipe.py. Considering that I do not use python in my application, and the error showing in the logs don't tell me what is wrong I can't get the deploy working and I don't know why... I need some help with this.

My pipeline config (edited of course):

image: atlassian/default-image:2

pipelines:
default:
- step:
name: Build fresh image and push to Amazon ECR
script:
# build the image
- docker build -t my-app --build-arg APP_ENV=$APP_ENV .

# use the pipe to push the image to AWS ECR
- pipe: atlassian/aws-ecr-push-image:1.1.0
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
IMAGE_NAME: my-app
- step:
name: Deployment in mokyun-kubernetes-nyc3-digitalocean
deployment: production
script:
- pipe: atlassian/kubectl-run:1.1.2
variables:
KUBE_CONFIG: $MOKYUN_KUBERNETES
KUBECTL_COMMAND: 'apply'
RESOURCE_PATH: '.kube-deployment/'

(I think the yaml will break because of code block, in original file, it's a valid yaml)

In the config, I have two steps, one for build and test the docker image and push to Amazon ECR, and another for deploying in DigitalOcean's Kubernetes Cluster. The docker image builds and push normally as expected, but the step to deploy on kubernetes do not work. The error occurs after pipe clone the atlassian/kubectl-run:1.1.2 and try to run pipe.py.

Here the log of the error:

Status: Downloaded newer image for bitbucketpipelines/kubectl-run:1.1.2
Traceback (most recent call last):
  File "/pipe.py", line 121, in <module>
    pipe.run()
  File "/pipe.py", line 107, in run
    self.configure()
  File "/pipe.py", line 100, in configure
    with open('config', 'w+') as kube_config:
IsADirectoryError: [Errno 21] Is a directory: 'config'
Searching for files matching artifact pattern .bitbucket/pipelines/generated/pipeline/pipes/**

Searching for test report files in directories named [test-results, failsafe-reports, test-reports, surefire-reports] down to a depth of 4
Finished scanning for test reports. Found 0 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.

I read the pipe source at bitbucket, but I can't tell what is wrong here, I need some help with this!

[Oleksandr Kyrdan]

Hi @Natan @c_lingle @Patrick Hynes @judezhu 

Apologies for the late reply and thanks a lot for your feedback.
For some reason, this question was not tagged with the tag "pipes", so we missed it.

We fixed this issue and released a new version of the kubectl-run pipe 

script:
  - pipe: atlassian/kubectl-run:1.3.3
    variables:
      KUBE_CONFIG: $KUBE_CONFIG
      KUBECTL_COMMAND: 'apply'
      RESOURCE_PATH: 'nginx.yml'

It would be nice if you provide us with more details of what improvement is expected for the kubectl-run pipe to make it better.

We're really appreciate and happy for any your contribution PR to the  kubectl-run pipe.

Thank you for your feedback!

Best regards,
Oleksandr Kyrdan

[Lucas Nunes da Silva]

This solved my problem, but I recommend you ask your team to update the Deploy documentation on Kubernetes (https://support.atlassian.com/bitbucket-cloud/docs/deploy-to-kubernetes/), as this can generate a certain headache..

[c_lingle]

I just ran in to the same issue. I'm attempting to deploy a Laravel project which has a config folder present in the root of the repository. Since the pipe attempts to create the kube config file in the same directory as the repo is cloned into it can't write to config as a file since it's already a directory.

[Natan]

Yea, I also trying to deploy a laravel app, but this config problem at line 100 is really annoying.. I'm considering to move to gitlab CI/CD

[c_lingle]

I realized there's a simple workaround to this that shouldn't cause any issues as long as you don't attempt to do anything with the repo that might need the config folder to be intact while in the same step definition as the kubectl pipe.

- step:
  name: Deployment in mokyun-kubernetes-nyc3-digitalocean
  deployment: production
  script:
    # Remove anything at the path $(pwd)/config to avoid naming
    # conflicts with the pipe. Ideally the pipe would allow the kube
    # config path to be configured to circumvent this.
    # Removing the folder isn't an issue since it's applying the
    # pushed image that was created in the previous step.
    - rm -rf config
    # Execute the pipe as normal.
    - pipe: atlassian/kubectl-run:1.1.2
      variables:
        KUBE_CONFIG: $MOKYUN_KUBERNETES
        KUBECTL_COMMAND: 'apply'
        RESOURCE_PATH: '.kube-deployment/'

 An alternative if you still need the config folder later in the same step:

- step:
  name: Deployment in mokyun-kubernetes-nyc3-digitalocean
  deployment: production
  script:
    # Rename the config folder to prevent naming collisions.
    - mv config config.bak
    # Execute the pipe as normal.
    - pipe: atlassian/kubectl-run:1.1.2
      variables:
        KUBE_CONFIG: $MOKYUN_KUBERNETES
        KUBECTL_COMMAND: 'apply'
        RESOURCE_PATH: '.kube-deployment/'
    # Move the config folder to its original location.
    - rm config && mv config.bak config
    # Do whatever else.
    - ls config

[Natan]

@c_lingle I tested and It works well, thanks for your help... But I see this still an issue with this pipe and I will try to contact bitbucket pipelines team to report that

[Patrick Hynes]

@Natan 

I am encountering the same issue. Has an issue been raised with the bitbucket pipelines team?

[Natan]

No, unfortunately I changed to Gitlab, I faced multiple issues with kubernetes integration in bitbucket and the support sucks so much

[judezhu]

This image atlassian/kubectl-run:1.1.2 sucks. This how I made it work:

You encode your kubeconfig as a base64 string and decode it during deployment.

- step:
    name: Deploy
    deployment: staging
    image: atlassian/pipelines-kubectl
    script:
      - export BITBUCKET_COMMIT_SHORT="${BITBUCKET_COMMIT::7}"
      - echo $BITBUCKET_COMMIT_SHORT
      - echo $KUBE_CONFIG_BASE64 | base64 -d > kubeconfig.yml
      - cat kubeconfig.yml
      - kubectl --kubeconfig=kubeconfig.yml apply -f k8s/svc.yaml
      - kubectl --kubeconfig=kubeconfig.yml apply -f k8s/deploy.yam