Issue with Bit bucket pre available pipes

[Midhun Kumar]

Hello  Community Team, 

We have encountered an Issue with pipe named  atlassian/aws-eks-kubectl-run:2.2.0 for deploying into cluster from the repo  https://bitbucket.org/ninjafactory/cardhost-staging/src/master/   with workspace url https://bitbucket.org/ninjafactory.

We are encountering file not found though file is there in the path to apply hibernates deployment.

Here is the build number for reference.

https://bitbucket.org/ninjafactory/cardhost-staging/addon/pipelines/home#!/results/20

[Pramodh M]

Hi @Midhun Kumar 

Welcome to the Community!!

Did you try giving the absolute path and checking for the RESOURCE_PATH variable?

https://bitbucket.org/atlassian/aws-eks-kubectl-run/src/master

[Midhun Kumar]

Hi Pramodh,

The file which we are applying has present in project root folder. If you see the build output for the failed stage it has the file over there. 

Best Regards,

Midhun A

[Midhun Kumar]

@Pramodh M Any update 

You can check repo if required.

[Pramodh M]

@Midhun Kumar 

Can you send over the raw logs in here?

[Midhun Kumar]

 umask 000

+ GIT_LFS_SKIP_SMUDGE=1 retry 6 git clone --branch="master" --depth 50 https://x-token-auth:$REPOSITORY_OAUTH_ACCESS_TOKEN@bitbucket.org/$BITBUCKET_REPO_FULL_NAME.git $BUILD_DIR
Cloning into '/opt/atlassian/pipelines/agent/build'...

+ git reset --hard 2bad71eeecfe0817445b637dca3717f5060667ec
HEAD is now at 2bad71e bitbucket-pipelines.yml edited online with Bitbucket

+ git config user.name bitbucket-pipelines

+ git config user.email commits-noreply@bitbucket.org

+ git config push.default current

+ git config http.${BITBUCKET_GIT_HTTP_ORIGIN}.proxy http://localhost:29418/

+ git remote set-url origin http://bitbucket.org/$BITBUCKET_REPO_FULL_NAME

+ git reflog expire --expire=all --all

+ echo ".bitbucket/pipelines/generated" >> .git/info/exclude

+ chmod 777 $BUILD_DIR

Artifact "target/sonar/report-task.txt": Downloading
Artifact "target/sonar/report-task.txt": Downloaded 318 B in 0 seconds
Artifact "target/sonar/report-task.txt": Extracting
Artifact "target/sonar/report-task.txt": Extracted in 0 seconds

Images used:
build : docker.io/atlassian/pipelines-awscli@sha256:3b632b0e1c0fc4ec4ab23ea7f0decf1fc97580a30782074ce6cbc4e816075901
+ export IMAGE_NAME_COMMIT=$BITBUCKET_BRANCH-$BITBUCKET_COMMIT

+ export DEPLOY_VERSION=$BITBUCKET_BRANCH-$BITBUCKET_COMMIT

+ sed -i 's/\$IMGTAG'"/$DEPLOY_VERSION/g" kube-uat-mum-eks-deployment.yaml

+ sed -i -e 's:$APIVERSION'":$KUBEAPIVERSION:g" kube-uat-mum-eks-deployment.yaml

+ ls -ltr & pwd
/opt/atlassian/pipelines/agent/build

total 112
-rw-rw-rw- 1 root root 4231 Jan 12 04:59 uat-deployment.yaml
drwxrwxrwx 4 root root 4096 Jan 12 04:59 src
-rw-rw-rw- 1 root root 3824 Jan 12 04:59 prod-ext-deployment.yaml
-rw-rw-rw- 1 root root 3807 Jan 12 04:59 prod-deployment.yaml
-rw-rw-rw- 1 root root 4605 Jan 12 04:59 prod-aw-cht-ext-deployment.yaml
-rw-rw-rw- 1 root root 5341 Jan 12 04:59 prod-aw-cht-deployment.yaml
-rw-rw-rw- 1 root root 6102 Jan 12 04:59 pom.xml
-rw-rw-rw- 1 root root 5811 Jan 12 04:59 mvnw.cmd
-rwxrwxrwx 1 root root 9114 Jan 12 04:59 mvnw
-rw-rw-rw- 1 root root 2416 Jan 12 04:59 docker-compose.yml
-rw-rw-rw- 1 root root 3209 Jan 12 04:59 docker-compose-graylog3.yml
-rw-rw-rw- 1 root root 2886 Jan 12 04:59 dev-deployment.yaml
-rw-rw-rw- 1 root root 8479 Jan 12 04:59 bitbucket-pipelines.yml
-rw-rw-rw- 1 root root 557 Jan 12 04:59 README.md
-rw-rw-rw- 1 root root 905 Jan 12 04:59 HELP.md
-rw-rw-rw- 1 root root 430 Jan 12 04:59 Dockerfile
drwxr-xr-x 3 root root 4096 Jan 12 04:59 target
-rw-rw-rw- 1 root root 4285 Jan 12 04:59 kube-uat-mum-eks-deployment.yaml
+ docker container run \
--volume=/opt/atlassian/pipelines/agent/build:/opt/atlassian/pipelines/agent/build \
--volume=/opt/atlassian/pipelines/agent/ssh:/opt/atlassian/pipelines/agent/ssh:ro \
--volume=/usr/local/bin/docker:/usr/local/bin/docker:ro \
--volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes \
--volume=/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/aws-eks-kubectl-run:/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/aws-eks-kubectl-run \
--workdir=$(pwd) \
--label=org.bitbucket.pipelines.system=true \
--env=BITBUCKET_STEP_TRIGGERER_UUID="$BITBUCKET_STEP_TRIGGERER_UUID" \
--env=BITBUCKET_REPO_FULL_NAME="$BITBUCKET_REPO_FULL_NAME" \
--env=BITBUCKET_GIT_HTTP_ORIGIN="$BITBUCKET_GIT_HTTP_ORIGIN" \
--env=BITBUCKET_PROJECT_UUID="$BITBUCKET_PROJECT_UUID" \
--env=BITBUCKET_REPO_IS_PRIVATE="$BITBUCKET_REPO_IS_PRIVATE" \
--env=BITBUCKET_WORKSPACE="$BITBUCKET_WORKSPACE" \
--env=BITBUCKET_REPO_OWNER_UUID="$BITBUCKET_REPO_OWNER_UUID" \
--env=BITBUCKET_BRANCH="$BITBUCKET_BRANCH" \
--env=BITBUCKET_REPO_UUID="$BITBUCKET_REPO_UUID" \
--env=BITBUCKET_PROJECT_KEY="$BITBUCKET_PROJECT_KEY" \
--env=BITBUCKET_REPO_SLUG="$BITBUCKET_REPO_SLUG" \
--env=CI="$CI" \
--env=BITBUCKET_REPO_OWNER="$BITBUCKET_REPO_OWNER" \
--env=BITBUCKET_STEP_RUN_NUMBER="$BITBUCKET_STEP_RUN_NUMBER" \
--env=BITBUCKET_BUILD_NUMBER="$BITBUCKET_BUILD_NUMBER" \
--env=BITBUCKET_GIT_SSH_ORIGIN="$BITBUCKET_GIT_SSH_ORIGIN" \
--env=BITBUCKET_PIPELINE_UUID="$BITBUCKET_PIPELINE_UUID" \
--env=BITBUCKET_COMMIT="$BITBUCKET_COMMIT" \
--env=BITBUCKET_CLONE_DIR="$BITBUCKET_CLONE_DIR" \
--env=PIPELINES_JWT_TOKEN="$PIPELINES_JWT_TOKEN" \
--env=BITBUCKET_STEP_UUID="$BITBUCKET_STEP_UUID" \
--env=BITBUCKET_DOCKER_HOST_INTERNAL="$BITBUCKET_DOCKER_HOST_INTERNAL" \
--env=DOCKER_HOST="tcp://host.docker.internal:2375" \
--env=BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes" \
--env=BITBUCKET_PIPE_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/atlassian/aws-eks-kubectl-run" \
--env=AWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID_NIYOX_UAT" \
--env=AWS_DEFAULT_REGION="$AWS_DEFAULT_REGION" \
--env=AWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY_NIYOX_UAT" \
--env=CLUSTER_NAME="AWS-MUM-NIYOX-UAT" \
--env=DEBUG="true" \
--env=KUBECTL_COMMAND="apply" \
--env=RESOURCE_PATH="kube-uat-mum-eks-deployment.yml" \
--add-host="host.docker.internal:$BITBUCKET_DOCKER_HOST_INTERNAL" \
bitbucketpipelines/aws-eks-kubectl-run:2.2.0
Unable to find image 'bitbucketpipelines/aws-eks-kubectl-run:2.2.0' locally
2.2.0: Pulling from bitbucketpipelines/aws-eks-kubectl-run
07aded7c29c6: Pulling fs layer
1242903d2b23: Pulling fs layer
6feb96d3e4f9: Pulling fs layer
36bf03acdc50: Pulling fs layer
366f5e2f7043: Pulling fs layer
2efba0da2be9: Pulling fs layer
3a7c21493639: Pulling fs layer
d92dcd533b57: Pulling fs layer
50cf76112433: Pulling fs layer
c5f0bcebca98: Pulling fs layer
6a3894d8edd4: Pulling fs layer
72b74681051b: Pulling fs layer
36bf03acdc50: Waiting
366f5e2f7043: Waiting
2efba0da2be9: Waiting
3a7c21493639: Waiting
d92dcd533b57: Waiting
50cf76112433: Waiting
c5f0bcebca98: Waiting
6a3894d8edd4: Waiting
72b74681051b: Waiting
1242903d2b23: Verifying Checksum
1242903d2b23: Download complete
6feb96d3e4f9: Verifying Checksum
6feb96d3e4f9: Download complete
07aded7c29c6: Verifying Checksum
07aded7c29c6: Download complete
36bf03acdc50: Verifying Checksum
36bf03acdc50: Download complete
366f5e2f7043: Verifying Checksum
366f5e2f7043: Download complete
2efba0da2be9: Verifying Checksum
2efba0da2be9: Download complete
50cf76112433: Verifying Checksum
50cf76112433: Download complete
c5f0bcebca98: Verifying Checksum
c5f0bcebca98: Download complete
d92dcd533b57: Verifying Checksum
d92dcd533b57: Download complete
6a3894d8edd4: Download complete
3a7c21493639: Verifying Checksum
3a7c21493639: Download complete
07aded7c29c6: Pull complete
72b74681051b: Download complete
1242903d2b23: Pull complete
6feb96d3e4f9: Pull complete
36bf03acdc50: Pull complete
366f5e2f7043: Pull complete
2efba0da2be9: Pull complete
3a7c21493639: Pull complete
d92dcd533b57: Pull complete
50cf76112433: Pull complete
c5f0bcebca98: Pull complete
6a3894d8edd4: Pull complete
72b74681051b: Pull complete
Digest: sha256:4a9e431a82bb96676ba3ae38c24e2e49b38d5ada4925fa973622ee4ee6ad7dfa
Status: Downloaded newer image for bitbucketpipelines/aws-eks-kubectl-run:2.2.0
[37mDEBUG: Starting new HTTPS connection (1): bitbucket.org:443 [0m
[37mDEBUG: https://bitbucket.org:443 "GET /bitbucketpipelines/official-pipes/raw/master/pipes.prod.json HTTP/1.1" 200 None [0m
[34mINFO: Configuring kubeconfig... [0m
Added new context arn:aws:eks:ap-south-1:914620821736:cluster/AWS-MUM-NIYOX-UAT to /root/.kube/config
Entries:

context:
cluster: arn:aws:eks:ap-south-1:914620821736:cluster/AWS-MUM-NIYOX-UAT
user: arn:aws:eks:ap-south-1:914620821736:cluster/AWS-MUM-NIYOX-UAT
name: arn:aws:eks:ap-south-1:914620821736:cluster/AWS-MUM-NIYOX-UAT

name: arn:aws:eks:ap-south-1:914620821736:cluster/AWS-MUM-NIYOX-UAT
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
args:
- --region
- ap-south-1
- eks
- get-token
- --cluster-name
- AWS-MUM-NIYOX-UAT
command: aws

cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EVXhNekExTkRjd09Wb1hEVE14TURVeE1UQTFORGN3T1Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBT0R1CjVuZWFURlFyUTE3b28wVDNKYkd5b3RqVHY0WjIzSlBKUUx6WUdXN1VRaEppWVd2aGlrSmxvTE93UUYvWkJGenUKa052THpUU0VsbFp2R3VOMnVvSUgxSG8zLzZCNG5mSnNrQVh0Vy8wdEZqNEJsL09XL2tMMFVtREQ0M21wMDFUVworcUtKSUNIcHpGU2NFSlRYaTNuRmx1NnQ2SFNqV045K1d2NjUrQkRwNXh6MmRIdlJ6eVhzVVhYTmdNNkdNZjdiCnpNV3NVVjZ1eXJwMUt2V0xFOHo3QVJFM0RKT2g5cVBRVGNXeWc0NVRRMVU3QjhJVGI1WituUnR0UzNVL1g3T3YKWDdkZXBmNVplYlRkWk4zbnRUWlkxZjFtU1U5MkFEdVVRMFR5UFhKNnhkazB4NlFzSjA3dzE4bDJ6YW4vc2tVbQowRXU1ZCsrYkhhSW9uVjkyb1NNQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZDR3pCeHJKbGlCTVZTcUlQQ2RDWFNoRm11S1hNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBakxXRFhuY0NEZ2dHbU9HUUtBTmF6a2huM0lwMWp3d2ZQSzBiRFl6WmJrbzVOVUUxYQp5cU1pTzFQZHBKd1VlQ1FIalBsc0hBUVluTGJJQjBTZys4N2NTTFA4bm1ZWTZMc3FmbHUzUFJsNkJXQStWM1N2CmpiOGgwcmw5L0lSejVIS3pST2wvYVBqR0kzaWlJWG4xTTZBaWtiTCtmUWhEeUtyRS9vM0NxWFprditGTXhVZUkKeEYzNmlUMHlncTJ0bStHWnBhb09scUdGTzNpZnpjNW92VHlaS3g4aXg2b3FyWVh2dUpkTXd2aXJhTkxjb2JmVwpzdzZIUXgzanRPRHppSzNMKzVRcHJtclhibVdNV1dHbTUzTFlLaXlHZklqNTUrV0JNL1VtRTZJYUlyaFBaUHk4CjBVdEl6anM1N0drYnRQRWUwV3ZPL1RzZlJQUnRIRkR6MTZVawotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
server: https://3CBD3548DC4B0A65EBF0E48057ADBA1F.gr7.ap-south-1.eks.amazonaws.com
name: arn:aws:eks:ap-south-1:914620821736:cluster/AWS-MUM-NIYOX-UAT

[34mINFO: Successfully updated the kube config. [0m
Traceback (most recent call last):
File "/pipe.py", line 47, in <module>
pipe.run()
File "/root/.local/lib/python3.8/site-packages/kubectl_run/pipe.py", line 157, in run
self.handle_apply()
File "/root/.local/lib/python3.8/site-packages/kubectl_run/pipe.py", line 114, in handle_apply
self.update_labels_in_metadata(template_file, labels)
File "/root/.local/lib/python3.8/site-packages/kubectl_run/pipe.py", line 29, in update_labels_in_metadata
with open(template, 'r') as template_file:
FileNotFoundError: [Errno 2] No such file or directory: 'kube-uat-mum-eks-deployment.yml'
Searching for files matching artifact pattern .bitbucket/pipelines/generated/pipeline/pipes/**

Searching for test report files in directories named [test-results, failsafe-reports, test-reports, TestResults, surefire-reports] down to a depth of 4
Finished scanning for test reports. Found 0 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.

[Midhun Kumar]

@Pramodh M 

I cannot able to upload logs with attachment over the chat. hasn't seen an option for attachment. You can get logs from build number which I have attached.

[Midhun Kumar]

https://bitbucket.org/ninjafactory/cardhost-staging/addon/pipelines/home#!/results/20

[Igor Stoyanov]

@Midhun Kumar  hi. Thanks for your question.

Your repository is private so we don't have access to it neiher to pipeline results.

To provide help we need configuration of your bitbucket-pipeline.yml file.

Currently from all info you provided in issue maybe the problem is with:

RESOURCE_PATH | Path to the kubernetes spec file. This option is required only if the KUBECTL_COMMAND is `apply`

Seems like you provided wrong value of argument RESOURCE_PATH and pipe log shows you error message:

FileNotFoundError: [Errno 2] No such file or directory: 'kube-uat-mum-eks-deployment.yml'

Example of how to use aws-eks-kubectl pipe: https://bitbucket.org/bitbucketpipelines/example-aws-eks-kubectl-run/src/master/

Keep in mind yml template should be in your repo.

Maybe the reason is the next:
From logs check you have here:

-rw-rw-rw- 1 root root 4285 Jan 12 04:59 kube-uat-mum-eks-deployment.yaml

but seems like in config you provided RESOURCE_PATH to yml

Regards, Igor.

[Midhun Kumar]

thanks for support.