Sometimes pipeline is successful and a few times it is failing. I assume the dynamic BITBUCKET_BRANCH name might be the issue. Traceback (most recent call last): File "/pipe.py", line 47, in pipe.run() File "/root/.local/lib/python3.8/site-packages/kubectl_run/pipe.py", line 157, in run self.handle_apply() File "/root/.local/lib/python3.8/site-packages/kubectl_run/pipe.py", line 114, in handle_apply self.update_labels_in_metadata(template_file, labels) File "/root/.local/lib/python3.8/site-packages/kubectl_run/pipe.py", line 30, in update_labels_in_metadata yamls = list(yaml.safe_load_all(template_file.read())) File "/root/.local/lib/python3.8/site-packages/yaml/__init__.py", line 130, in load_all yield loader.get_data() File "/root/.local/lib/python3.8/site-packages/yaml/constructor.py", line 45, in get_data return self.construct_document(self.get_node()) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 27, in get_node return self.compose_document() File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document node = self.compose_node(None, None) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node node = self.compose_mapping_node(anchor) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node item_value = self.compose_node(node, item_key) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node node = self.compose_mapping_node(anchor) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node item_value = self.compose_node(node, item_key) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node node = self.compose_mapping_node(anchor) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node item_value = self.compose_node(node, item_key) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node node = self.compose_mapping_node(anchor) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node item_value = self.compose_node(node, item_key) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node node = self.compose_sequence_node(anchor) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 111, in compose_sequence_node node.value.append(self.compose_node(node, index)) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node node = self.compose_mapping_node(anchor) File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 127, in compose_mapping_node while not self.check_event(MappingEndEvent): File "/root/.local/lib/python3.8/site-packages/yaml/parser.py", line 98, in check_event self.current_event = self.state() File "/root/.local/lib/python3.8/site-packages/yaml/parser.py", line 428, in parse_block_mapping_key if self.check_token(KeyToken): File "/root/.local/lib/python3.8/site-packages/yaml/scanner.py", line 116, in check_token self.fetch_more_tokens() File "/root/.local/lib/python3.8/site-packages/yaml/scanner.py", line 223, in fetch_more_tokens return self.fetch_value() File "/root/.local/lib/python3.8/site-packages/yaml/scanner.py", line 577, in fetch_value raise ScannerError(None, None, yaml.scanner.ScannerError: mapping values are not allowed here in " ", line 30, column 79: ... -1.amazonaws.com/aupris-frontend: INFO: Successfully updated the kube config. bitbucker-pipeline file image: atlassian/default-image:2 options: docker: true size: 2x pipelines: tags: release-*: - step: name: Build docker image services: - docker script: # build the image - docker build -t aupris-frontend . # use the pipe to push the image to AWS ECR - pipe: atlassian/aws-ecr-push-image:1.4.2 variables: AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION IMAGE_NAME: aupris-frontend TAGS: '${BITBUCKET_BRANCH}' - apt-get update && apt-get install gettext-base - envsubst aupris-frontend.yaml - sed -ie "s/THIS_STRING_IS_REPLACED_DURING_BUILD/$(date)/g" aupris-frontend.yaml - pipe: "atlassian/aws-eks-kubectl-run:2.2.0" variables: AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY AWS_DEFAULT_REGION: XXXXX CLUSTER_NAME: XXXXX KUBECTL_COMMAND: "apply" RESOURCE_PATH: "aupris-frontend.yaml" DEBUG: "true" definitions: services: docker: memory: 7128 yaml file apiVersion: v1 kind: Service metadata: name: aupris-frontend spec: # type: LoadBalancer selector: app: aupris-frontend ports: - name: http port: 80 targetPort: 80 --- apiVersion: apps/v1 kind: Deployment metadata: name: aupris-frontend spec: selector: matchLabels: app: aupris-frontend template: metadata: labels: app: aupris-frontend spec: terminationGracePeriodSeconds: 30 containers: - name: aupris-frontend image: XXXXXXXXX:${BITBUCKET_BRANCH} imagePullPolicy: "Always" env: - name: API_URL value: api/ - name: FOR_GODS_SAKE_PLEASE_REDEPLOY value: "THIS_STRING_IS_REPLACED_DURING_BUILD" resources: limits: memory: "128Mi" cpu: "100m" ports: - containerPort: 80

Hi @Srinivas Nekkalapudi , * It seems you have different python versions. Your original post shows python3.8 while the screenshot of the "pip install.." shows python2.7. You might need a different runner image for your pipeline. * 2+3 - Every step of the pipeline is running in different container, so by default, env parameters and files are not shared. You might need to define a cache or artifacts to pass files between steps. Once you can share files, you can also share env parameters using files, just dump and read to/from a shard file. This link might help: https://community.atlassian.com/t5/Bitbucket-questions/caches-vs-artifacts/qaq-p/864904

@Oday Rafeh HI, If you see my original post I have a single file with two documents separated by (---). so there are no changes to make I believe. To summarise, the following are the issues I have: * ImportError: No module named 'pkg_resources' * Could not access BITBUCKET_BRANCH_SAFE in steps 2 and 3. * Newly created files(e.g: sed -ie) could not be accessed in steps 2 and 3.

nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Pipeline error: yaml.scanner.ScannerError: mapping values are not allowed

Srinivas Nekkalapudi

Sometimes pipeline is successful and a few times it is failing.

I assume the dynamic BITBUCKET_BRANCH name might be the issue.



<span>Traceback (most recent call last):</span>

<span>  File "/pipe.py", line 47, in <module></span>

<span>    pipe.run()</span>

<span>  File "/root/.local/lib/python3.8/site-packages/kubectl_run/pipe.py", line 157, in run</span>

<span>    self.handle_apply()</span>

<span>  File "/root/.local/lib/python3.8/site-packages/kubectl_run/pipe.py", line 114, in handle_apply</span>

<span>    self.update_labels_in_metadata(template_file, labels)</span>

<span>  File "/root/.local/lib/python3.8/site-packages/kubectl_run/pipe.py", line 30, in update_labels_in_metadata</span>

<span>    yamls = list(yaml.safe_load_all(template_file.read()))</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/__init__.py", line 130, in load_all</span>

<span>    yield loader.get_data()</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/constructor.py", line 45, in get_data</span>

<span>    return self.construct_document(self.get_node())</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 27, in get_node</span>

<span>    return self.compose_document()</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document</span>

<span>    node = self.compose_node(None, None)</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node</span>

<span>    node = self.compose_mapping_node(anchor)</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node</span>

<span>    item_value = self.compose_node(node, item_key)</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node</span>

<span>    node = self.compose_mapping_node(anchor)</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node</span>

<span>    item_value = self.compose_node(node, item_key)</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node</span>

<span>    node = self.compose_mapping_node(anchor)</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node</span>

<span>    item_value = self.compose_node(node, item_key)</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node</span>

<span>    node = self.compose_mapping_node(anchor)</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node</span>

<span>    item_value = self.compose_node(node, item_key)</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node</span>

<span>    node = self.compose_sequence_node(anchor)</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 111, in compose_sequence_node</span>

<span>    node.value.append(self.compose_node(node, index))</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node</span>

<span>    node = self.compose_mapping_node(anchor)</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 127, in compose_mapping_node</span>

<span>    while not self.check_event(MappingEndEvent):</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/parser.py", line 98, in check_event</span>

<span>    self.current_event = self.state()</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/parser.py", line 428, in parse_block_mapping_key</span>

<span>    if self.check_token(KeyToken):</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/scanner.py", line 116, in check_token</span>

<span>    self.fetch_more_tokens()</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/scanner.py", line 223, in fetch_more_tokens</span>

<span>    return self.fetch_value()</span>

<span>  File "/root/.local/lib/python3.8/site-packages/yaml/scanner.py", line 577, in fetch_value</span>

<span>    raise ScannerError(None, None,</span>

<span>yaml.scanner.ScannerError: mapping values are not allowed here</span>

<span>  in "<unicode string>", line 30, column 79:</span>

<span>     ... -1.amazonaws.com/aupris-frontend:</span><span>INFO: Successfully updated the kube config.</span>

bitbucker-pipeline file

image: atlassian/default-image:2

options:     

  docker: true

  size: 2x

pipelines:

  tags:

    release-*:

      - step:

          name: Build docker image

          services:

            - docker

          script:

          # build the image

          - docker build -t aupris-frontend .

          # use the pipe to push the image to AWS ECR

          - pipe: atlassian/aws-ecr-push-image:1.4.2

            variables:

              AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID

              AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY

              AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION

              IMAGE_NAME: aupris-frontend

              TAGS: '${BITBUCKET_BRANCH}'

          - apt-get update && apt-get install gettext-base

          - envsubst < aupris-frontend.tpl.yaml > aupris-frontend.yaml

          - sed -ie "s/THIS_STRING_IS_REPLACED_DURING_BUILD/$(date)/g" aupris-frontend.yaml

          - pipe: "atlassian/aws-eks-kubectl-run:2.2.0"

            variables:

              AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID

              AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY

              AWS_DEFAULT_REGION: XXXXX

              CLUSTER_NAME: XXXXX

              KUBECTL_COMMAND: "apply"

              RESOURCE_PATH: "aupris-frontend.yaml"

              DEBUG: "true"

definitions:

  services:

    docker:

      memory: 7128

yaml file

apiVersion: v1

kind: Service

metadata:

  name: aupris-frontend

spec:

  # type: LoadBalancer

  selector:

    app: aupris-frontend

  ports:

    - name: http

      port: 80

      targetPort: 80

---

apiVersion: apps/v1

kind: Deployment

metadata:

  name: aupris-frontend

spec:

  selector:

    matchLabels:

      app: aupris-frontend

  template:

    metadata:

      labels:

        app: aupris-frontend

    spec:

      terminationGracePeriodSeconds: 30

      containers:

        - name: aupris-frontend

          image: XXXXXXXXX:${BITBUCKET_BRANCH}

          imagePullPolicy: "Always"

          env:

            - name: API_URL

              value: api/

            - name: FOR_GODS_SAKE_PLEASE_REDEPLOY

              value: "THIS_STRING_IS_REPLACED_DURING_BUILD"

          resources:

            limits:

              memory: "128Mi"

              cpu: "100m"

          ports:

            - containerPort: 80

Accepted answers

All comments

Oday Rafeh_2jhprw

@Srinivas Nekkalapudi

The error is due to a problem in your YAML file. To fix it:

Check for missing colons.
Ensure proper indentation.
Quote strings with special characters.
Fix unintended line breaks.
Review the YAML file, fix the issues, and retry running the pipeline.

IF this didn't help please provide the YAML file content so I can help more.

Srinivas Nekkalapudi

@Oday Rafeh Thanks for the quick response. I have edited my post and updated yaml file.

I wonder why my pipeline is successful few time with the same configuration and failing few other times.

I could not reply to your answer so editing my old comment.

I came across another error with your solution:



<span>Traceback (most recent call last):</span>

<span>  File "/usr/bin/yamllint", line 5, in <module></span>

<span>    from pkg_resources import load_entry_point</span>

<span>ImportError: No module named 'pkg_resources'</span><span>+ yamllint aupris-server.yaml</span>

---

@Oday Rafeh It still did not solve the issue. following is the screenshot:

Oday Rafeh_2jhprw

I noticed you have the image URL redacted with XXXXXXXXX. If the actual image URL in your YAML file contains any special characters, like colons, you need to wrap the value in quotes.

The updated section with quotes:

 containers:
- name: aupris-frontend
image: "XXXXXXXXX:${BITBUCKET_BRANCH}"
imagePullPolicy: "Always"

Oday Rafeh_2jhprw

To address the intermittent pipeline failures related to the dynamic BITBUCKET_BRANCH name, you can modify your bitbucket-pipelines. yml file by adding a step to log the branch name, validate the YAML file using yamllint, and replace special characters in the branch name with underscores.

Updated bitbucket-pipelines. yml:

image: atlassian/default-image:2

options:
docker: true
size: 2x

pipelines:
tags:
release-*:
- step:
name: Prepare and validate
script:
- echo "BITBUCKET_BRANCH: ${BITBUCKET_BRANCH}"
- apt-get update && apt-get install -y gettext-base yamllint
- export BITBUCKET_BRANCH_SAFE="$(echo ${BITBUCKET_BRANCH} | sed 's/[^a-zA-Z0-9]/_/g')"
- echo "BITBUCKET_BRANCH_SAFE: ${BITBUCKET_BRANCH_SAFE}"
- envsubst < aupris-frontend.tpl.yaml > aupris-frontend.yaml
- yamllint aupris-frontend.yaml
- step:
name: Build docker image
services:
- docker
script:
# build the image
- docker build -t aupris-frontend .
# use the pipe to push the image to AWS ECR
- pipe: atlassian/aws-ecr-push-image:1.4.2
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
IMAGE_NAME: aupris-frontend
TAGS: '${BITBUCKET_BRANCH_SAFE}'
- sed -ie "s/THIS_STRING_IS_REPLACED_DURING_BUILD/$(date)/g" aupris-frontend.yaml
- pipe: "atlassian/aws-eks-kubectl-run:2.2.0"
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: XXXXX
CLUSTER_NAME: XXXXX
KUBECTL_COMMAND: "apply"
RESOURCE_PATH: "aupris-frontend.yaml"
DEBUG: "true"

definitions:
services:
docker:
memory: 7128

I added new step called Prepare and validate". This step logs the branch name, installs yamllint, and creates a sanitized version of the branch name called BITBUCKET_BRANCH_SAFE. The branch name is then used in the subsequent steps for building the Docker image and deploying it.

Please update the aupris-frontend. tpl. yaml file to use ${BITBUCKET_BRANCH_SAFE} instead of ${BITBUCKET_BRANCH} for the image name:

# ... rest of the file
containers:
- name: aupris-frontend
image: "XXXXXXXXX:${BITBUCKET_BRANCH_SAFE}"
imagePullPolicy: "Always"
# ... rest of the file

Erez Maadani

Hi @Srinivas Nekkalapudi ,

It seems you have different python versions. Your original post shows python3.8 while the screenshot of the "pip install.." shows python2.7. You might need a different runner image for your pipeline.
2+3 - Every step of the pipeline is running in different container, so by default, env parameters and files are not shared. You might need to define a cache or artifacts to pass files between steps. Once you can share files, you can also share env parameters using files, just dump and read to/from a shard file. This link might help: https://community.atlassian.com/t5/Bitbucket-questions/caches-vs-artifacts/qaq-p/864904

Igor Stoyanov

@Srinivas Nekkalapudi hi.

You should not use dynamical variables inside your aupris-frontend yaml Kubernetes manifest file. And As @Oday Rafeh mentioned, you could create template instead, fill it with needed variables and create according manifest. Also, optionally, you could validate your generated manifest.

Regards, Igor

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

cloud

bitbucket

bitbucket-cloud

atl-gruland