nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How can one add build steps to a docker image before the pipeline steps?

RLogik

As part of my build process, I need to pull a python image, then run a few commands (install dos2unix, etc. via apt-get, change a few values in files). Essentially I run the equivalent of a Dockerfile-build script.

Obviously, I can do these inside the pipeline, but the problem is, it would be an absolute nightmare to figure out what artefacts (considering I'm installing things) to declare.

Hence I need to run these as preliminary build steps, upon which the rest of the pipeline is to build on.

I this possible. If so how do I do this?

Accepted answers

ktomk

Hello @RLogik , welcome to the Atlassian Community.

If I understand your question the way that you find it cumbersome to maintain the preliminary build operations in a step scripts (e.g. many lines), one solution is to create a shell script and commit it to the repository and call it at the beginning of the step script.

Another solution which is less flexible is to create your own build image (as you wrote, that is like a Dockerfile). Keep in mind as this is less flexible this is more for the time when the build process has settled and it is clear what there is to encode. Then having a dedicated build image will show its benefits well.

You show little in your question about your concrete scenario, so I hope this already matches with your needs. I had problems to understand the meaning of artifacts, are those the artifacts in the Bitbucket Pipeline YAML file format?

All comments

Tuan Nguyen Anh_mjdqrs

Hi , I'm not sure about this in pipeline but you can use "artifacts" in a step and use it for the next step.

ktomk

Hello @RLogik , welcome to the Atlassian Community.

RLogik

That's exactly what I want to avoid.

RLogik

This is about the bitbucket-pipelines.yml script. For example:

image:
  name: <some docker image like python:xyz>
  # here I want to *additionally* run ONCE and for all build steps,
  # as I do in my own Dockerfile.
  # This should result in an image (let's call it main:base),
  # with which ALL the following steps are to work with
  # (instead of python:xyz).
  # These build steps are NOT simply "write a string to these few files".
  # They're apt-get install ... and curl instructions etc.
  # which install who knows what files and where.
  # So there is no way I can do this below as a step, and then copy artefacts.
pipelines:
  default:
    - step:
        name: Code build step
        script:
            - ....
    - step:
        name: Unit tests
        script:
            - ....
    - step:
        name: Compilation
        script:
            - ....

I know I can pack things in bash scripts. And I already do this. But that does not address the key issue. At the moment the only solution I have is to get the image, and then do everything in one single step (i. e. the docker build steps, code build, unit test, etc.), which works, but it is quite silly to have a pipeline with one step.

ktomk

This is a good example, makes it much more clear.

This looks like looking for a build image, most fitting an intermediate one.

I know for my local pipelines runner it works straight forward as one step can build a docker container and anther step can re-use it. But this is on a local machine.

On Bitbucket this needs some extra work like pushing it to a remote docker image repository (here Docker Hub), which requires some setup like credentials and authentication:

image:
   name: vendor/main:base
definitions:
   main-base: *build-image
     name: Build Main Base
     services:
       - docker
     caches:
       - docker
     image: python:alpine
     script:
       - TAG=vendor/main:base
       - |
         <<'DOCKERFILE' docker build --build-arg FROM=python:alpine -t "$TAG" -
         ARG FROM
         FROM $FROM
         # ... base image modifications for main:base
         DOCKERFILE
       - docker login -u "$DOCKER_HUB_USER" -p "$DOCKER_HUB_PASSWORD"
       - docker push "$TAG"
pipelines:
  default:
    - step: &build-image
    - step:
        name: Code build step
        script:
          - echo ....
     - step:
         name: Unit tests
         script:
           - echo ....
      - step:
          name: Compilation
          script:
            - echo ....

Technically this work on Bitbucket, tested it.

This will have side-effects on multiple pipelines running in parallel as the build image can change while the build is running.

It is perhaps better to have one pipeline to build the main:base image and push it to the registry so other pipeline steps can just use it without taking care about the building.

You can then re-build and push the main:base image in regular intervals to keep it updated by scheduling that pipeline.

ktomk

And technically, if all three steps are using the same image, there is not really a need to have them in multiple steps. normally the image is the boundary of the step, as only per step the image can be specified.

Maybe you can give some more context what is stupid in your eyes of having all in one step. Just asking.

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

bitbucket

bitbucket-pipelines