Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
Community Members
Community Events
Community Groups

How does Pipelines start the build image?

Deleted user Apr 12, 2018

We're trying to using a custom build image for our Bitbucket pipeline, but there is some information about how the agent is started that is obscured. I'm curious about what commands are executed in order to start the agent, and then what commands are executed during the "Build setup" phase.

This would be helpful in making sure our build agent does what we want. For instance, we've setup our build agent to end in a certain WORKDIR, but that seems to be irrelevant to Pipelines since it looks like it sets its own WORKDIR. Even if we try to set things up in the '/opt/atlassian/pipelines/agent/build' directory which it seems to force, everything in there seems to get wiped by the time we get to our build steps.

I'm just looking for some total transparency in what's happening. We can at least see the commands that run during "Build setup" but simply preparing for those commands doesn't help. I assume it starts the build agent in slightly different ways (for instance docker:true gives the image access to dockerd and docker commands without installing docker on the image itself) based on the pipeline script.

It would probably be useful to know what happens during the "Build teardown" phase as well but that's less critical.

1 answer

0 votes
SebC Atlassian Team Apr 25, 2018

Hey @[deleted], happy to provide details where I can.

Pipelines are a logical concept that's made up of Steps, whenever you see a "Build Setup/Teardown" lifecycle, that's the Step lifecycle.

We start a Step by scheduling a pod in Kubernetes. The pod consists of at least 3 containers: Our agent that manages the lifecycle of the step, a build container where your script commands run, and a clone container that's responsible for cloning your repo from bitbucket.

To make sure there's shared state between the containers we create volumes to map the clone directory, as well as directories for logs, step state, artifacts and other features.

One all the containers have started, our agent executes the clone via the clone container (with data written to a shared volume). Once successful, the agent executes via the build container any set up tasks (download caches, configure ssh keys, etc. etc.), then executes your script tasks in order, then finally tears down the container and posts information back.

I'm a little worried that you're baking too much magic into your build container - if you're having issues with this simple set up tasks, I fear you're only going to have more frustration as features are added and expanded.

Build containers should create a stateless environment for the build to be run in - if you can't just run:

docker run --rm -t --entrypoint="/bin/bash -c" yourimage:latest "command"

I think you're going to have a hard time.




Suggest an answer

Log in or Sign up to answer
Community showcase
Published in Bitbucket

Git push size limits are coming to Bitbucket Cloud starting April 4th, 2022

Beginning on April 4th, we will be implementing push limits. This means that your push cannot be completed if it is over 3.5 GB. If you do attempt to complete a push that is over 3.5 GB, it will fail...

1,433 views 1 7
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you