How does Pipelines start the build image?

[Deleted user]

We're trying to using a custom build image for our Bitbucket pipeline, but there is some information about how the agent is started that is obscured. I'm curious about what commands are executed in order to start the agent, and then what commands are executed during the "Build setup" phase.

This would be helpful in making sure our build agent does what we want. For instance, we've setup our build agent to end in a certain WORKDIR, but that seems to be irrelevant to Pipelines since it looks like it sets its own WORKDIR. Even if we try to set things up in the '/opt/atlassian/pipelines/agent/build' directory which it seems to force, everything in there seems to get wiped by the time we get to our build steps.

I'm just looking for some total transparency in what's happening. We can at least see the commands that run during "Build setup" but simply preparing for those commands doesn't help. I assume it starts the build agent in slightly different ways (for instance docker:true gives the image access to dockerd and docker commands without installing docker on the image itself) based on the pipeline script.

It would probably be useful to know what happens during the "Build teardown" phase as well but that's less critical.

[SebC]

Hey @[deleted], happy to provide details where I can.

Pipelines are a logical concept that's made up of Steps, whenever you see a "Build Setup/Teardown" lifecycle, that's the Step lifecycle.

We start a Step by scheduling a pod in Kubernetes. The pod consists of at least 3 containers: Our agent that manages the lifecycle of the step, a build container where your script commands run, and a clone container that's responsible for cloning your repo from bitbucket.

To make sure there's shared state between the containers we create volumes to map the clone directory, as well as directories for logs, step state, artifacts and other features.

One all the containers have started, our agent executes the clone via the clone container (with data written to a shared volume). Once successful, the agent executes via the build container any set up tasks (download caches, configure ssh keys, etc. etc.), then executes your script tasks in order, then finally tears down the container and posts information back.

I'm a little worried that you're baking too much magic into your build container - if you're having issues with this simple set up tasks, I fear you're only going to have more frustration as features are added and expanded.

Build containers should create a stateless environment for the build to be run in - if you can't just run:

docker run --rm -t --entrypoint="/bin/bash -c" yourimage:latest "command"

I think you're going to have a hard time.

thanks,

Seb.