How to build an arm64 Docker image from a pipeline?

[Nick Van Osta]

For one of our projects we are working with Toradex' Torizon OS, an operating system based on embedded Linux, which primarily uses containers to run applications. We are using a Torizon device with an arm64 architecture. Toradex' Torizon project templates provide sample CI/CD configurations for GitLab and GitHub Actions, but not for Bitbucket.

We've been trying to translate one of the provided pipeline configurations to Bitbucket, however we've stumbled upon a roadblock. The build step within the pipeline spins up and runs the torizon/binfmt container in order to set up the host environment for running arm64 executables through QEMU. Herein lies the problem, this container needs access to the host system and is thus run with the --privileged flag, which is not allowed within Bitbucket Pipelines. We've also come across a ticket for supporting the building of multi-architecture Docker images, which could possibly solve our problem: https://jira.atlassian.com/browse/BCLOUD-15317

For certain reasons we'd prefer to not host the pipeline runners ourselves, but keep them cloud-hosted.

Is there any way to circumvent the need of the privileged flag here and build the container image for the arm64 architecture from a cloud-hosted pipeline?

Relevant links:

• Configure Build Environment for Torizon Containers | Toradex Developer Center
• torizon/binfmt - Docker Image | Docker Hub
• vscode-torizon-templates/assets/gitlab/.gitlab-ci.yml at dev · toradex/vscode-torizon-templates · GitHub

Relevant files:

docker-compose.yml

services:
  test-debug:
    build:
      context: .
      dockerfile: Dockerfile.debug
    image: ${LOCAL_REGISTRY}:5002/test-debug:${TAG}
    ports:
      - 2222:2222
    volumes:
      - /dev:/dev
    device_cgroup_rules:
      - "c 254:* rmw"

  test:
    build:
      context: .
      dockerfile: Dockerfile
    image: ${DOCKER_LOGIN}/test:${TAG}
    volumes:
      - /dev:/dev
    device_cgroup_rules:
      - "c 254:* rmw"

Dockerfile

# ARGUMENTS --------------------------------------------------------------------
##
# Board architecture
##
ARG IMAGE_ARCH=

##
# Base container version
##
ARG BASE_VERSION=3.0.0-8.0

##
# Directory of the application inside container
##
ARG APP_ROOT=
# ARGUMENTS --------------------------------------------------------------------

 

# BUILD ------------------------------------------------------------------------
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS Build

ARG IMAGE_ARCH
ARG APP_ROOT

COPY . ${APP_ROOT}
WORKDIR ${APP_ROOT}

# build
RUN dotnet restore test.sln && \
dotnet publish -c Release -r linux-${IMAGE_ARCH} --no-self-contained test.sln
# BUILD ------------------------------------------------------------------------

 

# DEPLOY -----------------------------------------------------------------------
FROM --platform=linux/${IMAGE_ARCH} \
torizon/dotnet:${BASE_VERSION} AS Deploy

ARG IMAGE_ARCH
ARG APP_ROOT

# !!! This line crashes within the pipeline
RUN apt-get -y update && apt-get install -y --no-install-recommends \
# ADD YOUR PACKAGES HERE
# DO NOT REMOVE THIS LABEL: this is used for VS Code automation
# __torizon_packages_prod_start__
# __torizon_packages_prod_end__
# DO NOT REMOVE THIS LABEL: this is used for VS Code automation
&& apt-get clean && apt-get autoremove && rm -rf /var/lib/apt/lists/*

# Copy the application compiled in the build step to the $APP_ROOT directory
# path inside the container, where $APP_ROOT is the torizon_app_root
# configuration defined in settings.json
COPY --from=Build ${APP_ROOT}/bin/Release/net8.0/linux-${IMAGE_ARCH}/publish ${APP_ROOT}

# "cd" (enter) into the APP_ROOT directory
WORKDIR ${APP_ROOT}

# Command executed in runtime when the container starts
CMD ["./test"]

# DEPLOY -----------------------------------------------------------------------

bitbucket-pipelines.yml

DOCKER_REGISTRY is a workspace variable.

definitions:
  stages:
    - stage: &build
        name: Build
        steps:
          - step:
              name: Build Docker Image
              script:
                - export DOCKER_BUILDKIT=0
                # Initialize Environment Variables
                - export TORIZON_ARCH=arm64
                - export APP_ROOT=/home/torizon/app
                - export TASKS_ITERATIVE="False"
                - export TASKS_OVERRIDE_ENV="False"
                # Build the production image
                - |
                  DOCKER_LOGIN=$DOCKER_REGISTRY \
                  TAG=latest \
                  docker compose build \
                  --build-arg APP_ROOT=/home/torizon/app \
                  --build-arg IMAGE_ARCH=$TORIZON_ARCH \
                  test
              services:
                - docker
        trigger: automatic
pipelines:
  default:
    - stage: *build

An error occurs in the pipeline upon reaching the "RUN apt-get -y update" step in the Dockerfile:

exec /bin/sh: exec format error

[Patrik S]

Hello @Nick Van Osta and welcome to the Community!

Unfortunately, the build of arm64 images is still not possible when using cloud-hosted runners (hosted by Atlassian). The implementation of this functionality is being tracked in the feature request BCLOUD-15317. 

We encourage you to add your vote in that feature to demonstrate your interest, and also add yourself as a watcher in case you want to be notified of any future updates.

While that feature is not available, it's only possible to run multi-architecture images when using self-hosted runners, which allows the usage of the --privileged flag and QEMU. If you decide to go with that route, we have the following article with instructions on how that can be configured in your pipeline: 

• Building multi-architecture docker images with Bitbucket Pipelines 

Thank you, @Nick Van Osta !

Patrik S