Why the LFS is not working in my pipeline?

[Matin Lotfaliei]

Hi. I have the bitbucket-pipeline.yml below but the LFS is not working.

image: atlassian/default-image:4

definitions:
  services:
    docker: # Define a custom docker daemon - can only be used with a self-hosted runner
      image: docker:dind
      memory: 2048

  set_env: &set_env
    >-
    export YOLO_MODEL_MODULE_NAME=yolo-model &&
    export YOLO_MODEL_MODULE_VERSION=${BITBUCKET_COMMIT::7} &&
    export DOCKER_CLI_EXPERIMENTAL=enabled &&

  download_buildx: &download_buildx
    >-
    wget https://github.com/docker/buildx/releases/download/v0.16.2/buildx-v0.16.2.linux-amd64 -O docker-buildx &&
    mkdir -p $HOME/.docker/cli-plugins/ &&
    mv docker-buildx $HOME/.docker/cli-plugins/ &&
    chmod +x ~/.docker/cli-plugins/docker-buildx

  steps:
    - step: &build_modules
        name: Build arm64 docker images
        runs-on:
          - self.hosted
          - linux
        script:
          - *download_buildx
          - *set_env
          - ./build-module.sh

    - step: &publish_modules
        name: Build and Publish Docker images
        runs-on:
          - self.hosted
          - linux
        script:
          - *download_buildx
          - *set_env
          - ./build-module.sh
          - docker login ${ACR_LOGIN_SERVER} --username ${ACR_NAME} --password ${ACR_ADMIN_PASSWORD}
          - docker push ${ACR_LOGIN_SERVER}/${YOLO_MODEL_MODULE_NAME}:${YOLO_MODEL_MODULE_VERSION}

    - step: &translate
        name: Run translation of model.zip if needed
        runs-on:
          - self.hosted
          - linux
        artifacts:
          - aux/drpai/model/**
        image: python:3.10
        clone:
          lfs: true
        caches:
          - pip
        script:
          - git clone https://github.com/MistySOM/aux.git
          - pip3 install -r aux/drpai/requirements.txt
          - ls -lah
          - unzip model.zip -d aux/drpai
          - cd aux/drpai && python3 ei2gst_drpai.py model

options:
  docker: true

pipelines:
  branches:
    master:
      - step: *translate
      - step: *publish_modules
    "**":
      - step: *translate
      - step: *build_modules
  pull-requests:
    "**":
      - step: *translate
      - step: *build_modules

I also place the output of ls -lah to show the LFS hasn't worked. The repo clearly has the LFS when I use git lfs clone

+ ls -lah
total 48K

drwxrwxrwx 5 root root 4.0K Sep  6 15:25 .
drwxr-xr-x 3 root root 4.0K Sep  6 15:25 ..
-rwxrwxrwx 1 root root  318 Sep  6 15:25 .dockerignore
drwxrwxrwx 8 root root 4.0K Sep  6 15:25 .git
-rw-rw-rw- 1 root root 7.9K Sep  6 15:25 .gitignore
-rw-rw-rw- 1 root root  510 Sep  6 15:25 README.md
drwxrwxrwx 7 root root 4.0K Sep 6 15:25 aux
-rw-rw-rw- 1 root root 2.1K Sep  6 15:25 bitbucket-pipelines.yml
-rwxrwxrwx 1 root root  399 Sep  6 15:25 build-module.sh
-rw-rw-rw- 1 root root  133 Sep  6 15:25 model.zip
drwxrwxrwx 3 root root 4.0K Sep 6 15:25 src

[Matin Lotfaliei · Accepted solution]

I just learned that the LFS option does not work with any docker images including `python:3.10`. This is a missing part of the documentation.

I had to install the `git-lfs`  package and pull manually in the script.

[Syahrul]

G'day, @Matin Lotfaliei 

Welcome to the community!

It seems that the LFS options are missing, which are necessary for cloning LFS during the build process. For more detailed information, please refer to our documentation. Essentially, you need to add the following:

clone:
   lfs: true

I hope this helps.

Regards,
Syahrul

[Matin Lotfaliei]

The LFS options are present.