Pipeline Error: Dirctory not found using TYPO3

[Mylene Kreiger]

Hi there,

i'm using TYPO3 and my pipelines always worked... yesterday I tried to release an update and I got the following error:

Generating optimized autoload files

> post-autoload-dump: @typo3-cms-scripts

> typo3-cms-scripts: typo3cms install:generatepackagestates

Uncaught TYPO3 Exception The "/opt/atlassian/pipelines/agent/build/vendor/bin/typo3/sysext/*/" directory does not exist.

thrown in file /opt/atlassian/pipelines/agent/build/vendor/symfony/finder/Finder.php in line 592

Script typo3cms install:generatepackagestates handling the typo3-cms-scripts event returned with error code 1

Script @typo3-cms-scripts was called via post-autoload-dump

I can't imagine what's wrong. I've nothing change within the pipeline. I tried to rerun a pipeline that worked out some months ago and suddenly got the same error. 

Can anyone help?

Here ist my bitbucket-pipelines.yml:

definitions:

caches:

composer: vendor/

node: frontend/node_modules

steps:

- step: &composer

name: "Setup composer in pipeline"

caches:

- composer

image: php:8.1.11

script:

- apt-get update && apt-get install -y libfreetype6-dev libjpeg62-turbo-dev libpng-dev libzip-dev git

- docker-php-ext-configure gd --with-freetype --with-jpeg

- docker-php-ext-install -j$(nproc) gd

- docker-php-ext-install -j$(nproc) zip

- curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

- composer install --verbose --prefer-dist --no-progress --no-interaction --no-dev --optimize-autoloader

artifacts:

- composer.json

- composer.lock

- packages/**

- public/**

- vendor/**

- step: &npm

name: "Generate frontend assets"

image: node:16

caches:

- node

script:

- npm ci

- npm run build

artifacts:

- packages/**

- step: &deploy

name: "Deploy on server"

image: php:8.1.1

deployment: production

script:

- ls -ls

- pwd

- apt-get update && apt-get install -y rsync openssh-client git

- vendor/bin/dep --file=./.deployer/deploy.php deploy -vv stage=${BITBUCKET_BRANCH}

- step: &deployStag

name: "Deploy on staging server"

image: php:8.1.1

deployment: staging

script:

- ls -ls

- pwd

- apt-get update && apt-get install -y rsync openssh-client git

- vendor/bin/dep --file=./.deployer/deploy-staging.php deploy -vv stage=${BITBUCKET_BRANCH}

pipelines:

branches:

master:

- step: *composer

- step: *npm

- step: *deploy

staging:

- step: *composer

- step: *npm

- step: *deployStag

Thanks!

Mylene

[Theodora Boudale · Accepted solution]

Hi Mylene,

Pipelines builds that run on Atlassian's infrastructure use Docker containers. For every step of your build, a Docker container starts (the build container) using the image you have specified in your bitbucket-pipelines.yml file. The repo is cloned in the build container, and then the commands of the step's script are executed. When the step finishes that Docker container gets destroyed.

Failures may sometimes be caused by changes in the Docker image you use as a build container or in the dependencies of the project.

I would suggest checking the following guide, specifically Scenario 1: Builds were successful earlier, but started failing recently:

• https://confluence.atlassian.com/bbkb/troubleshooting-bitbucket-pipelines-1141505226.html

There are possible causes listed there and steps you can follow to troubleshoot.

Since rerunning the last successful build also fails, you can use this guide to debug the failed build locally with Docker by following the steps here:

• https://confluence.atlassian.com/bbkb/troubleshoot-failed-bitbucket-pipelines-locally-with-docker-1318882094.html

Please feel free to reach out if you have any questions!

Kind regards,
Theodora

[Mylene Kreiger]

Hi Theodora,

thanks for your help!

I tested with Docker locally and the same step got through without errors. What can I do now?

Here is the log from my local testing:

And the exact same log after "Installing typo3/cms-tstemplate (v11.5.35): Extracting archive" from the bitbucket pipeline:

Kind regards, Mylene

[Theodora Boudale]

Hi Mylene,

• Can you please confirm if the step that is failing is the *composer one and if the command that is failing is the composer install or a different one?
  
  
• When you reproduced locally with Docker, did you provide the exact same image SHA from the failing pipeline as the base image in the Dockerfile (FROM statement)?

Kind regards,
Theodora

[Mylene Kreiger]

Hi Theodora,

yes the composer install is failing: composer install --verbose --prefer-dist --no-progress --no-interaction --no-dev --optimize-autoloader

I provided the same image SHA from the failing pipeline. I copied it from the build setup and put it in the Dockerfile:

And this was my Dockerfile:

Kind regards,

Mylene

[Theodora Boudale]

Hi Mylene,

Thank you for the info.

In the output of your local testing, there are several messages about fixing permissions. The output of the pipeline has warnings that plugins have been disabled because the build is running with root user and that this may be the cause of the failure.

Can you try adding the following command at the beginning of the script of the *composer step in your yml?

- export COMPOSER_ALLOW_SUPERUSER=1

Kind regards,
Theodora

[Mylene Kreiger]

Hi Theodora,

that worked! 🤩 Could you explain to my why it worked that way?

Kind regards,
Mylene

[Theodora Boudale]

Hi Mylene,

That's good to hear, thank you for the update!

The output in the screenshot you shared from the failed Pipelines build includes the following:

Plugins have been disabled automatically as you are running as root, this may be the cause of the script failure.
See also https://getcomposer.org/root

I checked that page, https://getcomposer.org/root, and the very first question mentions that as of Composer 2.4.2, plugins are disabled automatically when running as root, and Bitbucket Pipelines builds run by default with root user.

Even though I don't see in your output the error indicated in the first question, the message about disabled plugins made me think that this might be the issue. My suggestion comes from this page; if I understood correctly, plugins will not be disabled when setting this variable.

Kind regards,
Theodora

[Mylene Kreiger]

Hi Theodora,

well... that makes a lot sense 😛
Thanks again for your help!

Kind Regards,
Mylene

[Theodora Boudale]

You are very welcome, Mylene! Please feel free to reach out if you ever need anything else!