Pipeline Artifacts with wildcards for shared pipeline templates

Question

I am attempting to create a shared pipeline template that can be referenced and used by our repositories. The template should be as flexible as possible to account for the different types of projects/modules. I have a "Java project build" step where it uses maven or gradle to  build the project and at the end of the step, I want the build artifacts to be passed on to future steps.

script:
  - ./mvnw -B clean verify
artifacts:
download: false # Do not download artifacts in this step
  paths:
    - "**/target/**"

The intention of the above step snippet is to build a maven project and then find all of the build artifacts for the maven project. Typically for maven projects, they would be in a "target" directory so this wild card "**/target/**" should be doing just that.

In practice, this does not seem to work for a project structure as such:

- .mvn/
- src/
- target/
- .gitignore
- bitbucket-pipelines.yml
- mvnw
- mvnw.cmd
- pom.xml

I expected the target directory and all of it's content to be captured in the pipeline run but is not.

This is one project use case but it should also find multi-module projects with multiple target directories.

Any help is appreciated.

Theodora Boudale · Answer

Hi Fernando,

Thank you for your reply and you are very welcome!

I'm not familiar with the python glob package, the implementation may be different.

I talked with one of my colleagues, and you can also use the following in the artifact path in the yml:

paths:
  - "**target/**"

This is going to match both the target directory inside the clone directory, and also directories like dir1/target, etc.

Kind regards,
Theodora

Fernando Mota · Answer

This seems to work fine.

The python glob documentation states that the double asterisk will match 0 to more directories. I assumed that "**/target/**" would also capture "target/**". I tested this using the python glob package as well. Perhaps there are glob implementation differences?

Regardless, your recommendation will work for my use case.

Thank you!

Theodora Boudale · Answer

Hi Fernando and welcome to the community!

The path you are using, "**/target/**", is going to match any folders named target that exist within another folder in the clone directory. So, folders like dir1/target/**, dir2/target/**, etc. It is not going to match a folder named target that exists in the clone directory.

If you want to match the latter as well, you can adjust your definition as follows:

script:
  - ./mvnw -B clean verify
artifacts:
  download: false # Do not download artifacts in this step
  paths:
    - "**/target/**"
    - target/**

I have tried that in a repo of mine and it works. Does it work for you as well if you make this adjustment?

Kind regards,
Theodora