How can I integrate bitbucket yml file in another one ?

[fatma_elbehi]

Hi Team,

I need your help for this case please! It is extremely important us.

How can I integrate a bitbucket yml file in another one for different source code projects  to build a successful pipeline because I need the output of some project as an input to other ones.

[Theodora Boudale]

Hi @fatma_elbehi,

I'm afraid that it is not possible to include one Bitbucket yaml file in another one.

Could you please give us some more details on your use case, so we can see if there is a different way to achieve what you like?

For example:
- You mention different source code projects. Do you have a separate repository for each of these projects? Or are they all part of the same repository?
- If you have multiple repos, are you running builds on all of them?
- What kind of output do you want to use as input for the others? Is it artifacts that are generated?
- An example of how you would like this to work would also be helpful to better understand what you're trying to achieve.

Kind regards,
Theodora

[fatma_elbehi]

Hi Theodora,

Thanks for answering me!

yes sure!

So for the first Qte: You mention different source code projects. Do you have a separate repository for each of these projects? Or are they all part of the same repository? --> I have one project that contains 4 repositories 

Second Qte: If you have multiple repos, are you running builds on all of them? --> yes I have as I confirmed in my previous answer and I need to build all of them one by one in order (mvn clean install) to get from each one of them a jar file that will be as an input for the other repository.

Third Qte:  What kind of output do you want to use as input for the others? Is it artifacts that are generated? --> jar file and the 4th repository should generates the exe artifact after taking the jars from the other repos.

Last Qte: 

- An example of how you would like this to work would also be helpful to better understand what you're trying to achieve.--> honestly, I prefer to find a way to build all these repos in one bitbucket pipeline file.

I am waiting for your help!

Best,

Fatma

[Theodora Boudale]

Hi Fatma,

Thank you for your reply and for explaining your use case.

I mentioned earlier as well that it is not possible to reference the yml file of one repo inside the yml file of a different repo. We have a pipe that can be used in the yml file to trigger a pipeline in a different repo:

• https://bitbucket.org/atlassian/trigger-pipeline/src/master/

However, it is not possible to pass artifacts to the other repo when you use this pipe.

One suggestion I can make is the following: you mentioned you have 4 repos, let's assume they are named
- repo-A
- repo-B
- repo-C
- repo-D

You choose 1 repo where Pipelines builds will run (let's say it's repo-A), you create one yml file in that repo, and you run the builds for all repos in that one (I'll explain below how). The drawback of this approach is that builds for all repos will run and show in this one repo only, so you won't be able to see the builds of e.g. repo-B inside repo-B.

This approach could work as follows: you make 4 steps in the yml file, one for each repo.

1st step:

- repo-A is cloned (no special configuration needed for that -- since the build is running on repo-A, repo-A will get cloned)
- you run your builds commands
- you define your artifact(s)

2nd step: You'll run the build for repo-B

- For that step, set in the yml file

clone:
  enabled: false

so that repo-A won't get cloned.
- Clone repo-B.
When you clone repo-B, a new directory will be created in the build's directory, so you'll need to change directories with a cd command to navigate to repo's B source code. You can run any build commands there.
- The artifact of the 1st step will be available, but in the path specified in step 1. If you generated the artifact in the build directory of step 1, it will be available in the build directory of step 2 (and not inside repo-B clone)

This step could look as follows:

- step:
    name: 'repo-B Build'
    clone:
      enabled: false
    script:
      - git clone git@bitbucket.org:workspace-id/repo-b.git
      - cd repo-b
      - <build commands>

The clone command above will clone repo-B's main branch, but you can also specify in the clone command which branch you want to clone, the depth of clone, etc.

You can add a 3rd and 4th step for repo-C and repo-D, similar to the 2nd step.

In order to be able to clone other repositories from Pipelines of repo-A, you'll need to set up authentication. You can use Access keys for that, you can find steps on how to achieve that here: https://bitbucket.org/blog/cloning-another-bitbucket-repository-in-bitbucket-pipelines

Finally, I would just like to mention that we have a feature request in our issue tracker to allow yml files to be referenced from multiple repositories: https://jira.atlassian.com/browse/BCLOUD-20859

You can vote for that feature request, add yourself as a watcher, and also leave any feedback there.

If you have any questions, please feel free to let me know.

Kind regards,
Theodora

[fatma_elbehi]

Hi Theodora,

After I cloned the second repo in the first one 

I received the attached error window

 

Could you please help?

Thanks,

Fatma

[fatma_elbehi]

Hi Theodora,

For my previous error it is okay now it works fine with me but now I have another issue when I  built the last repository.

Could you please help?

best,

Fatma

[Theodora Boudale]

Hi Fatma,

The artifact that cannot be found by the build, is it a jar file that you defined as an artifact in a previous step?

And does the error occur at a step where you clone a repo in the script (like repo-B in my example)?

If this is the case, and if the jar file needs to be in the clone directory for the build to succeed, you may need to move it before you run the build commands.

If you run the following command in the script:

- git clone git@bitbucket.org:workspace-id/repo-b.git

a directory named repo-b will be created inside the build directory for the clone.

If you defined in a previous step a certain jar file as an artifact and if it exists in the build directory, you can move it inside repo-b clone with the following command

mv myfile.jar repo-b

Replace the name of the jar file and of the clone directory with the ones applicable for your case.

Kind regards,
Theodora

[fatma_elbehi]

Hi Team,

If I have different branches and I would like to run my pipeline in specific one so here the clone option does make sense or should I add another command after the clone to work on that particular branch?

Kindly I need your help in this case!

Best,

Fatma

[fatma_elbehi]

Hi Team,

Another plz question, I have defined a specific plugin which should be built when I run the pipeline but I did not receive any result when the build finished.

any idea plz?

Best,

Fatma

[Theodora Boudale]

Hi Fatma,

If I have different branches and I would like to run my pipeline in specific one so here the clone option does make sense or should I add another command after the clone to work on that particular branch?

Does you question concern my earlier suggestion to run builds in e.g. repo-A, and then do a manual clone for repos repo-B, repo-C and repo-D?

If so, are you referring to repo-B, repo-C, repo-D which you are cloning in the script?

If this is the case, you can specify which branch to clone in the git clone command. E.g. if you want to clone branch develop of repo-B, you can run a command as follows:

git clone -b develop --single-branch git@bitbucket.org:workspace-id/repo-b.git

If you don't want to clone the whole branch, you can also use the option depth and clone only the last commit of this branch:

git clone -b develop --single-branch --depth 1 git@bitbucket.org:workspace-id/repo-b.git

If your question concerns something different than the use case I suggested, please share some more details on your use case so I can better help you.

Another plz question, I have defined a specific plugin which should be built when I run the pipeline but I did not receive any result when the build finished.

I would like to ask if you could please create a separate question for this issue and share some more details, like what commands are executed in the build, the output of these commands, and what you were expecting to see instead. It's best to have a separate question for each issue as this allows for better troubleshooting and the question doesn't get cluttered with many different issues.

Kind regards,
Theodora