What is the recommende way to move files from one repository to another via pipeline?

Christof Koegler _Gaims GmbH_

We have a repository which hosts the code for our documentation server. Whenever one of our projects is updated, we run a pipeline to generate the documentation based on the latest code, push it to the documentation repository, and deploy it to the server from there.

The solution we are currently using relies on git to pull and push code from one repository to another. However, this requires SSH authentication, which in turn requires some hacky abuse of user SSH:

• Take the SSH key of the pushing repository
• Add it to a user
• Make sure that user has read/write permissions for the repository you want to push to

 

We would like to do this in a way that does not require a specific user, but the alternatives I can think of have their own downsides:

1. Push with git, but handle SSH authentication 'by hand'

Store the SSH keys in workspace variables and copy them to the required places during the pipeline.

Cons:

• Finicky to set up
• Pipeline might break if the build image is changed, e.g. files in unexpected locations etc.

2. Use the Bitbucket REST API to move the files

Set up an API token in a workspace variable and use a script to loop over files you want to push/pull

Cons:

• Files cannot be pulled at all - only their content
• Files must be added individually to the API request (might be 1000s of files)
• Difficult to determine what files have changed

 

Overall, it seems silly to use the API for handling tasks that git does so well. But there does not seem to be a good mechanism to authenticate between Bitbucket repositories.

What is the recommended way to move code between repositories?

Thanks,
Chris

Patrik S

Hey Chris!

By reading through the description of your use case, I would recommend leveraging Repository Access Tokens (RATs) as the authentication for the git command. 

Repository Access Tokens are attached to a repository, not to a specific user, and can be configured to have a limited scope (specified during creation). They are particularly useful when you need to authenticate your scripts or CI/CD tools.

So my suggestion, in this case, would be to follow the below sequence of steps : 

1. Create a Repository Access token in the destination repository and select the appropriate scopes, such as repo:read and repo:write.
   Make sure to save the access token code as it is displayed only once, and also the bot ID (<bot ID>@bots.bitbucket.org) that will be used later to push to this repository.
2. In the source repository - where the pipelines run - create two environment variables to store both the token and the bot email address. In this example, I will use BB_REPO_TOKEN and BB_REPO_BOT_ID
3. As part of your pipeline script in the source repository, use the RAT to clone the destination repository using the token : 
   

   git clone https://x-token-auth:$BB_REPO_TOKEN@bitbucket.org/{workspace}/{destination_repository}.git

4. On the path of your local cloned destination repository, set the Git author to be the bot account
   

   cd {destination_repository}
   git config user.email "${BB_REPO_BOT_ID}@bots.bitbucket.org"

5. Apply the updates to the clone of the destination repository inside the pipeline and commit the changes
   
6. Push the changes to the destination repository

   git push

You can try using the suggestion above and let us know how it goes.

For more details about Repository Access token, you're also welcome to check our documentation on that topic :

• https://support.atlassian.com/bitbucket-cloud/docs/repository-access-tokens/

Hope that helps! If you have any questions, let me know.

Thank you, @Christof Koegler _Gaims GmbH_ !

Patrik S