Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Celebration

Earn badges and make progress

You're on your way to the next level! Join the Kudos program to earn points and save your progress.

Deleted user Avatar
Deleted user

Level 1: Seed

25 / 150 points

Next: Root

Avatar

1 badge earned

Collect

Participate in fun challenges

Challenges come and go, but your rewards stay with you. Do more to earn more!

Challenges
Coins

Gift kudos to your peers

What goes around comes around! Share the love by gifting kudos to your peers.

Recognition
Ribbon

Rise up in the ranks

Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!

Leaderboard

Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,460,016
Community Members
 
Community Events
176
Community Groups

Cloning git repo from self-hosted runner

Edited

Hello!

I've successfully set up a self-hosted Linux runner to serve a Bitbucket Cloud pipeline.

On each build the git repo is cloned from Bitbucket, additionally some git submodules are pulled as part of the build script.

The total amount of data cloned from git in these operations weighs in at 2.4GB, and the number of builds per day, when fully operational, should be in the double digits. That's a lot of data to shuffle around the network.

Questions:

1. Will these git clone operations count towards some data rate limit and make things inaccessible/slower if exceeded?

2. Is there a recommended way to have a local git proxy of sorts that just keep up to date with Bitbucket Cloud and then let the self-hosted runner clone from that (local) location instead?

3. Is there any way to have submodules pulled as part of the built in cloning process?

/K

1 answer

1 accepted

0 votes
Answer accepted

Hi Kristoffer,

 

1. Will these git clone operations count towards some data rate limit and make things inaccessible/slower if exceeded?

The rate limits we have in place have to do with the number of requests, and not with the size of the data being transferred.

You can see these limits here:

Clones via HTTPS count towards Git web (HTTPS://) requests - 60,000 requests per hour
Clones via SSH count towards SSH requests - 30,000 requests per hour

If the submodules are private repos and you use authentication to clone them, the calls are measured against the user ID of the user you use to clone.

 

2. Is there a recommended way to have a local git proxy of sorts that just keep up to date with Bitbucket Cloud and then let the self-hosted runner clone from that (local) location instead?

We don't have any official documentation on such a configuration.

 

3. Is there any way to have submodules pulled as part of the built in cloning process?

I'm afraid that this is not possible at the moment.

We have a feature request in our issue tracker:

If you'd be interested in that feature, I would suggest adding your vote (by selecting the Vote for this issue link) as the number of votes helps the development team and product managers better understand the demand for new features. You are more than welcome to leave any feedback, and you can also add yourself as a watcher (by selecting the Start watching this issue link) if you'd like to get notified via email on updates.

Implementation of new features is done as per our policy here and any updates will be posted in the feature request.

In the meantime, submodules need to be initialized and updated with a command in your script.

Kind regards,
Theodora

Is there way to disable automatic cloning altogether? I'd rather do all that from my build scripts, especially when there's need for more than one repo to be included in the build environment.

Hi @Timo Laru,

Yes, this is possible. You can use in your bitbucket-pipelines.yml the following:

clone:
enabled: false

If you have multiple steps and you want to disable cloning in all of them, you can place this at the beginning of the file, like below:

image: atlassian/default-image:3

clone:
enabled: false

pipelines:
default:
- step:
name: 'Build and Test'
runs-on:
- 'self.hosted'
- 'linux'
script:
- echo "Your build and test goes here..."
- step:
name: 'Lint'
runs-on:
- 'self.hosted'
- 'linux'
script:
- echo "Your linting goes here..."

 

Alternatively, if you want to disable cloning only for a specific step you can add this to a certain step:

image: atlassian/default-image:3

pipelines:
default:
- step:
name: 'Build and Test'
runs-on:
- 'self.hosted'
- 'linux'
script:
- echo "Your build and test goes here..."
- step:
name: 'Lint'
runs-on:
- 'self.hosted'
- 'linux'
clone:
enabled: false
script:
- echo "Your linting goes here..."


Kind regards,
Theodora

Like Timo Laru likes this

Thank you! That solved my problem :)

Like Theodora Boudale likes this

That's good to hear, you are very welcome!

Suggest an answer

Log in or Sign up to answer
TAGS

Atlassian Community Events