It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Bitbucket Server Rest Api: Clone all repositories

Shreedhar Dhanawade Aug 17, 2017

I want to clone all repositories in my Bitbucket Server in order to have backups.

I trying to use Bitbucket rest api but not getting the required list of all repositories.

  curl -u username:password  https://servername:9090/rest/api/1.0/projects/~username/repos -k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    61    0    61    0     0    230      0 --:--:-- --:--:-- --:--:--   230{"size":0,"limit":25,"isLastPage":true,"values":[],"start":0}

 Is anything I am missing in command.

I have gone through the Bitbucket rest api doc but and using same command from there but not getting the required result.

3 answers

1 vote
Shawn Woodford Oct 18, 2017

Try this curl:

curl -sku username:password https://servername:9090/rest/api/1.0/users/usernamehere/repos
Salim Ben Allal Apr 11, 2018

Is there a way to move from Stash to Bitbucket cloud?

Firstborn Apr 11, 2018

Yes using the Stash server API and Bitbucket Cloud API but you have to do everything yourself manually or with a script, there is no Atlassian tool and they won't help you with the migration.

Salim Ben Allal Apr 11, 2018

So I guess for a few repositories, better to this manually. It is SHAME that atlassian does not provide a tool that allow the migration between Atlassian products

Shawn Woodford Apr 11, 2018

Yes I agree, I developed a bash script to handle a migration which I may release on GitHub if I can clean it up and make it easy for others to run.

Salim Ben Allal Apr 11, 2018

That would be great and I am sure it will help a lot of people :-)

Shawn Woodford Aug 29, 2018

Hi @Salim Ben Allal I just released my migration script which works on a Linux Bitbucket Server: https://github.com/swoodford/migrate_bitbucket_server_to_bitbucket_cloud

0 votes
Edwin Kyalangalilwa Community Leader Aug 20, 2017 • edited Aug 28, 2018

Try this. You may have to do some modifications.

Install stashy - https://pypi.python.org/pypi/stashy

#!/usr/bin/env python

import stashy, os stash = stashy.connect("http://localhost:7990/stash", "admin", "admin")

projects =
stash.projects.list()

for project in projects:
for repo in stash.projects["%s" %(project["key"])].repos.list():
for url in repo["links"]["clone"]:
// http or ssh
if (url["name"] == "ssh"):
os.system("git clone %s" %(url["href"]))
Shreedhar Dhanawade Aug 21, 2017

Thanks for the response edwin. I have alredy tried this and it will not work.

reasons:

1st- I am using bitbucket server.

2nd, With rest api of bitbucket server you can only list the personal repositories.

Edwin Kyalangalilwa Community Leader Aug 23, 2017 • edited

Try stashy instead (i've edited the script above), make sure you have access to the repos you want to backup.

Suresh Sakhare Aug 28, 2018
While executing above query I got this error: 'bitbucket' is not defined 
Edwin Kyalangalilwa Community Leader Aug 28, 2018

@Suresh Sakhare

for repo in stash.projects["%s" %(project["key"])].repos.list():
Suresh Sakhare Aug 28, 2018

Instead of bitbucket you can use stash

Edwin Kyalangalilwa Community Leader Aug 28, 2018

After the rename to Bitbucket is when I used:

bitbucket = stashy.connect()
Suresh Sakhare Aug 29, 2018 • edited

@Edwin Kyalangalilwa I downloaded all repos from server, now I am trying to push them to cloud bitbucket. Is there any way to push those downloaded repositories to bitbucket cloud instance with the branches and pull request history?

 

Thanks in advance :)

Edwin Kyalangalilwa Community Leader Aug 29, 2018

@Suresh Sakhare,

It will be a continuation of the current script. Basically you can join the script to do the cloning from server and push to cloud.

For each project - create it in cloud and the same for each repo

Please look into 

Repo.create()

Not sure but there might be a way of passing in the project to create the repo under.

https://stashy.readthedocs.io/en/latest/api.html#stashy.repos.Repos.create

pwd = os.getcwd()
for project in projects:    
cloud_bitbucket.projects.create(project["key"], project["name"])
for repo in bitbucket.projects["%s" %(project["key"])].repos.list():        
cloud_bitbucket.Repo.create()
  for url in repo["links"]["clone"]:
// http or ssh
if (url["name"] == "ssh"):
os.system("git clone %s" %(url["href"]))
os.chdir('%s.git' %(repo["name"]))                    
new_url = "ssh://git@bitbucket.org/<company>/%s.git" %(repo["name"])   
os.system("git remote add new_url %s" %(new_url))                    
os.system("git push -f --all new_url")                   
os.chdir(pwd)
Shawn Woodford Aug 29, 2018

Hi @Suresh Sakhare@Shreedhar DhanawadeI just released my migration script which works on a Linux Bitbucket Server: https://github.com/swoodford/migrate_bitbucket_server_to_bitbucket_cloud

Suresh Sakhare Aug 30, 2018

Thanks a lot@Shawn Woodford@Edwin Kyalangalilwa , It helps me.

Now I am working on pull-requets migration script, Doing the following steps.

 

Accessing pull requests history from server using server api.

  1. Get all projects
    1. Get repositories from project.
      1. Get PullRequests from repo.
        1. Get PullRequest from PullRequests
        2. Create PR for cloud instance with server PR details.
        3. Based on the Server PR state update the cloud PR state.
      2.  
    2.  

Is there any need to create temp branch and do cherry-pickup and then raise PR?

Also facing issue to setup the Author while raising PR. It always creator of PR as author even I mention the author detail in PR data.

PR Data

{

"title": "POP-1: Merged this PR",
"description": "* POP-1: commit 1 merged\r\n* POP-1: commit 2 merged",
"state": "OPEN",
"open": true,
"closed": true,
"createdDate": 1535637935000,
"updatedDate": 1535637945000,
"source": {
"id": "refs/heads/POP-2",
"displayId": "POP-2",

"repository": {
"slug": "mergedpr1",
"id": 1,
"name": "mergedpr1",
"scmId": "git",
"state": "AVAILABLE",
"statusMessage": "Available",
"forkable": true,
"public": false
},
"branch":{
"name":"POP-2"
}
},
"destination": {
"id": "refs/heads/master",
"displayId": "master",

"repository": {
"slug": "mergedpr1",
"id": 1,
"name": "mergedpr1",
"scmId": "git",
"state": "AVAILABLE",
"statusMessage": "Available",
"forkable": true,
"public": false
},
"branch":{
"name":"master"
}
},
"locked": false,
"author": {
"type": "author",
"user": {
"username": "sureshsakhare",
"displayName": "suresh sakhare",
"type":"user"
}

},
"reviewers": [
{

"username": "xyz",
"emailAddress": "xyz.pqr@www.com",
"displayName": "xyz pqr"

}
],
"participants": []
}

any suggestion on it ?

Shawn Woodford Aug 30, 2018

Great!  In my case I had never considered migrating PRs, but that all open PRs would be merged and closed before migration, so I have no experience creating a PR or migrating an open PR with the API.  I'd have to see how this could work.

Suresh Sakhare Sep 03, 2018 • edited

Hi @Shawn Woodford, I tried your script to clone and push repos from sever to cloud. It always return me the token expire error. I think this issue because of callback url field. If I am correct then what will be the value of callback url field.

{
"error": {
"message": "Access token expired. Use your refresh token to obtain a new access token."
},
"type": "error"
}

 

Suresh Sakhare Sep 03, 2018

@Shawn Woodford Its callback URL issue. I must set callback URL when using Oauth2 authentication.

Is there any need of ssh key? I am executing the this script with https protocal, In migrating repo to cloud step, I got the permission denied(publickey) issue.

Shawn Woodford Sep 04, 2018

Yes, you should follow the instructions from Bitbucket for creating your Oauth2 access, but the callback URL can be anything really since the script won't be using it for anything.

And yes, you do need to be able to run a `git clone` on all repos being migrated as the `root` user on the server that is running the script, so for this you can add an authorized ssh key to your Bitbucket Server with admin or system admin level permission, and make sure the root user can access that ssh key.  Save it in `/root/.ssh/id_rsa` and see lines 216-223, where it says `## If issues with SSH/git clone:`, just uncomment that block if you're still having trouble.

0 votes
Shreedhar Dhanawade Aug 29, 2018

That is nice Edwin. thanks for the answer. Just wondering, how we can clone all stuff (each branch and tag).

Shawn Woodford Aug 29, 2018

My script will do this, it only requires using an admin level account to access the Bitbucket server

Shreedhar Dhanawade Aug 31, 2018

Great ! I will have look, thank you. FYI, I am on windows environment.  

Suggest an answer

Log in or Sign up to answer
This widget could not be displayed.
This widget could not be displayed.
Community showcase
Published in Bitbucket Pipelines

Building a Bitbucket Pipe as a casual coder

...ipe.sh :  #!/bin/bash source "$(dirname "$0")/common.sh" enable_debug extra_args="" if [[ "${DEBUG}" == "true" ]]; then extra_args="--verbose" fi # mandatory variables R...

3,096 views 1 20
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you