Bitbucket Server Rest Api: Clone all repositories

I want to clone all repositories in my Bitbucket Server in order to have backups.

I trying to use Bitbucket rest api but not getting the required list of all repositories.

  curl -u username:password  https://servername:9090/rest/api/1.0/projects/~username/repos -k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    61    0    61    0     0    230      0 --:--:-- --:--:-- --:--:--   230{"size":0,"limit":25,"isLastPage":true,"values":[],"start":0}

 Is anything I am missing in command.

I have gone through the Bitbucket rest api doc but and using same command from there but not getting the required result.

3 answers

Try this curl:

curl -sku username:password https://servername:9090/rest/api/1.0/users/usernamehere/repos

Is there a way to move from Stash to Bitbucket cloud?

Yes using the Stash server API and Bitbucket Cloud API but you have to do everything yourself manually or with a script, there is no Atlassian tool and they won't help you with the migration.

So I guess for a few repositories, better to this manually. It is SHAME that atlassian does not provide a tool that allow the migration between Atlassian products

Yes I agree, I developed a bash script to handle a migration which I may release on GitHub if I can clean it up and make it easy for others to run.

That would be great and I am sure it will help a lot of people :-)

Hi @Salim Ben Allal I just released my migration script which works on a Linux Bitbucket Server: https://github.com/swoodford/migrate_bitbucket_server_to_bitbucket_cloud

Try this. You may have to do some modifications.

Install stashy - https://pypi.python.org/pypi/stashy

#!/usr/bin/env python

import stashy, os stash = stashy.connect("http://localhost:7990/stash", "admin", "admin")

projects =
stash.projects.list()

for project in projects:
for repo in stash.projects["%s" %(project["key"])].repos.list():
for url in repo["links"]["clone"]:
// http or ssh
if (url["name"] == "ssh"):
os.system("git clone %s" %(url["href"]))

Thanks for the response edwin. I have alredy tried this and it will not work.

reasons:

1st- I am using bitbucket server.

2nd, With rest api of bitbucket server you can only list the personal repositories.

Try stashy instead (i've edited the script above), make sure you have access to the repos you want to backup.

While executing above query I got this error: 'bitbucket' is not defined 

@Suresh Sakhare

for repo in stash.projects["%s" %(project["key"])].repos.list():

Instead of bitbucket you can use stash

After the rename to Bitbucket is when I used:

bitbucket = stashy.connect()

@Edwin Kyalangalilwa [Mumo Systems] I downloaded all repos from server, now I am trying to push them to cloud bitbucket. Is there any way to push those downloaded repositories to bitbucket cloud instance with the branches and pull request history?

 

Thanks in advance :)

@Suresh Sakhare,

It will be a continuation of the current script. Basically you can join the script to do the cloning from server and push to cloud.

For each project - create it in cloud and the same for each repo

Please look into 

Repo.create()

Not sure but there might be a way of passing in the project to create the repo under.

https://stashy.readthedocs.io/en/latest/api.html#stashy.repos.Repos.create

pwd = os.getcwd()
for project in projects:    
cloud_bitbucket.projects.create(project["key"], project["name"])
for repo in bitbucket.projects["%s" %(project["key"])].repos.list():        
cloud_bitbucket.Repo.create()
  for url in repo["links"]["clone"]:
// http or ssh
if (url["name"] == "ssh"):
os.system("git clone %s" %(url["href"]))
os.chdir('%s.git' %(repo["name"]))                    
new_url = "ssh://git@bitbucket.org/<company>/%s.git" %(repo["name"])   
os.system("git remote add new_url %s" %(new_url))                    
os.system("git push -f --all new_url")                   
os.chdir(pwd)

Hi @Suresh Sakhare@Shreedhar DhanawadeI just released my migration script which works on a Linux Bitbucket Server: https://github.com/swoodford/migrate_bitbucket_server_to_bitbucket_cloud

Thanks a lot@Shawn Woodford@Edwin Kyalangalilwa [Mumo Systems] , It helps me.

Now I am working on pull-requets migration script, Doing the following steps.

 

Accessing pull requests history from server using server api.

  1. Get all projects
    1. Get repositories from project.
      1. Get PullRequests from repo.
        1. Get PullRequest from PullRequests
        2. Create PR for cloud instance with server PR details.
        3. Based on the Server PR state update the cloud PR state.
      2.  
    2.  

Is there any need to create temp branch and do cherry-pickup and then raise PR?

Also facing issue to setup the Author while raising PR. It always creator of PR as author even I mention the author detail in PR data.

PR Data

{

"title": "POP-1: Merged this PR",
"description": "* POP-1: commit 1 merged\r\n* POP-1: commit 2 merged",
"state": "OPEN",
"open": true,
"closed": true,
"createdDate": 1535637935000,
"updatedDate": 1535637945000,
"source": {
"id": "refs/heads/POP-2",
"displayId": "POP-2",

"repository": {
"slug": "mergedpr1",
"id": 1,
"name": "mergedpr1",
"scmId": "git",
"state": "AVAILABLE",
"statusMessage": "Available",
"forkable": true,
"public": false
},
"branch":{
"name":"POP-2"
}
},
"destination": {
"id": "refs/heads/master",
"displayId": "master",

"repository": {
"slug": "mergedpr1",
"id": 1,
"name": "mergedpr1",
"scmId": "git",
"state": "AVAILABLE",
"statusMessage": "Available",
"forkable": true,
"public": false
},
"branch":{
"name":"master"
}
},
"locked": false,
"author": {
"type": "author",
"user": {
"username": "sureshsakhare",
"displayName": "suresh sakhare",
"type":"user"
}

},
"reviewers": [
{

"username": "xyz",
"emailAddress": "xyz.pqr@www.com",
"displayName": "xyz pqr"

}
],
"participants": []
}

any suggestion on it ?

Great!  In my case I had never considered migrating PRs, but that all open PRs would be merged and closed before migration, so I have no experience creating a PR or migrating an open PR with the API.  I'd have to see how this could work.

Hi @Shawn Woodford, I tried your script to clone and push repos from sever to cloud. It always return me the token expire error. I think this issue because of callback url field. If I am correct then what will be the value of callback url field.

{
"error": {
"message": "Access token expired. Use your refresh token to obtain a new access token."
},
"type": "error"
}

 

@Shawn Woodford Its callback URL issue. I must set callback URL when using Oauth2 authentication.

Is there any need of ssh key? I am executing the this script with https protocal, In migrating repo to cloud step, I got the permission denied(publickey) issue.

Yes, you should follow the instructions from Bitbucket for creating your Oauth2 access, but the callback URL can be anything really since the script won't be using it for anything.

And yes, you do need to be able to run a `git clone` on all repos being migrated as the `root` user on the server that is running the script, so for this you can add an authorized ssh key to your Bitbucket Server with admin or system admin level permission, and make sure the root user can access that ssh key.  Save it in `/root/.ssh/id_rsa` and see lines 216-223, where it says `## If issues with SSH/git clone:`, just uncomment that block if you're still having trouble.

That is nice Edwin. thanks for the answer. Just wondering, how we can clone all stuff (each branch and tag).

My script will do this, it only requires using an admin level account to access the Bitbucket server

Great ! I will have look, thank you. FYI, I am on windows environment.  

Suggest an answer

Log in or Sign up to answer
Community showcase
Published Aug 21, 2018 in Bitbucket

Branch Management with Bitbucket

As a project manager, I have discovered that different developers want to bring their previous branching method with them when they join the team. Some developers are used to performing individual wo...

2,411 views 9 12
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you