What to do with your Mercurial repos when Bitbucket sunsets support

526 comments

kalthad November 3, 2019

Hi 

Ed, this is very helpful. I have the feeling that named branches are supported but only  'one way ', that is when converting a hg repository, they are converted to git branches, but when I clone the converted git 
repository with hg (and the hg-plugin) the question is: will I see named branches or bookmarks?

The hg-git plugin saves some extra meta information (which is not relevant for git) and will allow the
bi-directorial conversion. I am keen to see what happens with your script. I will try it out and report back .

I have already some questions (since I don't know really git) but I better will rise issues in your repository.
thanks again

Uwe

Like sreedevin likes this
philipstarkey November 4, 2019

Just thought I would let people know I'm still working on a tool to automate the move of repositories to GitHub along with preservation of all meta-data in an archive that can be published on github pages.

It will also be of use to people not moving to GitHub (you'll just have to find somewhere else to publish the archive of issues/pull-requests/etc).

It's not quite ready to share yet, but everything is coming along nicely. I haven't encountered any show-stoppers yet! Currently the tool backs up everything necessary from the BitBucket API and I'm half-way through generating the HTML/javascript for reading this backup (for the github pages archive). I'm also part way through automating the import to GitHub. Hopefully it won't be to much longer before it's ready for testers!

 

@Edgard Schmidt Thanks, I'll check it out soon! I also found some code here that should do something along those lines too, but I haven't tested to see if it works yet (it's quite old...)

Like # people like this
kevin_karplus November 4, 2019

@philipstarkey  When you say "all metadata" are you including both the issue-tracking database and the wiki?  That would be very useful for me to move the one repository I have left on BitBucket.  

Edgard Schmidt November 4, 2019

@philipstarkey 

I also found some code here that should do something along those lines too, but I haven't tested to see if it works yet (it's quite old...)

Nice find! However, by looking at the code, I guess this tool is not able to translate markup-less commit references.

philipstarkey November 5, 2019

@kevin_karplus Issue tracking: yes. Currently I can reproduce the issue tracker (including attachments) on a static github pages site. It's a little rough around the edges, but I'm aiming to get something minimal working before I do cosmetic improvements (especially since cosmetic improvements can be applied later, even after BitBucket kills off mercurial support). I also plan to import issues to the GitHub tracker like https://github.com/jeffwidman/bitbucket-issue-migration does, but with some different formatting and hopefully better URL rewriting. My understanding is that attachments don't work all that well though, and all comments are attributed to the import user account, which is why I also went for the static option on github pages as I felt it was a better preservation option (I also needed the static site for pull-request discussions which I don't think there is any way to migrate nicely to an active github repository).

As for wikis...this was something I forgot about! I should be able to add it in though. If you want to do it yourself by hand though, these are the steps I'll be doing in my script (assuming I can automate it...there is no API for GitHub wikis...*sigh*) :

  1. Use the GitHub import tool to import the master repository (the code repo that the wiki is attached to).
  2. Enable the GitHub wiki on this new repo, and create a single blank page. This step is needed to initialise the wiki's git repo. This is what I'll call the "actual wiki" for the purposes of these instructions.
  3. Use the GitHub import tool to import the hg wiki to a new github repository. Give it the URL <bitbucket repo>/wiki (which is what you would use to clone the repo locally). We'll call this "temp wiki" for the purposes of these instructions.
  4. Clone the temp wiki to your PC.
  5. Add the actual wiki as a git remote: git remote add wiki https://github.com/<owner>/<repo>.wiki.git
  6. Pull from the new git remote into your temp wiki local clone: git pull wiki master --strategy-option ours --allow-unrelated-histories
    This is a bit of a hack, but allows you to merge two unrelated git repos (effectively)
  7. Push the merged repo back to the actual wiki: git push wiki master

In theory, this then results in a conversion from a hg wiki to a git(hub) wiki, and keeps it attached to the code repository as well. I'd be interested to know if there are any issues with this as I don't have complex wikis to test with. If you have a public one, please shoot me the URL!

Like Joseph Heenan likes this
philipstarkey November 5, 2019

@Edgard Schmidt Ah, interesting! That's definitely something I'd want to do.

kevin_karplus November 5, 2019

@philipstarkey  The repository I'm trying to move to github is https://bitbucket.org/abe_k/pterodaq/wiki/Home  (I've taken over from Abe as the maintainer of the software).  Everything on the repository should currently be publicly readable, but not writable.

Milan Brezovsky November 6, 2019

You admirable fools. You're just doing what Atlassian should be paying their developers to do. I truly hope their morally bankrupt management will suffer mass migrations away from their mediocre products.

Like # people like this
philipstarkey November 6, 2019

@kevin_karplus Thanks! I'll use that as a test case and will share a link to the results once I've finished writing the tool.

@Milan Brezovsky Oh I know! That's specifically why my tool helps you move to GitHub and not to BitBucket git. If Atlassian want us to move to the industry standard of git, and don't want to spend any money to help us, then they can deal with us moving to the industry standard of GitHub.

Like # people like this
Milan Brezovsky November 6, 2019

Amen, and more power to you, sir!

Our company was ok with BitBucket, even with their frequent mysterious downtimes and bugs like "now all users is pull requests have a hash instead of a name" and Firefox-specific bugs... and we were planning to move to git anyway, but after this we're discussing moving to github as well. And honestly I'll have a hard time recommending Atlassian products to anyone ever again.

Like Chloroform Cowboy likes this
Joseph Heenan November 8, 2019

@philipstarkey Your tool sounds really good, I like your approach of converting issues/pull request to static html - looking forward to seeing it & thank you for working on this!

blais November 10, 2019

Atlassian: Please provide some simple way to archive all the tickets for download before you delete repos. Doesn't matter which format. Otherwise deleting those tickets  you will seriously piss off a *lot* of OSS developers.

Like eao197 likes this
Steven Peters November 10, 2019

@blais you can export issues manually into a zip file:

https://confluence.atlassian.com/bitbucket/export-or-import-issue-data-330797432.html

It would be nice if pull request conversations could be exported as well.

TaronM November 25, 2019

So is there seriously no way to convert a Mercurial repository into a Git repository and keep all the existing Issues, Wiki, Comments, Downloads, etc? It's one thing to expect to figure out the conversion to Git for the repository itself, but how do I prevent all this other valuable data from being deleted? 

CodeRock76 December 6, 2019

Seems that GitHub is importing a mercurial reposirtory easily from bitbucket.

Nicolas Carrier December 6, 2019

Is there a possibility to backup a whole hg repository, along with all its PR history in a compressed archive so that one can at least have a backup?

Ruslan Berezyuk December 6, 2019

@Nicolas Carrier oh boy! This is what everybody here is expecting - something reasonable that any reputable company would do - let you take away your data when the service is discontinued.

Unfortunately, there is no way to do so in this case. I understand there is always a risk to lose your data when keeping it online, for example, if a company goes out of business. This is not a case of Atlassian ... yet ... 

Nicolas Carrier December 6, 2019

@Ruslan BerezyukI mean, I'm not even asking for any hg -> git conversion tool, just a way to retrieve our data.

I guess if any european equivalent of the french CNIL put its nose in this affair, Atlassian would be in trouble as the right to access to a user's data is one of the fundamental rights they strive to protect...

Like jhw likes this
Ruslan Berezyuk December 6, 2019

@Nicolas Carrier right exactly. Download the data (issue, PR, comments, etc) in some formate (HTML or even raw db) to keep it as a backup. 

What you mention is actually interesting, I'm wondering if there is some European user who could bring this case up to the right place (ie some agency related to the user data protection, etc). 

a4z December 6, 2019

just converted some repos, code and wiki works, download, that you of course need to move manual, will reset the counter, so you start again at 0

issues will be lost.It's a shame that there is no official way to migrate these, and not losing the download counter, but at least in my case I can't complain since I use this service for free

kevin_karplus December 6, 2019

I moved my last repository from BitBucket to GitHub.  The GitHub importer moved the repository itself,

https://github.com/jeffwidman/bitbucket-issue-migration moved the issues, and I moved the Wiki (which had only 2 pages) manually.

GitHub seems to want new repositories (making import fairly easy), while BitBucket wants to drive users away (making converting repositories from hg to git in place difficult).

Like # people like this
Fabien Ninoles December 11, 2019

I see this to see some people experience with migration.
I never have good experience with hg-git or fast-export but have done git-to-hg multiple times with hg convert, as well as a few hg-to-git, including mirroring.  The experience were generally good and the complexity were mostly related to submodules (which convert mapping help a lot).

pepoluan December 14, 2019

Well, it's been a blast, BitBucket.

I came to you only because of Mercurial.

Now, I'm migrating away my repositories to other Mercurial hosting. I've migrated* my FOSS projects to SourceForge (which actually has, surprisingly, a good Markdown renderer!), and I'm going to migrate my personal projects to SourceHut.

I'm seriously considering of deploying Heptapod on a VPS to store my private code.

* by "migrate" here means doing "hg clone" from my repo in BitBucket then "hg push" to the other Mercurial hosting... followed by a deletion of the repo + link to new hosting.

I consider myself lucky I don't have many Issues / Tickets so I can simply up and go.

I shudder to think about the nightmare of migrating Issues / Tickets and Wiki and other things for big projects.

Like # people like this
webmarka December 14, 2019

I choosed BitBucket many years ago for our repositories since we are using Mercurial HG. This decision to drop Mercurial support is very disappointing to me and will lead to a lot of overtime to do. We choosed BitBucket specifically for the fact that it was supporting Mercurial. it was an element that distinguished your service. Now, you have almost nothing different from Gitlab or Github... Sad.

Like # people like this
moritz December 15, 2019

I migrated everything to the “industry standard”, GitHub. 

Comment

Log in or Sign up to comment
TAGS
AUG Leaders

Atlassian Community Events