Ed, this is very helpful. I have the feeling that named branches are supported but only 'one way ', that is when converting a hg repository, they are converted to git branches, but when I clone the converted git repository with hg (and the hg-plugin) the question is: will I see named branches or bookmarks?
The hg-git plugin saves some extra meta information (which is not relevant for git) and will allow the bi-directorial conversion. I am keen to see what happens with your script. I will try it out and report back .
I have already some questions (since I don't know really git) but I better will rise issues in your repository. thanks again
Just thought I would let people know I'm still working on a tool to automate the move of repositories to GitHub along with preservation of all meta-data in an archive that can be published on github pages.
It will also be of use to people not moving to GitHub (you'll just have to find somewhere else to publish the archive of issues/pull-requests/etc).
It's not quite ready to share yet, but everything is coming along nicely. I haven't encountered any show-stoppers yet! Currently the tool backs up everything necessary from the BitBucket API and I'm half-way through generating the HTML/javascript for reading this backup (for the github pages archive). I'm also part way through automating the import to GitHub. Hopefully it won't be to much longer before it's ready for testers!
@Edgard Schmidt Thanks, I'll check it out soon! I also found some code here that should do something along those lines too, but I haven't tested to see if it works yet (it's quite old...)
@philipstarkey When you say "all metadata" are you including both the issue-tracking database and the wiki? That would be very useful for me to move the one repository I have left on BitBucket.
@kevin_karplus Issue tracking: yes. Currently I can reproduce the issue tracker (including attachments) on a static github pages site. It's a little rough around the edges, but I'm aiming to get something minimal working before I do cosmetic improvements (especially since cosmetic improvements can be applied later, even after BitBucket kills off mercurial support). I also plan to import issues to the GitHub tracker like https://github.com/jeffwidman/bitbucket-issue-migration does, but with some different formatting and hopefully better URL rewriting. My understanding is that attachments don't work all that well though, and all comments are attributed to the import user account, which is why I also went for the static option on github pages as I felt it was a better preservation option (I also needed the static site for pull-request discussions which I don't think there is any way to migrate nicely to an active github repository).
As for wikis...this was something I forgot about! I should be able to add it in though. If you want to do it yourself by hand though, these are the steps I'll be doing in my script (assuming I can automate it...there is no API for GitHub wikis...*sigh*) :
Use the GitHub import tool to import the master repository (the code repo that the wiki is attached to).
Enable the GitHub wiki on this new repo, and create a single blank page. This step is needed to initialise the wiki's git repo. This is what I'll call the "actual wiki" for the purposes of these instructions.
Use the GitHub import tool to import the hg wiki to a new github repository. Give it the URL <bitbucket repo>/wiki (which is what you would use to clone the repo locally). We'll call this "temp wiki" for the purposes of these instructions.
Pull from the new git remote into your temp wiki local clone: git pull wiki master --strategy-option ours --allow-unrelated-histories This is a bit of a hack, but allows you to merge two unrelated git repos (effectively)
Push the merged repo back to the actual wiki: git push wiki master
In theory, this then results in a conversion from a hg wiki to a git(hub) wiki, and keeps it attached to the code repository as well. I'd be interested to know if there are any issues with this as I don't have complex wikis to test with. If you have a public one, please shoot me the URL!
@philipstarkey The repository I'm trying to move to github is https://bitbucket.org/abe_k/pterodaq/wiki/Home (I've taken over from Abe as the maintainer of the software). Everything on the repository should currently be publicly readable, but not writable.
You admirable fools. You're just doing what Atlassian should be paying their developers to do. I truly hope their morally bankrupt management will suffer mass migrations away from their mediocre products.
@kevin_karplus Thanks! I'll use that as a test case and will share a link to the results once I've finished writing the tool.
@Milan Brezovsky Oh I know! That's specifically why my tool helps you move to GitHub and not to BitBucket git. If Atlassian want us to move to the industry standard of git, and don't want to spend any money to help us, then they can deal with us moving to the industry standard of GitHub.
Our company was ok with BitBucket, even with their frequent mysterious downtimes and bugs like "now all users is pull requests have a hash instead of a name" and Firefox-specific bugs... and we were planning to move to git anyway, but after this we're discussing moving to github as well. And honestly I'll have a hard time recommending Atlassian products to anyone ever again.
@philipstarkey Your tool sounds really good, I like your approach of converting issues/pull request to static html - looking forward to seeing it & thank you for working on this!
Atlassian: Please provide some simple way to archive all the tickets for download before you delete repos. Doesn't matter which format. Otherwise deleting those tickets you will seriously piss off a *lot* of OSS developers.
So is there seriously no way to convert a Mercurial repository into a Git repository and keep all the existing Issues, Wiki, Comments, Downloads, etc? It's one thing to expect to figure out the conversion to Git for the repository itself, but how do I prevent all this other valuable data from being deleted?
@Nicolas Carrier oh boy! This is what everybody here is expecting - something reasonable that any reputable company would do - let you take away your data when the service is discontinued.
Unfortunately, there is no way to do so in this case. I understand there is always a risk to lose your data when keeping it online, for example, if a company goes out of business. This is not a case of Atlassian ... yet ...
@Ruslan BerezyukI mean, I'm not even asking for any hg -> git conversion tool, just a way to retrieve our data.
I guess if any european equivalent of the french CNIL put its nose in this affair, Atlassian would be in trouble as the right to access to a user's data is one of the fundamental rights they strive to protect...
@Nicolas Carrier right exactly. Download the data (issue, PR, comments, etc) in some formate (HTML or even raw db) to keep it as a backup.
What you mention is actually interesting, I'm wondering if there is some European user who could bring this case up to the right place (ie some agency related to the user data protection, etc).
just converted some repos, code and wiki works, download, that you of course need to move manual, will reset the counter, so you start again at 0
issues will be lost.It's a shame that there is no official way to migrate these, and not losing the download counter, but at least in my case I can't complain since I use this service for free
GitHub seems to want new repositories (making import fairly easy), while BitBucket wants to drive users away (making converting repositories from hg to git in place difficult).
I see this to see some people experience with migration. I never have good experience with hg-git or fast-export but have done git-to-hg multiple times with hg convert, as well as a few hg-to-git, including mirroring. The experience were generally good and the complexity were mostly related to submodules (which convert mapping help a lot).
Now, I'm migrating away my repositories to other Mercurial hosting. I've migrated* my FOSS projects to SourceForge (which actually has, surprisingly, a good Markdown renderer!), and I'm going to migrate my personal projects to SourceHut.
I'm seriously considering of deploying Heptapod on a VPS to store my private code.
* by "migrate" here means doing "hg clone" from my repo in BitBucket then "hg push" to the other Mercurial hosting... followed by a deletion of the repo + link to new hosting.
I consider myself lucky I don't have many Issues / Tickets so I can simply up and go.
I shudder to think about the nightmare of migrating Issues / Tickets and Wiki and other things for big projects.
I choosed BitBucket many years ago for our repositories since we are using Mercurial HG. This decision to drop Mercurial support is very disappointing to me and will lead to a lot of overtime to do. We choosed BitBucket specifically for the fact that it was supporting Mercurial. it was an element that distinguished your service. Now, you have almost nothing different from Gitlab or Github... Sad.
531 comments