Subversion, FishEye/Crucible (the Source + Review Bundle) on OnDemand End of Service

[hhung]

Atlassian will be responding to your questions about the end of service of Subversion, FishEye/Crucible (the Source + Review Bundle) on this dedicated thread.

Ask a question by adding a comment below.

Visit the End of Service FAQ to see if we've already answered your question.

[Stefaan Lesage]

Hi,

As a result to the announcement, I am trying to migrate my Atlassian OnDemand hosted SVN to Git, and I am following your guide at http://go-dvcs.atlassian.com/display/aod/Migrating+from+Subversion+to+Git+on+Bitbucket

So in fact, I downloaded a backup of my SVN, installed that in a local repository on my mac, then did the initial clone, then set up the Syncing ... but now comes my problem.

Our developers will still be comitting to the hosted OnDemand SVN. From what I can see, the sync part is only syncing between my local SVN and my local Git. But how do I actually get the changes made to the OnDemand SVN into my local SVN so it can be synced with the git repository ? Or isn't that the way you should approach this ?

Regards,

Stefaan

[Nanda Kumar]

Stefaan were you able to resolve this?

[Stefaan Lesage]

Not really Ron ... I tried quite a few things without any luck.

[Peter Drier]

@Helen, support is nice, but it doesn't each other as much as public forums do.. Even better would be to have comments enabled on the actual instruction pages.. so people like me could tell others that the instructions on this page: https://go-dvcs.atlassian.com/display/aod/initial+clone for example are just wrong and prevent any resyncing against the original svn repository. or I'd add that the clean-git on this page https://go-dvcs.atlassian.com/display/aod/Clean+Up is necessary for any branches to be moved into git. or that the bitbucket-push command on this page https://go-dvcs.atlassian.com/display/aod/Upload needs an admin account for the *-ondemand team as the first parameter, followed by the password, team account (*-ondemand) followed by the repo name.. but alas, people looking at those pages can't receive my help.

[Paul O'Flynn]

I had the same thot Peter re. comments on those pages. However I could see them possibly getting out of hand wrt to the volume of comments. A more open-ended discussion forum would be optimal (and simple/free) IMHO.Thx again for sharing your wisdom!

[hhung]

Peter, Paul -

Thanks for your feedback. You raise a good point Peter, but on the flipside, we wanted to keep the guides and that microsite as orderly as possible, as Paul pointed out.

Why? I'm sure you can imagine that if we had let comments open on all pages there, the possibility of it turning into pages filled with a lot of comments, opinions and discussions would be high. When you have someone new coming in to use the guides, the experience of them realising that they need to trawl through all the comments would be painful and not necessarily relevant.

To avoid this potential mess, this is why we have *this* dedicated forum to let people discuss exactly what you are raising as a point. And for those who are seeking more 1:1 technical support, then via our Support system: https://support.atlassian.com

[Paul O'Flynn]

The instructions on how to create a backup of your SVN repo are also incorrect (step 2 on the first (i.e. "Prepare") page. However I don't mind as I'm going with Peter's suggestion of not doing this, his is a much better strategy.

[hhung]

@Paul - we've just updated the documentation there on the site, after taking into account your comments and Peter's. Cheers

[Igor Lobanov]

As of today Bitbucket and OnDemand have different user accounts. DVCS JIRA connector can infer an association between user accounts, but as far as account management is concerned (passwords, permissions, etc.) these two systems are completely separate. Even though we're happy to embrace DVCS, if we switched from OnDemand SVN to Bitbucket today, we'd be required to have all the users creating yet another account and yet another password, and then some amount of admin overhead to make sure all permissions are set correctly.

Could you advise if there any improvement for the situation planned for the next 12 month, while migration window lasts?

[AudraA]

Hi Igor,

Yes - we are investing in improving the user management across multiple systems including OnDemand and Bitbucket. You will start to see a consolidation of user accounts in the next year. We are not sure if it will all be complete by October 2013, but we are striving to complete this work within 2013.

Cheers,

Audra Eng
VP, Product Management

[Igor Lobanov]

Audra, thanks for the information. We'll wait with migration until later to reduce admin overhead and security concerns. Hope some form of synchronisation for logins and passwords will be introduced before by October 2013.

[Peter Drier]

I can't express how important this is to us as well. Telling all of our developers that they have to go and create a bitbucket account before I can even add them to our git repo and having them all do that correctly is a management nightmare. It would be better at the very least if the invitation from the team repo would work with accounts not already setup & essentiallly auto create the account for the person.. As it is now, the "invitation" doesn't work if the user doesn't have an account already, which confuses most people and introduces a poor chicken and egg situation which could have easily been avoided.

[Peter Drier]

At the very least, Atlassian should be able to make a quick tool which can link to the existing JIRA user base, and auto create all of those accounts within the bitbucket team container.. it's a one off process, and that would solve my issues, and likely smooth things over for most of the people in my situation..

As it stands now, it seems like Atlassian didn't think this out before throwing out this forced conversion. User management is a big deal, and forcing us into two different user administration systems isn't what we're paying you for.

[eirikwahl]

Two facts, and a question:

1. Atlassian is ending Fisheye/Crucible OnDemand service
2. Atlassian is removing GIT-repository from Fisheye, and creating a new product Stash (with simple (?) code-review functionality)

Does this mean that Fisheye/Crucible is a dying product (for installation on our own server)? Or is Atlassian still actively developing Fisheye/Crucible. Are you going to develop new features/versions for FishEye/Crucible?

If the answer is something along the lines "FishEye is not going away", can you please state how big development team you are going to have on FishEye 6 months from now, compared to 6 months ago? And do you have any roadmap for FishEye (to ease my distress)?

Thanks!

[Sten Pittet]

Hi,

I'll go straight to the point: FishEye/Crucible isn't going away and we're actively investing in it.

Now, I'll break it down to the different points that you mentioned:

(a) The 2 facts
Your facts are right. But those decisions were hard calls that we had to make to give FishEye/Crucible a better future. These applications were not originally architected for the OnDemand platform nor FishEye/Crucible was designed to handle repository management. We tried for a long time to stretch the original nature of the products and we decided then that it would be better to get back to its original strengths: be a great solution to index, track and review code across different SCMs behind the firewall. More about these decision can be found in the announcements that we made:
- End of life announcement of Internally Managed Repositories for FishEye
- End of life announcement for FishEye/Crucible OnDemand

(b) The team
We're growing the team. The size of the engineering team 6 months from now will have doubled from what it was 6 months ago, you will be able to judge it yourself by looking at the release notes for each release where the people on the FishEye/Crucible team are listed.

(c) The roadmap
The current priority is to improve the performance of FishEye/Crucible for large instances. In other words, our short term focus is on making sure that existing features are fast, even with large sets of data.
To achieve this goal we are revamping the way we store and treat data coming from SCMs but we are also working on making everything more intuitive and simpler to use as we believe that usability helps a lot on productivity. All new improvements and features are planned to achieve this goal.

I hope that this helps to understand better our plans and if you still have some concern don't hesitate to get in touch with me directly. You can send me an email at spittet@atlassian.com.

Cheers,

Sten Pittet
FishEye/Crucible Product Manager

[Norman Abramovitz]

How does these end of life OnDemand products affect the self hosted versions?

1) Will there still be any subversion support?

2) Will subversion plugins be maintained?

[hhung]

Hi Norman, there is no impact at all on our installed, self-hosted versions of FishEye and Crucible.

We are continuing to invest in both self-hosted products.

1) After 15 October 2013, there will no longer be any hosted Subversion support. However you can continue to use Subversion, as it will be supported until the end of service period lapses. Our installed versions will continue to support Subversion.

2) If you're asking about the JIRA/Subversion plugin, then yes that is still is an Atlassian-supported plugin to hook up Subversion to self-hosted JIRA.

[Chris Wagner]

How do we link our BitBucket account and OnDemand account for billing/user management purposes?

Answering my own question...

http://go-dvcs.atlassian.com/display/aod/Configuring+a+Bitbucket+account+for+your+team

[hhung]

Hi Chris, this depends on whether it is an existing or new Bitbucket account:

If you select to migrate to Bitbucket via your my.atlassian.com account, you will be issued with a new free license. In this case, your new Bitbucket and existing OnDemand accounts will be automatically integrated for billing purposes. Then, when you create new users in OnDemand, you can also invite them to Bitbucket simultaneously. More info is available here.

Unfortunately, if you want to link an existing Bitbucket account with OnDemand for billing/users, it is not currently possible.

I hope this answers your question, thanks.

[Henry Widman]

Thanks Charles. I will give that a try. If I do have problems getting Fisheye up and running then I will create a new thread or possibly submit a support ticket.

Have a great holiday!

[cofarrell]

Hi Henry,

(And for anyone reading this). I'm not sure posting question on this page is the best place any more, they are just getting lost in the noise. :(

Regarding your question I think the warning says it all - you should be able to add 'Dfile.encoding=UTF-8' to the fisheyectl.bat script where it is executing the java command.

If you get stuck create another Answers page and/or feel free to raise a support ticket if you get stuck.

Cheers,

Charles

[Henry Widman]

Although this may be more of a Windows server or Java admin question maybe someone has seen this issue and found a fix for it. I have a copy of Fisheye installed on a local server and have a copy of the Fisheye respository that I downloaded from Atlassian (back in Sept.) and have attempted to load the backup respository to Fisheye but I got an error message regarding the codeset being used. The command and error are below:

D:\fecru-3.1.2\bin>fisheyectl.bat restore --file=d:\fisheye_crucible\respository
_backups_JIRA\FeCru-backup-20131014mod.zip --dbtype=sqlserver2008 --jdbcurl="jdb
c:jtds:sqlserver://limdsqlprdtx01.berry.local:1433;databaseName=FishEye;" --user
name=crucible password=fisheye

WARN: your default encoding (windows-1252) does not support unicode.
This is likely to cause problems if any of the filenames, branch names or tag na
mes in one of your repositories contain unicode characters.
Update your locale settings or start with the -Dfile.encoding=UTF-8 option to av
oid these problems.
INFO - Using log4j configuration file: D:\fecru-3.1.2\log4j-client.xml
INFO - FishEye arguments: [--file=d:\fisheye_crucible\respository_backups_JIRA\
FeCru-backup-20131014mod.zip, --dbtype=sqlserver2008, --jdbcurl=jdbc:jtds:sqlser
ver://limdsqlprdtx01.berry.local:1433;databaseName=FishEye;, --username=crucible
, password=fisheye]

It seems that I need to make either a Windows server configuration change or a Java configuration change so that Windows and/or Java uses the unicode codeset. From what I have seen in web searches the command mentioned in the error is a Java configuration setup change but not sure. In any case if anyone knows how to do what needs to be done to get the repository loaded and Fisheye happy it would be a big part of making the holiday season a merry one for me.

Thanks,

Henry

[cofarrell]

Hi Tina,

Sorry for the delay. Did you manage to remove the references? If not I would suggest raising a support ticket to make sure your question is answered.

Cheers,

Charles

[Tina Harron]

Now that the End of Service is officially here, I notice that I am redirected to a migration page whenever I try to use the old repository. That's ok - our repository has been moved.

When I try to view the source tab in JIRA, I get a timeout. But all my project links seem to be intact - that is I have a Fisheye link that points to the OnDemand Fisheye instance in all of the projects. (I am using a hosted Fisheye version 2.7.15) I was expecting these links would be gone when OnDemand Fisheye went away. In the Application Links page for the project, I also see an option to add more links using the OnDemand Fisheye.

How can I get rid of all references to the OnDemand Fisheye? Went into the Atlassian account last week but was not able to deactivate the subscription.

Thanks!

[Curtis Harvey]

I guess technically the migraiton script works - it found the repositry and wouldn't let me do anything. :-) However your suggestion on adding the remote repo from BB and then running the git push commands worked perfectly. Thanks!

[Curtis Harvey]

According to the last page of the Switch to Git documentation at https://go-dvcs.atlassian.com/display/aod/Switch, I should be able to still sync svn with a Bitbucket repo after the upload process, but the Sync page only give instructions on how to sync with the local git repo. Syncing of the local repos and the cleanup appeared to be successful. If I try to run the command on the upload page (java -jar svn-migration-scripts.jar bitbucket-push <team_admin_account> <team_account> <repository-name>) I get the error message that the team account already has a repository with that name. How can I sync to a repo that is in Bitbucket?

[cofarrell]

Hi Curtis,

That doesn't sound right, I'm fairly sure that command should check if the repository exists. In any case it doesn't matter - all you should theoretically need to do it run 'git push --all && git push --tags' in your Git repository, and if it has been previously configured it should do the rest. Otherwise, you can go the repository in BB, copy the 'clone' url, run 'git remote add origin $URL' and then run those commands.

Let me know if that helps?

Charles

[Curtis Harvey]

I guess technically the migraiton script works - it found the repositry and wouldn't let me do anything. :-) However your suggestion on adding the remote repo from BB and then running the git push commands worked perfectly. Thanks!

[CCOT]

Charles O'Farrell [Atlassian] ·

I ran the git branch -all command and I see all the branches listed at the path remotes/<..>. However, when I push the repo to Bitbucket I dont see any branches listed.

[CCOT]

Also I did the migration from a linux box.

[cofarrell]

HI Ira,

So currently they're not technically 'branches', so when you push they won't go anywhere. You need to convert them first, and then everything should work as expeted. You want to run the 'clean-git --force' command as documented here:

https://go-dvcs.atlassian.com/display/aod/Clean+Up

Let me know if that helps.

Cheers,

Charles

[collegescheduler]

Hey not sure if anyone will be helped by this, but I did the migration this weekend. It took about 3 to 4 hours of actual work. Now I wanted to stick to Subversion for sure, and I also didn't want to host my own repository. So what I did was buy an account here:

http://beanstalkapp.com/pricing

I bought the $50/month account. Basically just did a dump of Subversion out of JIRA and then uploaded it to Beanstalk. Mine was 1.5gb so I had to put it on a web server for Beanstalk to download, you can also put it on FTP, etc.

Once that was dialed in, then I got Fisheye/Crucible up and did an import of that data into my local instance. I'm using Microsoft SQL Server. Here's the command I ran once I had edited config.xml. (FeCru-backup-20130914\config\config.xml) Obviously that's my filename, yours will be different.

The last part of the import was assembling the command to run. (I'm in a Windows environment) Here's what I ran:

C:\CrucibleHome\bin\fisheyectl.bat restore -f C:\FisheyeCrucibleRestoreFile\FeCru-backup-20130914.zip --dbtype sqlserver2008 --jdbcurl jdbc:jtds:sqlserver://localhost;databaseName=CrucibleFisheye; --username CrucibleSQL --password mypasswordhere --force

So I had made a SQL Server DB called "CrucibleFisheye" and setup a SQL user with the above username and pass.

Then I set up my repositories to point to Beanstalk.

The initial indexing takes a while because Fisheye is reaching out via SSL to Beanstalk's servers, not accessing a local Subversion repository.

Here's my logic: I'm not experienced in running Subversion myself. Or Fisheye/Crucible. However, if Fisheye/Crucible goes down or I mess it up in the future, it's all good because Beanstalk is still hosting my code and my developers can check in/out etc and it's business as usual.

Anyways, hope this helps someone!

[collegescheduler]

Also just for the record, I don't work for Beanstalk and I'm not affiliated with them, yada yada.

[CCOT]

I recently converted our SVN repository over to Git and pushed it to BitBucket, however, I am having an issue. It seems that branches did not come over. The tags and trunk were fine , but I don't see the branches. Our SVN had a standard layout and I followed the migration guide to the letter.

[cofarrell]

Hi Ira,

Do you have the original Git repository where you ran the conversion? If you run 'git branch --all' or 'git log --graph --decorate --oneline --all' do you see those branches listed somewhere, perhaps under something called 'remotes/'?

It might also be worth raise a separate support ticket as well so we can help you further. If you could attach the output of those commands, and $TMPDIR/svn-git-migration.log (under %TEMP% under Windows) if it exists that will help too.

Cheers,

Charles

[CCOT]

I recently converted our SVN repository over to Git and pushed it to BitBucket, however, I am having an issue. It seems that branches did not come over. The tags and trunk were fine , but I don't see the branches. Our SVN had a standard layout and I followed the migration guide to the letter.

[collegescheduler]

What if everyone pays $100 a month for this service and Atlassian keeps offering it?! :) I'd gladly pay $100 a month or even $150, much cheaper than running my own server, and no admin headaches... I realize this was shut down probably because it wasn't profitable. How about 100 clients paying for it at $100 a month.... 500? At some point it has to make sense.

[Kinto Soft]

Do you want to reduce your admin headaches? Try Subversion Plus.

https://marketplace.atlassian.com/plugins/pbeltranl.subversion.jira.pbeltranl-jira-subversion-plugin

The coming soon 4.0 version (which hopefully will be released within a week) will support some new reports/statistics, so you will be able to track the activity of your repositries almost at real time from the artifacts (files & directories) and users points of view. Furthermore, you will save a server as it runs within the JIRA add-on context and it is installed on JIRA with a single click.

Much more free features are in the queue, like professional support for merge. Dozens of companies have written me to say they simply love it.

[Henry Widman]

Just putting this out there to get some feedback as to installing Subversion (also Fisheye/Crucible) to a Redhat VM. Has anyone done this? If so which of the supported SVN packages did you install from http://subversion.apache.org/packages.html and to what version of Redhat? How difficult was it? How does it work? These sites are really sparse on information, I don't even see any system resources requirements listed except for Redhat version numbers. Also I didn't see any Subversion downloads on the redhat.com site itself, is there supposed to be?

Not being a systems guy I was wondering also if the "client" referred to in packages was just a server side command line interface or is it an actual user side client? We are currently using TortoiseSVN as the user side client and would like to continue to use that.

Also some packages say they come with an Apache server, is this in place of the svnserver and must that be used in place of svnserver (or am I getting the terminology mixed up)?

Thanks for any feedback.

Henry

[MiroslavA]

Nancy,

If you run scripts with the new updated svn dump on the same local mercurial repo (initial clone), then it should work. But you you will need full svn dump everytime as far as I know.

It will certainly not work if you create new local mercurial repo and don't use the former initial clone.

For further reference about converting subversion to mercurial:

http://mercurial.selenic.com/wiki/ConvertExtension

<tt>REVMAP</tt> is a a simple text file that maps each source commit ID to the destination ID for each revision. Unless specified, it defaults to the <tt>.hg/shamap</tt> in the destination directory. This file is automatically created and updated on each commit copied, its purpose is to track which commits were already imported and which were not - and thanks to it allow to resume interrupted import and to make incremental updates. It is important to note that this <tt>REVMAP</tt> file is not copied when you a clone a repository. So you need to manually move it over if you are going to make incremental updates on a clone of the original import repository.

Cheers,

Miroslav

[MiroslavA]

Hi Nancy,

Installed curl. Running script again. Will have an update when it completes.

However, additional question - when we perform the clone/upload steps ongoing (between now and switchover) -

we start with creating a new svn-dump file, then clone, then upload. Correct?

This will just update Mercurial with the additional committed changes?

It's not possible to update Mercurial with additional commit changes after creating a new svn-dump.

Cheers,

Miroslav

[Nancy Broderson]

Miroslav,

Thank you for the response. In that case I am confused by the migration instructions. As it states here:

~~~

https://go-dvcs.atlassian.com/display/aod/Ongoing+Sync+Mercurial

That after the initial clone, the team should commit only to Subversion until ready to switch, and:

You can automate the synchronisation so that it runs every n minutes/hours/days.

To do this, automate the initial clone and upload steps with a tool that you are comfortable with.

~~~

Your answer above is not in line with the documentation. If we are planning to review the code in Mercurial before the actual switch, and obviously people are still committing to Subversion during this time - how then do we get the new commits over to Mercurial?

Thank you,

Nancy

[Paul O'Flynn]

We are moving from SVN/FishEye/Crucible to BitBucket/Git. We have about 40 developers and one repo. Formal code review capability seems extremely light from what I can tell. If one is coming from Fisheye etc, what would you recommend as the best code review workflow/tools? Thanks.

[Patrick Hill]

Paul,

If you are moving to Bitbucket, I suggest you try out Bitbucket's Pull Request functionality. Since the end of life of Crucible in OnDemand, internally we've been switching our teams to use Bitbucket. As part of this change, we have been incorporating feedback from both customers and the internal teams who are using Bitbucket for code review. Part of the new tool has required us to address missing features, and in other cases to adjust the process we have to incorporate the advantages Git has over Subversion.

You should have a read of the features that we have been rolling out since the end of life announcement:

• http://blog.bitbucket.org/2013/05/07/beautiful-and-relevant-notifications/
• http://blog.bitbucket.org/2013/04/02/finnovation-brings-you-a-heap-of-new-features/
• http://blog.bitbucket.org/2013/03/04/the-new-bitbucket-dashboard-all-your-code-activity-all-in-one-place/
• http://blog.bitbucket.org/2013/02/25/pull-requests-now-with-reviewers-and-smarter-notifications/
• http://blog.bitbucket.org/2012/12/21/autocompleting-your-username-mentions/
• http://blog.bitbucket.org/2012/11/27/innovation-week-launches-several-new-features/

As for workflows, have a read through our Git website for ideas about what workflow best suits your team: https://www.atlassian.com/git/workflows

Patrick

[Paul O'Flynn]

Thanks for the prompt reply Patrick, but I'm looking for something that integrates with Jira onDemand. Since posting this question I see a new idea out there: https://jira.atlassian.com/browse/DCON-334 which I believe would satisfy my requirements.

[Paul O'Flynn]

Thanks for the prompt reply Patrick, but I'm looking for something that integrates with Jira onDemand and allows the grouping of commits for review w/o doesn't requiring you to create feature branches.

[Nancy Broderson]

Hello, I am working on moving to Bitbucket/Mercurial.

I am not sure if this question belongs here or on a different support page. Please let me know.

I am running everything on an Ubuntu virtual machine. Just ran the uploadRepos.sh script, and the messages I receive state first:

./uploadRepos.sh: line 47: curl: command not found

and I get a warning butbucket.org certificate with fingerprint not verified

but then it gives a message "Repository XX was successfully pushed to (correct bitbucket team account)

I log in to BitBucket and go to the team account, but I see no repositories?

Please help!

[Nancy Broderson]

Installed curl. Running script again. Will have an update when it completes.

However, additional question - when we perform the clone/upload steps ongoing (between now and switchover) -

we start with creating a new svn-dump file, then clone, then upload. Correct?

This will just update Mercurial with the additional committed changes?

[Nancy Broderson]

Update - one repository appears to have made it to Bitbucket - but the next one seems 'stuck'... it says 'sending' in my linux window, but stuck at 37.73%. Does anyone know if there is a timeout setting that I need to change?

[Nancy Broderson]

Hello, I am working on moving to Bitbucket/Mercurial.

I am not sure if this question belongs here or on a different support page. Please let me know.

I am running everything on an Ubuntu virtual machine. Just ran the uploadRepos.sh script, and the messages I receive state first:

./uploadRepos.sh: line 47: curl: command not found

and I get a warning butbucket.org certificate with fingerprint not verified

but then it gives a message "Repository XX was successfully pushed to (correct bitbucket team account)

I log in to BitBucket and go to the team account, but I see no repositories?

Please help!

[cofarrell]

Hi Norman,

If you want to continue to SVN you can connect a local FishEye/Crucible server via application links as described here:

https://go-dvcs.atlassian.com/display/aod/connect

As far as I'm aware we don't have any plans to introduce a new mechanism for SVN repository interactions. Bitbucket is certainly the recommended OnDemand source hosting from now on, and will only support Git/Mercurial.

Cheers,

Charles