Scaling Fisheye/Crucible for a larger environment

[Russell H]

We currently host multiple SCM instances on different networks and we are looking at moving all these into one big enterprise class SCM system. One of our problems is that the smaller SCM instances include Fisheye/Crucible.

We currently have performance issues sometimes on the smaller instances when people try and do large code reviews and it can adversely affect Jira which is linked. This could have severe consquences if the entire site was using one big system.

Is there any known way of load-balancing and mirroring Fisheye and Crucible?

The other question I have is, is there currently a known way of performing single project migrates from Fisheye/Crucible into another instance?

Thanks,

Russell

[Daniel R · Accepted solution]

Hello Russell,

We've made some recent improvements in the way FishEye/Crucible interacts with JIRA (FishEye/Crucible version 2.9 and the latest JIRA FishEye/Crucible plugin) so I'd like to know what versions of the products you are using in regard to your performance issues. Unfortunately, we don't currently have a method to load balance or mirror a FishEye/Crucible instance. Perhaps one of our experts can help in this area. Please open a support ticket in our private JIRA instance at support.atlassian.com if you'd like me to assist in finding you a partner. Lastly, you can move repositories from one instance to another but you can't yet move projects -- the feature request is here:

https://jira.atlassian.com/browse/CRUC-681

In order to move repositories, look for this heading "How to Re-Index a Single Repository on a Test Server" on this page.

Thank you,

Daniel

[Russell H]

Thanks Daniel,

I thought that was the case, I just wanted to check.

I'll raise a ticket with Atlassian Support to see if anyone can point me in the right direction.

I've voted for that feature request for single project migrate. In the meantime, I'll start writing a script to interact the APIs and see if it can be done...

Cheers,

Russell

[Sergey Svishchev]

Russell, please share your findings, if you don't mind.

If API way doesn't pan out, I'm thinking of doing it on lower level (altering the database itself).

[Russell H]

@s2s:

No worries, will report back any info I get back from Atlassian on my support ticket for this one.

The problem we have is that even if we manage to get a single project migration working by using scripted API calls then we have the problem that this probably wouldn't be supported by Atlassian if anything went wrong. We really need the functionality to be supported since we run an enterprise SCM system.

Anyway, we'll have a go at scripting a crucible project migrate with APIs in the meantime just to see if it can be done...!

[JamieA]

Could you not just copy the entire database, then delete everything you didn't want from it using the web UI. At least this will take you down a "supported" path... I don't believe there is anything in the published API that will allow you to successfully migrate a crucible project with all reviews...

[Russell H]

Thanks for your comment Jamie. Unfortunately, we cannot do this because we have more than one instance that we are migrating projects from.

In total we have 6 instances of Crucible/Fisheye/Jira/SVN that we want to consolidate into one. We want to migrate one project at a time to make the migration process as simple as possible.

We have already managed to do a scripted single project migrate with all dependent entities (reviews etc) using a combination of APIs with a hefty perl script in an older version of Crucible. I have no doubt we could probably make it work on the newer version.

I guess my point is that it's about time that this was included in Crucible as standard functionality.

[JamieA]

Fair enough. I would check if possible that one instance can handle it. I cannot make one instance of fisheye/crucible cope with the amount of repos and projects we have. 2.9 seems to have a memory or resource leak where it needs restarting every two days.

[Russell H]

hmmm, that memory leak issue sounds ominous! I haven't tried 2.9 yet... I've found that it seems to be the size of individual commits/reviews that affects Fisheye/Crucible rather than the amount. If the java heap isn't optimized it can really crash out on scanning.

[Sergey Svishchev]

There's a problem with InfinityDB cache (its size is not capped and may exhaust heap of any size) --https://jira.atlassian.com/browse/FE-4427

There's a workaround (turns the cache off), but it causes a 10-15% performance hit at indexing time at least.

[Russell H]

Sigh, it seems like with every new version there are new bugs...! Thanks for pointing that out, it's useful to know.