What's the dealio with jira clustering?

[JamieA]

Anyone have any inside information on the state of play re clustering?

I see 6.1 has new tables that look like they are to support replication, eg ReplicatedIndexOperation and ClusterMessage. I have heard about a new (expensive) jira archiving product - are these tables just for that, or is there genuine movement on clustering, or horizontal scaling?

[rmk · Accepted solution]

Hi Jamie and all.

Yes the Atlassian JIRA team is working on a clusterable solution for JIRA that will provide both HA and Scale across several nodes. This is in addition to another solution we are looking at to help with large instance performance - JIRA Archiving.

We're still early in the development and we hope to have it released mid next year.

Regards,

Roy Krishna
JIRA Product Management

[Dieter]

Cluster? Yes, but not before JIRA scales better with #fields/field configs/projects. My instance has 2100/400/1100 and this one would already have become unmanageable without performance fixes in JIRA core Update: i am affected by https://jira.atlassian.com/browse/JRA-25721and https://jira.atlassian.com/browse/JRA-27799and can't see yet how this should be resolved by clustering. Yet hopefully a cluster version will also fix these issues. If Atlassian could comment on that it would be great. They have my fixes to partly address these issues. I am already restrictive with fields and try to convince ussers to use labels but if you provide JIRA services for a 300000 user company with many different application domains you will face this problem sooner or later. This is simply a matter of mass imho.

[Nic Brough -Adaptavist-]

2100 fields is not being restrictive with fields, it implies you've got business and configuration problems, as well as technical ones. One of my clients had 800 when I started with them, and by simply talking to the users, we were able to reduce it to 90, which helped a bit with performance, and a LOT with business reporting and consistency.

Atlassian are trialling clustering, which should help with most of the issues with larger installations.

[Dieter Greiner]

Wow, can we hire you to talk to users ? (Not kidding) We have some divisions that clear their requirements and in these cases we just we need 10 extra fields for 2000 users. But now and then very special projects show up that need 100 custom fields following the customers terminology which are very different between and even inside divisions. We hope we can keep the number of fields at that level by improving our lifecycle management (detecting unused projects, archiving them, removing unused fields)

[Dieter Greiner]

Cluster? Yes, but not before JIRA scales better with #fields/field configs/projects. My instance has 2100/400/1100 and this one would already have become unmanageable without performance fixes in JIRA core

[Nic Brough -Adaptavist-]

Clustering the way Atlassian are looking at will help with that.

Although I'd say with 2100 fields, you've got business process problems as well as scaling.

[Harry Chan]

I think the problem with clustering is the Confluence / JIRA home folders. Unless that mostly disappears and a few other things, clustering won't happen. I don't see how they'd get customers to cluster and also arrange a distributed file system and other things all just to get it working.

[Radu Dumitriu]

Configuration files are easy to replicate (attachments too!). IBM Websphere did it this way. I think you refer to Lucene information (text indexing), for that there's SOLR to be used (not fully funny, but will work).

So a solution exists for sure.

[Harry Chan]

Anything is doable. Usability is the key. Once it gets too complex, add a lot of instructions - it's no longer feasible. It'll be a support/maintenance hell and neither the vendor nor the customers want to touch it.

It also greatly increases costs and diminishes the competitive advantages vs many other products in the market.

Having seen a large Websphere cluster being deployed and run, I'd say I don't want to deal with anything like it in the near future.

[JamieA]

I don't see why it will be so difficult. If you index something you also index it on the other nodes. It's the same database at the backend.

Also, this is not going to be free - my guess is each node will be the same price as jira, therefore the support you get will be commensurate with that.

BTW, you should be a bit more pro-clustering, given your company name!

[Harry Chan]

I am pro clustering. I did NOT say clustering is difficult. I'm merely pointing out the implementation problems due to the way Atlassian has currently structured it and how it can improve.

It's difficult because it's been made difficult. I'm sure Atlassian are aware of this and will try to address it. No sharing would be easiest to cluster with a traditional model of web servers, server side sessions etc - hence the home folder being an issue.

[Radu Dumitriu]

@Harry

Can you define "too complex"? If you think, the way WAS does clustering is quite simple. Of course, they could offer you a button and a list for you to say "cluster these instances using the default values", but in some cases, it will not be what you want.

What is trully killing is the clustering of an WebSphere Portal (which has been a nightmare from the very beginning).

Anyway, clustering is not *usually* attempted by stupid people ....

[Brett Taylor]

Interested here as well.

[Radu Dumitriu]

We have customers requiring clustering. At AtlasCamp I discussed with them (Atl) and they said to me that they are looking into the clustering - again. They said it's going to be very expensive (I personally don't care, but customers will do).

The target for a solution (but it's kinda blurry) would be half of 2014 (at least that's what I understood).

I will let Atl comment on this, I'm intrested as well ....