Removing search template to improve performance

[Emre Toptancı _OBSS_]

Many Atlassian documents state that creating custom fields greatly effects system performance so creating a large number of custom fields (several hundreds of fields maybe close to a thousand) is not recommended. The documentation also states that not every field effects the system equally. For example selectlist fields are the least expensive and multiline text fields are the most expensive, since they hold much more information to index.

This makes me wonder, what happens if I set the "Search Template" of a custom field to None? I know this will remove the field from serach indexes and will make search for that field impossible. But some, if not most, of the fields on my customers' systems are for form information only. They do no use that information for searching or reporting.

How much performance benefit would I get by setting Search Template on these fields to None? Will removing the search template of many such fields improve performance greatly or will it be close to nothing?

[Nic Brough -Adaptavist- · Accepted solution]

It's hard to say.  You'll get some performance increase from removing each searcher, but whether your users will notice removal of one field is unlikely.  If you removed 20 searchers, you might get noticed a bit more.  It very much varies by type of field too.  As you've picked up from the docs, single select lists have much more simple index entries than text fields for example.

The differences are broadly proportional to the size of your data too, so if you've got quite a small JIRA, there's absolutely no point in doing it.   Most important though is where you expect this to have an impact.  The size of the index and complexity of fields affect performance in different ways.  If you're looking for "everything needs to run faster", attacking the problem at the index level is not going to help, as it's only going to improve bits of your user experience (and often not the bits they're complaining about)

[Emre Toptancı _OBSS_]

What we do for performance improvement on large systems is to limit contexts of custom fields to specific projects and/or issuetypes. This proved to be useful since it reduces the information to be indexed significantly. I see "setting the searcher to None" technique as a next step to this approach.

Let's say we are talking about one specific multiline text field. What is your gut feeling about the performance gain of setting the searcher to None vs actually removing the custom field from the system?

[Nic Brough -Adaptavist-]

Yes, setting searcher to none will help on the indexing side.

Removing the custom field will do a little bit more, as it removes the overhead of looking after the field too.  But my instinct is that the overhead of an individual field is fixed, and relatively small.  So deleting a text field and deleting a select list will have very similar impacts on performance, unlike the very different impacts of removing their searchers (I'm ignoring the searchers when I say "overhead").

If it helps, I got a measurable performance increase on one system I inherited from actually widening the context of fields.  We had 40 pairs of "start date/end date" fields, mostly limited to a handful of project contexts each with minor differences in description and usages.  We made two pairs global, merged the other 38 into them, and were able to delete 76 fields completely.  We got a 29% increase in the response time to the complex searches users were complaining about (although we did similar things on some other fields too, so it wasn't just the date fields.  I use them because 40 pairs of fields doing the same thing is just silly)

[Emre Toptancı _OBSS_]

Actually our current situation is deeper than that. Customer has a system with around 1000 custom fields. Many of these fields are kept just to use the hold information on the form, they are not used for JQL searches.

On one side I am pushing the customer to consolidate and reduce the number of fields, on the other side I really want to do what can be done technically. When talking about setting Search Templates to None, I will be forcing the customer to do this on probably more than a hundred fields. That should make a difference.

[Nic Brough -Adaptavist-]

I'd say you're doing the right thing - consolidate the fields you can, see if there are any you can remove completely, and remove searchers for fields that don't really need them.  With 100 fields changed, I'd expect to see some improvements in some areas as a result.

[Emre Toptancı _OBSS_]

I'll see what I can do. Thanks...

[Jens Kisters //SeibertSolutions]

@Emre Toptancı _OBSS_ : Currently thinking about the same idea.

I am curious to know:
What experiences did you make following your discussion with Nic here?

[Emre Toptancı _OBSS_]

Hello Jens,

Following this discussion me and my team went through all custom fields on the aforementioned customer system and configured them to have localized contexts rather than global contexts. After that we set search indexers to None for a lot of fields in the system which were being used as form fields and were never used for JQL searches.

This overall configuration effort caused a very very major performance improvement on the system. On the other hand it is really hard to say what percentage of this improvement is caused by configuring contexts and what percentage is caused by field indexers. I should also note that the rate of improvement also highly dependent on issue count since each field is indexed for each issue.

Finally, I should also note that during the time of this discussion our customer was using Jira 6.x. Atlassian made many performance improvements over time. I still expect a similar configuration work on a system with latest Jira version to cause major performance improvements but the actual improvement rate probably have changed.

[Jens Kisters //SeibertSolutions]

Emre, thanks a lot for your valuable feedback,

i will try this out on our clients Jira 7.13 Instance.