Problem Re-Indexing with Groovy Runner plugin

I had an index fail with NoClassDefFoundError - this then meant that all Jira functionality was lost (couldn't search for or create issues, admin tools all messed up)

Stopping Jira, deleting indexes and removing plugins from installedplugins and then re-starting overcame and I gradually added plugins re-indexing as I went and re-index went fine at each stage

I upgraded Groovy Runner from 2.1 Beta 7 to 2.1 Beta 8 and I didn't get the NoClassDefFoundError but the re-index did fail, this time with Too Many Open Files

I upped ullimit to 4096 and it now re-indexes successfully

I don't have that many Groovy Files (1 scripted field is a file and a few workflow transitions have groovy files) and this is a test system with relatively low volume of issues so I'm reticent to move it to production until I overcome the issue.

Any ideas why it might be keeping too many files open (or indeed why the first time failed with NoClassDefFoundError and not the 2nd when I didn't change any scripted files or script transitions/fields)?

6 answers

1 accepted

Was resolved by https://jamieechlin.atlassian.net/browse/GRV-211 apologies for the late answer

0 votes

> NoClassDefFoundError and not the 2nd when I didn't change any scripted files or script transitions/fields

Which class was not found?

No... sounds improbable that nothing changed but you got different results. Have you got the log files?

> I upped ullimit to 4096 and it now re-indexes successfully

4096 is really very low... the ulimit for open files is probably not going to be related to the volume of issues.

Hi Andy,

> it would need to open a file for each issue that has a scripted field that has groovy file?

Possibly... although it would also close it. Seems related to https://jamieechlin.atlassian.net/browse/GRV-206 - maybe you could watch that issue.

Hi Jamie

Thanks for the super quick answer - the only change between the 2 times was upping the version of the groovy plugin.

It didn't specify the class that it couldn't find

I upped ulimit from 1024 to 4096 as outlined as it

https://confluence.atlassian.com/display/JIRAKB/Loss+of+Functionality+due+to+Too+Many+Open+Files+Error

I'm not sure how the indexing works and whether it would need to open a file for each issue that has a scripted field that has groovy file?

I do have log files and screenshots - what would be the best way to get them to you?

Cheers

Andy

Would I be better changing to inline groovy for the custom fields and transitions rather than opening the file?

Here is a snippet of the log from the first failure the NoClassDef.. so it appeared then that it failed to find the class for the Index failure.. but the original error was the same too many files open - I suspect it must have failed to read from the class path due to too many files open issue

Exception in thread "ISSUE-indexQueue:thread-1" java.lang.NoClassDefFoundError: com/atlassian/jira/index/DefaultIndex$Failure

at com.atlassian.jira.index.DefaultIndex.perform(DefaultIndex.java:45)

at com.atlassian.jira.index.QueueingIndex$Task.index(QueueingIndex.java:144)

at com.atlassian.jira.index.QueueingIndex$Task.run(QueueingIndex.java:125)

at java.lang.Thread.run(Unknown Source)

...

at java.lang.Thread.run(Unknown Source)

Mar 8, 2013 1:09:22 PM org.apache.tomcat.util.net.JIoEndpoint$Acceptor run

SEVERE: Socket accept failed

java.net.SocketException: Too many open files

at java.net.PlainSocketImpl.socketAccept(Native Method)

at java.net.PlainSocketImpl.accept(Unknown Source)

at java.net.ServerSocket.implAccept(Unknown Source)

at java.net.ServerSocket.accept(Unknown Source)

at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:60)

Hi - so it seems GRV-206 is not related to my problem but to the groovy version.

I've unearthed something deeper in the log - there was a nullpointer being thrown on one of my equalsIgnoreCase checks which was occuring on a null object

Perhaps the script runner is not closing files when hitting exceptions like this and that is why it is now hitting the Too many open files issue?

Shall I create a GRV issue so I can upload logs etc?

No, it will be not be leaving files open. Why don't you see what files are open with lsof? You can create an issue, sure, but I'm not seeing a real problem yet.

OK I now have it narrowed down to one scripted field that uses a file - if I delete that field it indexes fine, with the field there it dies with too many files open

It could just be that the script is taking a while to pull results so that during indexing more and more files get open

I'll raise an issue and send you the lsof output and further details

is this resolved ?? I am also facing the same issue.

Script Runner version :: 2.1.9 .. index goes corrupt after adding the scripted fields into the project.

when a null is returned as an output of the issue on which the scripted field are loaded.

0 votes

Are you sure that's the reason - can you test with a script that just returns null?

What indexer are you using?

We have the same problem here - we had it before and it still exists with the upgraded versions of JIRA and Script Runner. We are now at JIRA 6.3.7, and Script Runner 3.0.6.

If we keep Script Runner enabled, and perform a Reindex of JIRA (full or background), the indexes become corrupted. The problem is solved by disabling Script Runner and relaunching the indexing sad

I am surprised that we are just a few to report this problem, even though it has been existing with our company for a while. We are keeping the plugin disabled, and we're enabling it only when we need to Copy a project in the Built In scripts section...

I was wondering if there is a way to disable indexing Custom Scripted Field made by Script Runner? In our environment we are creating html content. Apparently, when we remove groovy script and add "return '' " indexing works fine but when complete script is generating html index is corrupt.

to disable indexing just set the custom field to have no indexer.

Thank you very much for your tip! Accept my apologies for not thinking and rushing with question.

no apology necessary, but thanks.

Thank you for posting the solution on how to disable the indexer! I was really in troubles because the index couldn't run through with script runner enabled. We have two html customfields. After editing the customfiels to have "no search template" it all works fine again.

Suggest an answer

Log in or Join to answer
Community showcase
Sarah Schuster
Posted Jan 29, 2018 in Jira

What are common themes you've seen across successful & failed Jira Software implementations?

Hey everyone! My name is Sarah Schuster, and I'm a Customer Success Manager in Atlassian specializing in Jira Software Cloud. Over the next few weeks I will be posting discussion topics (8 total) to ...

2,944 views 12 18
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you
Atlassian Team Tour

Join us on the Team Tour

We're bringing product updates and pro tips on teamwork to ten cities around the world.

Save your spot