Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

JIRA 8.13 indexing problem

Recently we have updated our JIRA from version 7.9 to 8.13 , All thing were good until we had to re index JIRA , our architecture is base on three node in cluster mode . after full re index jira( we chose this way because our JIRA server provides thousand issues per day). .Although we haven't seen any error during and after re index , tomorrow after jira server were under the load we have seen some strange error and later on cpu has been used lots of by jira also memory has been used full and got use swap also some time it get full gc problem we monitored logs and those logs indicated we have problem in indexing document ,subsequently we decided to full re-index again but despite indexing has completeed successfully we saw some error implies that in re index process skipped from some document:

Structure-Jobs60b18591 Queue-Thread#137213 WARN anonymous     [c.a.j.structure.cluster.ExclusiveClusterJob] Full reindex is in progress (Structure cannot lock JIRA against full reindex) or index is inconsistent with the database. "ApplyJob" is skipped this time

Here is some logs have been appeared as below when jira server starting to work after reindex :

2021-06-22 10:53:36,092+0430 Structure-Jobs60b18591 Generator-Thread#136762 INFO a.sharifi 595x47427692x2 1jlsbdh 10.20.xx.xx,10.xxx.0.xx8 /rest/structure/2.0/poll [c.a.
jira.index.MonitoringIndexWriter] [lucene-stats] flush stats: snapshotCount=298, totalCount=233028, periodSec=302, flushIntervalMillis=1016, snapshotNoDocs=0, totalNoDo
cs=0, indexDirectory=/opt/atlassian/application-data/jira/caches/indexesV1/issues, indexWriterId=com.atlassian.jira.index.MonitoringIndexWriter@3fbf80fa, indexDirectory
Id=MMapDirectory@/opt/atlassian/application-data/jira/caches/indexesV1/issues lockFactory=org.apache.lucene.store.NativeFSLockFactory@1665dd9e
--------------------------------------------------------------------------------------------------------------------2021-06-22 11:43:17,594+0430 Structure-Jobs60b18591 Generator-Thread#136758 WARN s--aleki 595x47426680x2 1isjx1g 10.xx0.151.xx,10.1xx.0.xx /rest/structure/2.0/poll [c.a
.j.s.forest.gfs.AbstractTransformingForestSource] SecuredForestSource(a660e59b)(GeneratingForestSource(eea658af)(null)({"structureId":10965})) cannot acquire lock in 30
000ms

2 answers

@Saleh Houshangi - Structure is a heavy plugin - consumes more system resources and can cause performance issues. I would recommend that you

  1. Disable structure for a day and monitor the performance.
  2. Take a node out of your load balancer and reindex it.
  3. After completion of indexing on one node, copy the indexes to other nodes.
  4. Analyse GC logs and work with the plugin vendor (Structure) and make changes as requried. You may need to follow the best practices that the plugin vendor may recommend. 

Hello @Saleh Houshangi

David from ALM Works here.  We will be happy to help review the logs and work with you to identify the cause of your re-index error.  

To facilitate this, it will be best to work directly through our support channel.   Please send us a quick email to support@almworks.com. 

I look forward to working with you!

Best Regards,
David Niro
www.almworks.com

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Trello

🏊‍♀️ Atlympic Event: Trello

Hello Community!  Quick disclaimer: We are running a contest on Community (The Atlympics!) from July 23rd - August 8th of 2021. If you are interested in participating in this contest (prizes! ...

109 views 1 11
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you