Jira suddenly went slow and unresponsive for about an hour

Hi

Last night our production instance of jira suddenly went extremely slow and unresponsive for about an hour. After about an hour and a half things went back to normal.

Can anyone please help me. I have attached the atlassian-jira and UI error logs.

On UI we got

com.atlassian.jira.exception.DataAccessException: org.ofbiz.core.entity.GenericEntityException: while updating: [GenericEntity:Issue][summary,Build_3[XXXXXX]: .][status,1][votes,0][fixfor,null][assignee,aaaaaaa][security,null][resolution,null][resolutiondate,null][type,1][timespent,null][id,7735][reporter,eli77][project,1052][environment,Platform: TCH73 FGR Box Build: Build 25 Build Type: s_tf][component,null][updated,2013-01-15 12:58:20.0][created,2013-01-15 12:57:31.0][workflowId,959][description,Steps to reproduce: <DELETED ON PURPOSE> (CL# 338027)][priority,2][timeestimate,null][duedate,null][timeoriginalestimate,null][key,TFE2E-825] (SQL Exception while executing the following:UPDATE jiraissue SET pkey=?, PROJECT=?, REPORTER=?, ASSIGNEE=?, issuetype=?, SUMMARY=?, DESCRIPTION=?, ENVIRONMENT=?, PRIORITY=?, RESOLUTION=?, issuestatus=?, CREATED=?, UPDATED=?, DUEDATE=?, RESOLUTIONDATE=?, VOTES=?, TIMEORIGINALESTIMATE=?, TIMEESTIMATE=?, TIMESPENT=?, WORKFLOW_ID=?, SECURITY=?, FIXFOR=?, COMPONENT=? WHERE ID=? (Lock wait timeout exceeded; try restarting transaction))

On atlassian-jira.log, i have about 15k lines of repeating errors at the same instant (22:13:57 Hrs). Here are the unique lines.


2013-01-15 22:13:57,216 http-443-19 ERROR lxxxxxx 8x32075x3 1a6ljxd 172.22.6.81 /secure/WorkflowUIDispatcher.jspa [com.opensymphony.workflow.AbstractWorkflow] Error checking available actions
com.opensymphony.workflow.StoreException: Could not find workflow instance #95670: root cause: SQL Exception while executing the following:SELECT NAME, INITIALIZED, STATE FROM OS_WFENTRY WHERE ID=? (The last packet successfully received from the server was 79,505,636 milliseconds ago. The last packet sent successfully to the server was 79,505,636 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.)

2013-01-15 22:13:57,232 http-443-20 ERROR cstev 514x50962x7 1vo5lhl 10.4.42.84 /secure/CommentAssignIssue.jspa [atlassian.jira.workflow.SimpleWorkflowManager] Error occurred while rolling back transaction.
org.ofbiz.core.entity.GenericTransactionException: Error occurred while rolling back transaction. (No operations allowed after connection closed.Connection was implicitly closed by the driver.)

2013-01-15 22:13:57,420 http-443-14 ERROR xcwa 100x37714x6 wox3k6 172.22.7.0 /secure/WorkflowUIDispatcher.jspa [webwork.util.ValueStack] query="locale" {[id="locale" type="8" values=""]}
java.lang.reflect.InvocationTargetException

2013-01-15 22:13:57,373 http-443-20 ERROR cstev 514x50962x7 1vo5lhl 10.4.42.84 /secure/CommentAssignIssue.jspa [webwork.util.ValueStack] METHOD: "unescapedText", exception:
java.lang.IllegalStateException: getAttribute: Session already invalidated

2013-01-15 22:13:57,310 http-443-20 ERROR cstev 514x50962x7 1vo5lhl 10.4.42.84 /secure/CommentAssignIssue.jspa [velocity] Method getText threw exception for reference $i18n in template templates/jira/issue/field/attachment-edit.vm at [50,21]

2013-01-15 22:13:57,310 http-443-20 ERROR cstev 514x50962x7 1vo5lhl 10.4.42.84 /secure/CommentAssignIssue.jspa [velocity] VMProxyArg.getObject() : method invocation error getting value : org.apache.velocity.exception.MethodInvocationException: Invocation of method 'getText' in class com.atlassian.jira.web.action.issue.CommentAssignIssue threw exception class java.lang.IllegalStateException : getAttribute: Session already invalidated

2013-01-15 22:13:57,310 http-443-20 ERROR cstev 514x50962x7 1vo5lhl 10.4.42.84 /secure/CommentAssignIssue.jspa [com.atlassian.velocity.DefaultVelocityManager] MethodInvocationException occurred getting message body from Velocity: java.lang.IllegalStateException: getAttribute: Session already invalidated

013-01-15 22:13:57,310 http-443-20 ERROR cstev 514x50962x7 1vo5lhl 10.4.42.84 /secure/CommentAssignIssue.jspa [velocity] Method getText threw exception for reference $i18n in template templates/jira/issue/field/resolution-edit.vm at [1,385]
2013-01-15 22:13:57,295 http-443-18 ERROR xcwang 92x36934x7 wox3k6 172.22.7.0 /secure/WorkflowUIDispatcher.jspa [webwork.util.ValueStack] query="locale" {[id="locale" type="8" values=""]}
java.lang.reflect.InvocationTargetException

2013-01-15 22:13:57,279 http-443-20 ERROR cstev 514x50962x7 1vo5lhl 10.4.42.84 /secure/CommentAssignIssue.jspa [webwork.util.ValueStack] query="searchRequest" {[id="searchRequest" type="8" values=""]}
java.lang.reflect.InvocationTargetException

3 answers

1 accepted

A bad day for you, huh?

Check out https://confluence.atlassian.com/display/BAMKB/MySQL+-+Lock+wait+timeout+exceeded+-+try+restarting+transaction

There are some stuff in that link (link is for Bamboo, works for JIRA as well). Also need to check if users are selecting huge volume of data? How much is set as your query limit jira.search.views.max.limit?

Sounds like a database issue; have you contacted your local DBA to see if anything happened to the database around that time? Based on your information that it lasted for approximately an hour, I am guessing your database went down for a backup or some sort of maintenance that was preventing connections.

Looks like DB error. Did you talk to your DB guys ? is there any issue with DataBase ?

Suggest an answer

Log in or Sign up to answer
How to earn badges on the Atlassian Community

How to earn badges on the Atlassian Community

Badges are a great way to show off community activity, whether you’re a newbie or a Champion.

Learn more
Community showcase
Published Monday in Jira Software

How large do you think Jira Software can grow?

Hi Atlassian Community! My name is Shana, and I’m on the Jira Software team. One of the many reasons this Community exists is to connect you to others on similar product journeys or with comparabl...

565 views 6 12
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you