JIRA Freezes with DB deadlock

Devu Heda
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
April 20, 2012

Hi recently moved to JIRA 446 and JIRA freezes because of the following deadlocks ...one another observation ...JIRA 402 worked with 100 Active DB Instances in our system and now we had to bump it 300 as we were running out of DB Connections leading me to beleive JIRA is having issues with DB Connection Pool handling

Any thoughts around this and how this can be resolved ... working with support as well ...but wanted to reach out to the community

Thanks

{code}

"StreamsCompletionService::thread-2232" prio=10 tid=0x00002aaabca3b000 nid=0x756c waiting for monitor entry [0x0000000059240000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:300)
- waiting to lock <0x000000058001b568> (a org.apache.commons.dbcp.PoolableConnectionFactory)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:816)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:140)
at org.ofbiz.core.entity.transaction.DBCPConnectionFactory$1.call(DBCPConnectionFactory.java:153)
at org.ofbiz.core.entity.transaction.DBCPConnectionFactory$1.call(DBCPConnectionFactory.java:150)
at org.ofbiz.core.entity.jdbc.interceptors.connection.ConnectionTracker.trackConnection(ConnectionTracker.java:53)
at org.ofbiz.core.entity.transaction.DBCPConnectionFactory.trackConnection(DBCPConnectionFactory.java:149)
at org.ofbiz.core.entity.transaction.DBCPConnectionFactory.getConnection(DBCPConnectionFactory.java:69)
at org.ofbiz.core.entity.ConnectionFactory.tryGenericConnectionSources(ConnectionFactory.java:69)
at org.ofbiz.core.entity.transaction.JNDIFactory.getConnection(JNDIFactory.java:146)
at org.ofbiz.core.entity.TransactionFactory.getConnection(TransactionFactory.java:101)
at org.ofbiz.core.entity.ConnectionFactory.getConnection(ConnectionFactory.java:59)
at org.ofbiz.core.entity.jdbc.SQLProcessor.getConnection(SQLProcessor.java:367)
at org.ofbiz.core.entity.jdbc.SQLProcessor.prepareStatement(SQLProcessor.java:515)
at org.ofbiz.core.entity.GenericDAO.selectListIteratorByCondition(GenericDAO.java:1034)
at org.ofbiz.core.entity.GenericHelperDAO.findListIteratorByCondition(GenericHelperDAO.java:175)
at org.ofbiz.core.entity.GenericDelegator.findListIteratorByCondition(GenericDelegator.java:1001)
at sun.reflect.GeneratedMethodAccessor799.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.atlassian.multitenant.impl.MultiTenantComponentFactoryImpl$AbstractMultiTenantAwareInvocationHandler.invokeInternal(MultiTenantComponentFactoryImpl.java:181)
at com.atlassian.multitenant.impl.MultiTenantComponentFactoryImpl$MultiTenantAwareMethodInterceptor.intercept(MultiTenantComponentFactoryImpl.java:230)
at org.ofbiz.core.entity.GenericDelegator$$EnhancerByCGLIB$$7d4fdd03.findListIteratorByCondition(<generated>)
at com.atlassian.jira.ofbiz.DefaultOfBizDelegator.findListIteratorByCondition(DefaultOfBizDelegator.java:464)
at com.atlassian.jira.issue.changehistory.DefaultChangeHistoryManager.doFindUserHistory(DefaultChangeHistoryManager.java:345)
at com.atlassian.jira.issue.changehistory.DefaultChangeHistoryManager.findUserHistory(DefaultChangeHistoryManager.java:259)
at sun.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

{code}

2 answers

1 accepted

1 vote
Answer accepted
Devu Heda
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
May 12, 2012

Thanks to Jamie...the java melody plugin is awesome ... we found it was issue with the HTTP threads and not actually DB threads ...

As suggested by Jamie we also reduced the number of threads

Thanks!!

1 vote
JamieA
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
April 20, 2012

300 is too many, 100 is too many... you have a plugin that is leaking database connections.

If you install the java melody plugin, it will show you all the database connections, along with stacktraces from what created the connection. Using this, you can find out which plugin (it probably is a plugin) is creating them.

Devu Heda
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
April 21, 2012

Thanks Jamie,

I have installed the plugin to try out on my dev instance ....any idea what are the log4j properties to configure tracing for this plugin

Suggest an answer

Log in or Sign up to answer