I posted a question couple of minutes back and am contining here since i am unable to access that post.
Please find the threaddump attached. I am unable to attach the complete dump but i found nearly 150 threads blocked. (dump1.txt)
I don't think you're going to get a correct answer based on the information in your question.
But, if you install the java melody plugin, it will show you all the database connections, along with stacktraces from what created the connection. Using this, you can find out which plugin (it probably is a plugin) is creating them.
Please add your comments to this post. I am unable to open the actual post(https://answers.atlassian.com/questions/52523/connection-pool-exceptions) when i click the link on your comments email. This is another issue i posted to atlassian. Melody plugin is already installed and I am analysing it and so far no luck yet. I cant attach here because it is huge.
I am using Jira 4.4.5 with oracle database.
1) 150 threads and 300 connections ? This is wrong. N connections < N threads (one thread should use at most one connection).
2) From your partial thread dump, I see that the threads are blocked on the monitor
waiting to lock <0x000000059a5bf828> (a org.apache.commons.dbcp.PoolableConnectionFactory) on validate.
2.a) you need to look in your thread dump for the string: locked 0x000000059a5bf828 (same address as above). Please
show us that thread, this is the one that is blocking everything
2.b) you need to look a bit over the configuration of the DBCP datasource. Are you using validation? testWhileIdle ? Would
be helpful to show us the configuration, too
2.c) Disable any 3rd party plugins before doing these tests.
That's a good point... I found using a validation query made things massively slower (on oracle at any rate). The validation query is run before every single normal query. As Radu says 300 connections seems way too much... I manage a jira site with 10k users and the connection pool is 30.
HI Radu/Jamie, Please find the db configuration below.
<?xml version="1.0" encoding="UTF-8"?>
<jira-database-config> <name>defaultDS</name> <delegator-name>default</delegator-name> <database-type>oracle10g</database-type> <jdbc-datasource> <url>jdbc:oracle:thin:@xxxxxxxxxxxxxx:1571:JIRAPROD</url> <driver-class>net.bull.javamelody.JdbcDriver</driver-class> <connection-properties>driver=oracle.jdbc.OracleDriver</connection-properties> <username>jira44app</username> <password>jira44app</password> <removeAbandoned>true</removeAbandoned> <removeAbandonedTimeout>600</removeAbandonedTimeout> <pool-size>300</pool-size> <connection-properties>SetBigStringTryClob=true</connection-properties> <validation-query>select 1 from dual</validation-query> </jdbc-datasource> </jira-database-config>
In the absence of the thread dump, here is a wild educated guess:
1) 300 connections are way too much. You may have messages on ORCL side that it is in a kind of limbo.
2) Remove abandoned has something to do with your deadlock (remove abandoned is called AFAIK from another thread that the one that aquired the connection). The ORCL drivers are sensitive to these kinds of manipulation.
1) Delete these lines
3) Restart Jira & retest
Really, I do not think you have a deadlock. The behavior is just that Jira is waiting too much to create a connection.
You need to look at the following:
a) JDBC driver (oracle has released some really bad drivers lately)
b) sqlnet.log (or whatever) for the ORCL listener (maybe authentication takes a long time, maybe there are not configured enough sockets on accept, etc)
c) network between your Jira server and ORCL server. Does "telnet orclhost 1571" responds in no time ?
Connect with like-minded Atlassian users at free events near you!Find a group
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no AUG chapters near you at the moment.Start an AUG