All,
I am having a weird problem with an app. We were having concurrency issues,
so we setup some simple pessimistic locking - we lock a single table using
select for update. We have developed a jmeter script that allows us to test
the app when loaded. Repeatedly after different intervals, but less than say
2 mins, the app locks up. By locks up, any attempt to access a webpage that
connects to the database doesn't respond.
When looking in the database, I can see several threads waiting to lock the
row, and one thread having the lock. Thing is that thread with the lock is
inactive as far as the database is concerned. So what I deduce is that a
connection goes to server, obtains the lock, and then somehow swapped out
and so never commits transaction, hence lock is never released, hence issue.
What I would like to know is how to track down what is causing this. I have
tried taking stack traces - happy to share but they are long not sure if
relevant. ThreadDumpAnalyzer reports
Overall Thread Count 151
Overall Monitor Count 209
Number of threads waiting for a monitor 0
Number of threads locking a monitor 127
Number of threads sleeping on a monitor 105
Number of deadlocks 0
Number of Monitors without locking threads 0
69% of all threads are sleeping on a monitor.
This might indicate they are waiting for some external resource (e.g.
database) which is overloaded or not available or are just waiting to get to
do something (idle threads). You should check the sleeping threads with a
filter excluding all idle threads.
Connection to the db configured like this
<Resource name="jdbc/xts"
auth="Container"
type="javax.sql.DataSource"
username="username
password="password"
driverClassName="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:@localhost:1521:xe"
maxActive="20"
maxIdle="2"
minIdle="1"
initialSize="2"
/>
Windows 7 64 bit
java version "1.6.0_24"
Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode)
tomcat 7.0.11
OracleXe database
I tried to use org.apache.tomcat.jdbc.pool.DataSourceFactory - broadly same
problem happens, but it gives up after 30 seconds saying "Pool empty. Unable
to fetch a connection in 30 seconds, none available[20 in use]."
So any ideas as to what to look into next? I appreciate that this app stack
is java, tomcat, spring, hibernate, oracle. Not sure that this is a tomcat
specific issue, but hoping someone might have hit this before, or be able to
point me in the direction to go. Had a quick look through the dbcp source -
can't see how to make it log when it does things. Any pointers
Thanks for any help
Chris
