On 07/09/18 22:56, Bruce Milner wrote: > Hello, > > I am using commons-pool2-2.5.0 for a MySQL connection pooler. The reason > for not using out-of-the-box is that the existing code relies on > changing catalogs at runtime reusing an existing connection. The > original design was to use multiple databases using the same connection > and this cannot be changed. > > I recently replaced a lot of hand crafted code with the commons-pool2 > implementation. > > The issue I have is that one server I manage went into a state where > there are plenty of connections, but none are being returned to the > pool. They are all stuck on a lock inside of > GenericKeyedObjectPool.returnObject. > > The config is basically > GenericKeyedObjectPoolConfig config = new > GenericKeyedObjectPoolConfig(); > config.setBlockWhenExhausted(true); > config.setMaxTotal(120); > config.setMaxTotalPerKey(60); > config.setTestOnBorrow(true); > config.setTimeBetweenEvictionRunsMillis(60000); > config.setMinEvictableIdleTimeMillis(0); // don't starve > connections because of catalog switches. > /** > * For database connections, use FIFO so that we get rid of > older connections first before newer ones. > */ > config.setLifo(false); > return new GenericKeyedObjectPool<String, PooledConnection>(new > PooledConnectionFactory(), config); > > There are 150 of these threads waiting on a lock to release connections > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x00000006471cd7d8> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199) > > at > java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209) > > at > java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285) > at > org.apache.commons.pool2.impl.LinkedBlockingDeque.hasTakeWaiters(LinkedBlockingDeque.java:1389) > > at > org.apache.commons.pool2.impl.GenericKeyedObjectPool.hasBorrowWaiters(GenericKeyedObjectPool.java:849) > > at > org.apache.commons.pool2.impl.GenericKeyedObjectPool.returnObject(GenericKeyedObjectPool.java:551) > > at > com.ilrn.util.sql.connectionpooler.ConnectionPooler.releaseConnection(ConnectionPooler.java:358) > > at > com.ilrn.util.sql.connectionpooler.PooledConnection.close(PooledConnection.java:141) > > at > com.ilrn.util.sql.connectionpooler.ConnectionPooler.safeClose(ConnectionPooler.java:480) > > > and 158 of these threads waiting to open connections. > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x00000006471cd7d8> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199) > > at > java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209) > > at > java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285) > at > org.apache.commons.pool2.impl.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:560) > > at > org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:356) > > at > org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:281) > > at > com.ilrn.util.sql.connectionpooler.ConnectionPooler.getConnection(ConnectionPooler.java:197) > > at com.ilrn.util.sql.Database.getConnection(Database.java:1273) > > For the "- parking to wait for <0x00000006471cd7d8> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync)" there are 155. > Though I see no other messages with that object in the thread dump. > > Has anyone run into this? It seems like some sort of deadlock.
Do you still have the full thread dump? Can you post it somewhere (where we can look at it)? Mark --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
