[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928371#comment-16928371
]
ASF subversion and git services commented on HTTPCORE-596:
--
Commit 478e1cf6723475f546a0a68e6aee149da3b345c2 in httpcomponents-core's branch
refs/heads/master from Oleg Kalnichevski
[ https://gitbox.apache.org/repos/asf?p=httpcomponents-core.git;h=478e1cf ]
HTTPCORE-596: connection pools to use an optional callback to close out
disposed connections
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(AbstractConnPool
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928355#comment-16928355
]
ASF subversion and git services commented on HTTPCORE-596:
--
Commit 478e1cf6723475f546a0a68e6aee149da3b345c2 in httpcomponents-core's branch
refs/heads/HTTPCORE-596 from Oleg Kalnichevski
[ https://gitbox.apache.org/repos/asf?p=httpcomponents-core.git;h=478e1cf ]
HTTPCORE-596: connection pools to use an optional callback to close out
disposed connections
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(AbstractCo
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925842#comment-16925842
]
Oleg Kalnichevski commented on HTTPCORE-596:
[~b.eckenfels]
bq. But the callback you added to 5.0 seems to allow the same logic.
It should also allow disposal by a dedicated thread without holding the pool
lock like you proposed.
bq. I noticed however that the CloseMode from the discard method is not
propagated.
Corrected. Please review.
Oleg
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(AbstractConnPool.java:69)}}
>
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925840#comment-16925840
]
ASF subversion and git services commented on HTTPCORE-596:
--
Commit eb9d7c4dc6886ce7bbcaaa3af6c8d54c4c22a0ee in httpcomponents-core's branch
refs/heads/HTTPCORE-596 from Oleg Kalnichevski
[ https://gitbox.apache.org/repos/asf?p=httpcomponents-core.git;h=eb9d7c4 ]
HTTPCORE-596: connection pools to use an optional callback to close out
disposed connections
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(AbstractCo
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925736#comment-16925736
]
Bernd Eckenfels commented on HTTPCORE-596:
--
I am waiting for the results of our Hotfix in production (after all it was not
clear why the timeout was not honored). But the callback you added to 5.0 seems
to allow the same logic. I noticed however that the CloseMode from the discard
method is not propagated.
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(AbstractConnPool.java:69)}}
> \{{ at
> org.apache.http.pool.AbstractConnP
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925698#comment-16925698
]
ASF subversion and git services commented on HTTPCORE-596:
--
Commit b959fda7f39b33fcd53fc6866b295607265b7f55 in httpcomponents-core's branch
refs/heads/HTTPCORE-596 from Oleg Kalnichevski
[ https://gitbox.apache.org/repos/asf?p=httpcomponents-core.git;h=b959fda ]
HTTPCORE-596: connection pools to use an optional callback to close out
disposed connections
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(AbstractCo
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924827#comment-16924827
]
Oleg Kalnichevski commented on HTTPCORE-596:
[~b.eckenfels] [~sartissohn]
In HttpCore master (5.0) I have added an optional connection disposal callback
that could be used by the caller to implement custom connection disposal logic,
for instance, delegating connection shutdown execution to a background thread.
Please review and let me know if that addresses the issue for you.
https://github.com/apache/httpcomponents-core/commit/c9cf7c6863197e20bbea7fbff45cecbf88e29492
Oleg
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEnt
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924826#comment-16924826
]
ASF subversion and git services commented on HTTPCORE-596:
--
Commit c9cf7c6863197e20bbea7fbff45cecbf88e29492 in httpcomponents-core's branch
refs/heads/HTTPCORE-596 from Oleg Kalnichevski
[ https://gitbox.apache.org/repos/asf?p=httpcomponents-core.git;h=c9cf7c6 ]
HTTPCORE-596: connection pools to use an optional callback to close out
disposed connections
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(AbstractCo
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924791#comment-16924791
]
ASF subversion and git services commented on HTTPCORE-596:
--
Commit af8feefb43806f7d77e0a4b01ab288b1dec21cf4 in httpcomponents-core's branch
refs/heads/4.4.x from Oleg Kalnichevski
[ https://gitbox.apache.org/repos/asf?p=httpcomponents-core.git;h=af8feef ]
HTTPCORE-596: connection pools to reduce socket timeout to 1 second when
closing managed connections
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(AbstractC
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924032#comment-16924032
]
Oleg Kalnichevski commented on HTTPCORE-596:
[~b.eckenfels] [~sartissohn] How about this change-set for the stable 4.5.x
branch?
https://github.com/apache/httpcomponents-core/commit/3964b1254f0f18223ec5dfa6a7ce92e2af6fb24d
You are very welcome to propose something better for 5.0 or 4.6.
Oleg
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(AbstractConnPool.java:69)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$2.get(A
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924028#comment-16924028
]
ASF subversion and git services commented on HTTPCORE-596:
--
Commit 3964b1254f0f18223ec5dfa6a7ce92e2af6fb24d in httpcomponents-core's branch
refs/heads/HTTPCORE-596 from Oleg Kalnichevski
[ https://gitbox.apache.org/repos/asf?p=httpcomponents-core.git;h=3964b12 ]
HTTPCORE-596: connection pools to reduce socket timeout to 1 second when
closing managed connections
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(Ab
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923666#comment-16923666
]
Oleg Kalnichevski commented on HTTPCORE-596:
[~b.eckenfels]
bq. I wonder if such a strictnes is needed asuming the connections which do not
close properly are broken already.
Most likely not, it would be bad to break the contract in a bug fix release,
though we are guilty of having done so in the past.
bq. Maybe it could be a option for the strict pool to have a size limited
blocking "close queue". The current behavior would be size=0.
We have already sneaked way too many features into 4.5.x without doing the
proper ALPHA/BETA/GA cycle. We could do in a new 4.6.x branch but I strongly
suspect 5.0 may go GA before 4.6.
What I certainly can do now is to automatically set a lower socket timeout for
`CLOSE_NOTIFY` handshake.
Oleg
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSy
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923516#comment-16923516
]
Bernd Eckenfels commented on HTTPCORE-596:
--
Ah I see. If the connection would be counted as still in use/open for the
purpose of the max limit it should be fine if the lock is released meanwhile
(but I guess tricky to implement). In the specific case the block is not
because the limit is reached but because the whole pool is blocked.
I wonder if such a strictnes is needed asuming the connections which do not
close properly are broken already. Maybe it could be a option for the strict
pool to have a size limited blocking "close queue". The current behavior would
be size=0.
Hm, maybe multiple pools would help in our case. HC5 will take longer to get
into production...
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.Reent
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923497#comment-16923497
]
Oleg Kalnichevski commented on HTTPCORE-596:
[~b.eckenfels]
bq. is there a reason why the close happens inside the critical section?
This all happens inside a critical section in order to honor strict max limit
guarantees. With some connections still kept in the queue the total and per
route max limits can be exceeded while queued connections are being shut down.
If do not care about max limit guarantees you might be better served by the lax
connection pool in 5.0 but I am open to suggestions for 4.5.x as well.
bq. Also, in the above loop do you know where it is aborted after timeout?
Presently connections do not get aborted after a timeout. This would need to be
added.
Oleg
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.con
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923483#comment-16923483
]
Bernd Eckenfels commented on HTTPCORE-596:
--
Oleg, is there a reason why the close happens inside the critical section? Even
when the sockets are well behaved that looks like a scalability issue. I
remeber that I had exactly the same issue in a homegrown db pool and collecting
the candidates for close in a list and processing it in a background
thread(pool) after lock is released is good for scalability.
Also, in the above loop do you know where it is aborted after timeout?
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlo
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923466#comment-16923466
]
Oleg Kalnichevski commented on HTTPCORE-596:
[~sartissohn] It certainly must.
You have two options
# Upgrade to HC 5.0 and use new lax (concurrent) pool implementation that does
not have a global pool lock.
# Try reducing the socket timeout on those connections
For 4.5.x and the strict pool implementation in 5.0 we could automatically
reduce the socket timeout and hard reset slow connections .
Oleg
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923460#comment-16923460
]
Thomas Sartissohn commented on HTTPCORE-596:
Just checked the SSLSocketImpl. Not sure if socketTimeout may help here. Not
that easy to test as it is hard to simulate the package drop on close.
{{while (((state = getConnectionState()) != cs_CLOSED) &&}}
{{ (state != cs_ERROR) && (state != cs_APP_CLOSED)) {}}
{{ // create the InputRecord if it isn't initialized.}}
{{ if (inrec == null) {}}
{{ inrec = new InputRecord();}}
{{ }}}{{// Ask for app data and then throw it away}}
{{ try {}}
{{ readRecord(inrec, true);}}
{{ } catch (SocketTimeoutException e) {}}
{{ // if time out, ignore the exception and continue}}
{{ }}}
{{ }}}
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
>
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923426#comment-16923426
]
Oleg Kalnichevski commented on HTTPCORE-596:
[~sartissohn] I am not sure I see how it can take up to 15 minutes to close the
connection even if it involves a `CLOSE_NOTIFY` handshake. Could you please
double-check?
Oleg
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(AbstractConnPool.java:69)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java:245)}}
> \{{ - locked org.apache.http.pool.AbstractC
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923425#comment-16923425
]
Thomas Sartissohn commented on HTTPCORE-596:
The timeout is configured to 3 minutes.
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(AbstractConnPool.java:69)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java:245)}}
> \{{ - locked org.apache.http.pool.AbstractConnPool$2@1e25f993}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java:193)}}
> \{{ at
> org.apache.http.im
[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool
[
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923403#comment-16923403
]
Oleg Kalnichevski commented on HTTPCORE-596:
[~sartissohn] What is the socket timeout setting of those connections?
Oleg
> Closing idle connections may end up in a blocked connection pool
>
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
> \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
> \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
> \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
> \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
> \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
> \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
> \{{ - locked java.lang.Object@7b9c4e14}}
> \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
> \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
> \{{ at
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
> \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
> \{{ at
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
> \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
> \{{ at
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
> \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
> \{{ at
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
> \{{ at
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
> \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
> \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
> \{{ at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
> }}
>
> While the thread is waiting here for the close the whole connection pool is
> blocked due to some lock in the enumAvailble method. Any try to lease a new
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to
> that specific route the close is pending. Finally my whole application is
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
> \{{ at sun.misc.Unsafe.park(Native Method)}}
> \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
> \{{ at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
> \{{ at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool.access$200(AbstractConnPool.java:69)}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java:245)}}
> \{{ - locked org.apache.http.pool.AbstractConnPool$2@1e25f993}}
> \{{ at
> org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java:1
