Re: Using Conscrypt provider for server uses 100% CPU after first request

[Roy Hashimoto]

With a hybrid server that negotiates HTTP1 or HTTP2, the server does *not*
spin after an HTTP2 request, only after an HTTP1 request.

Roy

On Thu, Sep 12, 2019 at 2:24 PM Roy Hashimoto 
wrote:

> In the attached HttpCore 5.0 server program (gist
> ),
> I'm using Conscrypt for the TLS provider. It works, but I noticed that
> after each request the CPU gets pegged to 100% usage.
>
> I'm using org.conscrypt:conscrypt-openjdk-uber:2.2.1 and HttpCore 5.0 from
> source at the HTTPCORE-599 fix
> .
> I see the same behavior on macOS Mojave and Debian Jessie. I'm using
> Conscrypt for ALPN on Java 8, though the minimal sample program here
> doesn't enable it .
>
> What I see in the debugger is that SingleCoreIOReactor.doExecute() is
> repeatedly called with this line
> 
>  returning readyCount=1. It looks like the selector is set to
> SelectionKey.OP_WRITE,
>
> Not sure if this is a HttpCore bug or a Conscrypt bug. I looked through
> the Conscrypt issues  but
> nothing caught my attention as related.
>
> Roy
>

Using Conscrypt provider for server uses 100% CPU after first request

[Roy Hashimoto]

In the attached HttpCore 5.0 server program (gist
), I'm
using Conscrypt for the TLS provider. It works, but I noticed that after
each request the CPU gets pegged to 100% usage.

I'm using org.conscrypt:conscrypt-openjdk-uber:2.2.1 and HttpCore 5.0 from
source at the HTTPCORE-599 fix
.
I see the same behavior on macOS Mojave and Debian Jessie. I'm using
Conscrypt for ALPN on Java 8, though the minimal sample program here
doesn't enable it .

What I see in the debugger is that SingleCoreIOReactor.doExecute() is
repeatedly called with this line

 returning readyCount=1. It looks like the selector is set to
SelectionKey.OP_WRITE,

Not sure if this is a HttpCore bug or a Conscrypt bug. I looked
through the Conscrypt
issues  but nothing caught my
attention as related.

Roy

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCORE-600) SocketTimeoutException when remote server closes connection after payload has been delivered

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928780#comment-16928780
 ] 

Oleg Kalnichevski commented on HTTPCORE-600:


[~gujunlong] It is a shame we missed 4.4.12 release by a few days. I will look 
into it.

Oleg

> SocketTimeoutException when remote server closes connection after payload has 
> been delivered
> 
>
> Key: HTTPCORE-600
> URL: https://issues.apache.org/jira/browse/HTTPCORE-600
> Project: HttpComponents HttpCore
>  Issue Type: Bug
>  Components: HttpCore NIO
>Affects Versions: 4.4.10, 4.4.11, 4.4.12
>Reporter: Brian B
>Assignee: Oleg Kalnichevski
>Priority: Major
> Attachments: asyncbugtest.zip
>
>
> Initially I created this under CXF (CXF-8113), but I think the consensus is 
> it is an NIO issue.  The description from that Jira edited to remove 
> CXF-specific changes:
> When using the Asynchronous Client HTTP Transport to handle soap web 
> services, we get a SocketTimeout exception when a remote server closes the 
> connection after the payload has been delivered.
> The problem occurs when the client receives a TLS {{close_notify}} _after_ 
> the payload has been received.  The payload must be larger than the allocated 
> byte buffer receiving the decrypted payload (~16K in the default case). This 
> results in decrypted data being available in the {{httpcore-nio}} library, 
> but not immediately consumed. 
> When these conditions are met, the following section of code is invoked:
>  
> {{org.apache.cxf.transport.http.asyncclient.SharedInputBuffer#_consumeContent_:{color:#0747a6}111{color}}}:
> {code:java}
> if (!this.buffer.hasRemaining() && this.ioctrl != null && !this.endOfStream) {
> this.ioctrl.suspendInput();
> }
> {code}
> The suspension of input when combined with the connection close and the 
> following change introduced in {{httpcore-nio}} 4.4.10 (and it's revised 
> 4.4.11 version here) truly removes the {{READ}} {{EventMask}} from the 
> underlying {{IOSessionImpl}}:
>  
> {{org.apache.http.nio.reactor.ssl.SSLIOSession#_updateEventMask_:{color:#0747a6}402{color}}}
> {code:java}
> if (this.endOfStream && (this.appBufferStatus == null || 
> !this.appBufferStatus.hasBufferedInput())) {
> newMask = newMask & ~EventMask.READ;
> }
> {code}
> Once this happens, requests for input by 
> {{SharedInputBuffer#_waitForData(int)_}} enter the {{_updateEventMask_}} 
> method as expected, but the new {{httpcore-nio}} code above prevents the read 
> operation from being reenabled despite the remainder of the decrypted payload 
> being available in the {{SSLIOSession}} {{inPlain}} buffer.  The call 
> ultimately fails with a {{SocketTimeoutException}}.
> I created a unit test to demonstrate the failure and attached it to the JIRA. 
>  Given the complexity of the code and multiple buffers at play, I have not 
> been able to come up with a fix beyond modifying the {{SSLIOSession}} code 
> above to account for the buffered decrypted content.  It seems in all cases 
> where {{appBufferStatus.hasBufferedInput()}} is queried, 
> {{inPlain.hasData()}} is also queried, with the exception of the 
> {{updateEventMask}} code above.
> Please let me know if there's anything else you need from me.
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Assigned] (HTTPCORE-600) SocketTimeoutException when remote server closes connection after payload has been delivered

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski reassigned HTTPCORE-600:
--

Assignee: Oleg Kalnichevski

> SocketTimeoutException when remote server closes connection after payload has 
> been delivered
> 
>
> Key: HTTPCORE-600
> URL: https://issues.apache.org/jira/browse/HTTPCORE-600
> Project: HttpComponents HttpCore
>  Issue Type: Bug
>  Components: HttpCore NIO
>Affects Versions: 4.4.10, 4.4.11, 4.4.12
>Reporter: Brian B
>Assignee: Oleg Kalnichevski
>Priority: Major
> Attachments: asyncbugtest.zip
>
>
> Initially I created this under CXF (CXF-8113), but I think the consensus is 
> it is an NIO issue.  The description from that Jira edited to remove 
> CXF-specific changes:
> When using the Asynchronous Client HTTP Transport to handle soap web 
> services, we get a SocketTimeout exception when a remote server closes the 
> connection after the payload has been delivered.
> The problem occurs when the client receives a TLS {{close_notify}} _after_ 
> the payload has been received.  The payload must be larger than the allocated 
> byte buffer receiving the decrypted payload (~16K in the default case). This 
> results in decrypted data being available in the {{httpcore-nio}} library, 
> but not immediately consumed. 
> When these conditions are met, the following section of code is invoked:
>  
> {{org.apache.cxf.transport.http.asyncclient.SharedInputBuffer#_consumeContent_:{color:#0747a6}111{color}}}:
> {code:java}
> if (!this.buffer.hasRemaining() && this.ioctrl != null && !this.endOfStream) {
> this.ioctrl.suspendInput();
> }
> {code}
> The suspension of input when combined with the connection close and the 
> following change introduced in {{httpcore-nio}} 4.4.10 (and it's revised 
> 4.4.11 version here) truly removes the {{READ}} {{EventMask}} from the 
> underlying {{IOSessionImpl}}:
>  
> {{org.apache.http.nio.reactor.ssl.SSLIOSession#_updateEventMask_:{color:#0747a6}402{color}}}
> {code:java}
> if (this.endOfStream && (this.appBufferStatus == null || 
> !this.appBufferStatus.hasBufferedInput())) {
> newMask = newMask & ~EventMask.READ;
> }
> {code}
> Once this happens, requests for input by 
> {{SharedInputBuffer#_waitForData(int)_}} enter the {{_updateEventMask_}} 
> method as expected, but the new {{httpcore-nio}} code above prevents the read 
> operation from being reenabled despite the remainder of the decrypted payload 
> being available in the {{SSLIOSession}} {{inPlain}} buffer.  The call 
> ultimately fails with a {{SocketTimeoutException}}.
> I created a unit test to demonstrate the failure and attached it to the JIRA. 
>  Given the complexity of the code and multiple buffers at play, I have not 
> been able to come up with a fix beyond modifying the {{SSLIOSession}} code 
> above to account for the buffered decrypted content.  It seems in all cases 
> where {{appBufferStatus.hasBufferedInput()}} is queried, 
> {{inPlain.hasData()}} is also queried, with the exception of the 
> {{updateEventMask}} code above.
> Please let me know if there's anything else you need from me.
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2015) Connect timeout value is effective doubled

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928777#comment-16928777
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2015:
---

[~francium25] It is likely the latter. Anyway connect operation in HttpClient 
is not atomic. It may involve a lot of TCP packet exchanges.

Oleg

> Connect timeout value is effective doubled
> --
>
> Key: HTTPCLIENT-2015
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2015
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.10
> Environment: Debian-based Linux
> openjdk version "1.8.0_222"
> OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1-b10)
> OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
>Reporter: Chanseok Oh
>Priority: Minor
>
> The connection times out, say, after 6 seconds when calling 
> `setConnectionTimeout(3000)`. The timeout value is effectively doubled. I've 
> provided sample code, which demonstrates that the JDK Socket times out after 
> 3 seconds, but the Apache HTTP Client times out after 6 seconds.
> This may seem like a minor bug to others, but our product functionality 
> depends on timeouts in part and allows the user to adjust the timeouts, and 
> this timeout-doubling-effect is causing some trouble to us.
> {code:java}
>   public static void main(String[] args) {
> long started = System.nanoTime();
> try (Socket socket = new Socket()) {
>   socket.connect(new InetSocketAddress("example.com", 81), 3000);
> } catch (IOException ex) {
>   ex.printStackTrace(System.out);
> } finally {
>   System.out.println("JDK elapsed (s): " + (System.nanoTime() - started) 
> / 10L);
> }
> try (CloseableHttpClient client = HttpClients.createDefault()) {
>   HttpGet httpGet = new HttpGet("https://example.com:81;);
>   
> httpGet.setConfig(RequestConfig.custom().setConnectTimeout(3000).build());
>   long started2 = System.nanoTime();
>   try (CloseableHttpResponse res = client.execute(httpGet)) {
>   } finally {
> System.out.println("Apache elapsed (s): " + (System.nanoTime() - 
> started2) / 10L);
>   }
> } catch (IOException ex) {
>   ex.printStackTrace(System.out);
> }
>   }
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCLIENT-2015) Connect timeout value is effective doubled

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2015.
---
Resolution: Not A Problem

> Connect timeout value is effective doubled
> --
>
> Key: HTTPCLIENT-2015
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2015
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.10
> Environment: Debian-based Linux
> openjdk version "1.8.0_222"
> OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1-b10)
> OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
>Reporter: Chanseok Oh
>Priority: Minor
>
> The connection times out, say, after 6 seconds when calling 
> `setConnectionTimeout(3000)`. The timeout value is effectively doubled. I've 
> provided sample code, which demonstrates that the JDK Socket times out after 
> 3 seconds, but the Apache HTTP Client times out after 6 seconds.
> This may seem like a minor bug to others, but our product functionality 
> depends on timeouts in part and allows the user to adjust the timeouts, and 
> this timeout-doubling-effect is causing some trouble to us.
> {code:java}
>   public static void main(String[] args) {
> long started = System.nanoTime();
> try (Socket socket = new Socket()) {
>   socket.connect(new InetSocketAddress("example.com", 81), 3000);
> } catch (IOException ex) {
>   ex.printStackTrace(System.out);
> } finally {
>   System.out.println("JDK elapsed (s): " + (System.nanoTime() - started) 
> / 10L);
> }
> try (CloseableHttpClient client = HttpClients.createDefault()) {
>   HttpGet httpGet = new HttpGet("https://example.com:81;);
>   
> httpGet.setConfig(RequestConfig.custom().setConnectTimeout(3000).build());
>   long started2 = System.nanoTime();
>   try (CloseableHttpResponse res = client.execute(httpGet)) {
>   } finally {
> System.out.println("Apache elapsed (s): " + (System.nanoTime() - 
> started2) / 10L);
>   }
> } catch (IOException ex) {
>   ex.printStackTrace(System.out);
> }
>   }
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2015) Connect timeout value is effective doubled

[Chanseok Oh (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928736#comment-16928736
 ] 

Chanseok Oh commented on HTTPCLIENT-2015:
-

Oh, okay. I think I got it but somewhat vaguely. Excuse my ignorance, as I'm 
not familiar with "multi-home DNS addresses".

So, in my test case above, does this mean that my machine has, say, two network 
cards (NIC or whatever) each assigned a different IP address connecting two 
different networks and HttpClient tries the first route that times out after 3 
seconds and then tries the second route, making the total "timeout" feel like 6 
seconds? Or, is it the {{example.com}} host that has two IPs?

> Connect timeout value is effective doubled
> --
>
> Key: HTTPCLIENT-2015
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2015
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.10
> Environment: Debian-based Linux
> openjdk version "1.8.0_222"
> OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1-b10)
> OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
>Reporter: Chanseok Oh
>Priority: Minor
>
> The connection times out, say, after 6 seconds when calling 
> `setConnectionTimeout(3000)`. The timeout value is effectively doubled. I've 
> provided sample code, which demonstrates that the JDK Socket times out after 
> 3 seconds, but the Apache HTTP Client times out after 6 seconds.
> This may seem like a minor bug to others, but our product functionality 
> depends on timeouts in part and allows the user to adjust the timeouts, and 
> this timeout-doubling-effect is causing some trouble to us.
> {code:java}
>   public static void main(String[] args) {
> long started = System.nanoTime();
> try (Socket socket = new Socket()) {
>   socket.connect(new InetSocketAddress("example.com", 81), 3000);
> } catch (IOException ex) {
>   ex.printStackTrace(System.out);
> } finally {
>   System.out.println("JDK elapsed (s): " + (System.nanoTime() - started) 
> / 10L);
> }
> try (CloseableHttpClient client = HttpClients.createDefault()) {
>   HttpGet httpGet = new HttpGet("https://example.com:81;);
>   
> httpGet.setConfig(RequestConfig.custom().setConnectTimeout(3000).build());
>   long started2 = System.nanoTime();
>   try (CloseableHttpResponse res = client.execute(httpGet)) {
>   } finally {
> System.out.println("Apache elapsed (s): " + (System.nanoTime() - 
> started2) / 10L);
>   }
> } catch (IOException ex) {
>   ex.printStackTrace(System.out);
> }
>   }
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

Re: TLS/SSL NIO improvements / 5.0 GA likely delayed

[Gary Gregory]

That all makes sense. It would be good to have another beta once this
non-trivial work is done.

Gary

On Thu, Sep 12, 2019, 10:32 Oleg Kalnichevski  wrote:

> Folks
>
> I have decided to revisit the NIO based TLS/SSL implementation and see
> if I could resolve the issue with internal data buffering that makes
> protocol handler code more difficult and more error-prone as a result.
>
> You can see what I am trying to do here:
>
> https://github.com/apache/httpcomponents-core/commits/tls-nio
>
> This is likely to cause a delay with 5.0 GA as I will inevitably break
> some low level APIs. There should be no user visible changes.
>
> Oleg
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
> For additional commands, e-mail: dev-h...@hc.apache.org
>
>

[jira] [Created] (HTTPCORE-600) SocketTimeoutException when remote server closes connection after payload has been delivered

[Brian B (Jira)]

Brian B created HTTPCORE-600:


 Summary: SocketTimeoutException when remote server closes 
connection after payload has been delivered
 Key: HTTPCORE-600
 URL: https://issues.apache.org/jira/browse/HTTPCORE-600
 Project: HttpComponents HttpCore
  Issue Type: Bug
  Components: HttpCore NIO
Affects Versions: 4.4.12, 4.4.11, 4.4.10
Reporter: Brian B
 Attachments: asyncbugtest.zip

Initially I created this under CXF (CXF-8113), but I think the consensus is it 
is an NIO issue.  The description from that Jira edited to remove CXF-specific 
changes:

When using the Asynchronous Client HTTP Transport to handle soap web services, 
we get a SocketTimeout exception when a remote server closes the connection 
after the payload has been delivered.

The problem occurs when the client receives a TLS {{close_notify}} _after_ the 
payload has been received.  The payload must be larger than the allocated byte 
buffer receiving the decrypted payload (~16K in the default case). This results 
in decrypted data being available in the {{httpcore-nio}} library, but not 
immediately consumed. 

When these conditions are met, the following section of code is invoked:
 
{{org.apache.cxf.transport.http.asyncclient.SharedInputBuffer#_consumeContent_:{color:#0747a6}111{color}}}:
{code:java}
if (!this.buffer.hasRemaining() && this.ioctrl != null && !this.endOfStream) {
this.ioctrl.suspendInput();
}
{code}
The suspension of input when combined with the connection close and the 
following change introduced in {{httpcore-nio}} 4.4.10 (and it's revised 4.4.11 
version here) truly removes the {{READ}} {{EventMask}} from the underlying 
{{IOSessionImpl}}:
 
{{org.apache.http.nio.reactor.ssl.SSLIOSession#_updateEventMask_:{color:#0747a6}402{color}}}
{code:java}
if (this.endOfStream && (this.appBufferStatus == null || 
!this.appBufferStatus.hasBufferedInput())) {
newMask = newMask & ~EventMask.READ;
}
{code}
Once this happens, requests for input by 
{{SharedInputBuffer#_waitForData(int)_}} enter the {{_updateEventMask_}} method 
as expected, but the new {{httpcore-nio}} code above prevents the read 
operation from being reenabled despite the remainder of the decrypted payload 
being available in the {{SSLIOSession}} {{inPlain}} buffer.  The call 
ultimately fails with a {{SocketTimeoutException}}.

I created a unit test to demonstrate the failure and attached it to the JIRA.  
Given the complexity of the code and multiple buffers at play, I have not been 
able to come up with a fix beyond modifying the {{SSLIOSession}} code above to 
account for the buffered decrypted content.  It seems in all cases where 
{{appBufferStatus.hasBufferedInput()}} is queried, {{inPlain.hasData()}} is 
also queried, with the exception of the {{updateEventMask}} code above.

Please let me know if there's anything else you need from me.

 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

TLS/SSL NIO improvements / 5.0 GA likely delayed

[Oleg Kalnichevski]

Folks

I have decided to revisit the NIO based TLS/SSL implementation and see
if I could resolve the issue with internal data buffering that makes
protocol handler code more difficult and more error-prone as a result.

You can see what I am trying to do here:

https://github.com/apache/httpcomponents-core/commits/tls-nio

This is likely to cause a delay with 5.0 GA as I will inevitably break
some low level APIs. There should be no user visible changes.

Oleg  


-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCORE-596.

Fix Version/s: 5.0-beta9
   Resolution: Fixed

Merged to master.

Oleg

> Closing idle connections may end up in a blocked connection pool
> 
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
> Fix For: 5.0-beta9
>
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime 
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing 
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
>  \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
>  \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
>  \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
>  \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
>  \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
>  \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
>  \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
>  \{{ - locked java.lang.Object@7b9c4e14}}
>  \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
>  \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
>  \{{ at 
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
>  \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
>  \{{ at 
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
>  \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
>  \{{ at 
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
>  \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
>  \{{ at 
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
>  \{{ at 
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
>  \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
>  \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
>  \{{ at 
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
>  }}
>  
> While the thread is waiting here for the close the whole connection pool is 
> blocked due to some lock in the enumAvailble method. Any try to lease a new 
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to 
> that specific route the close is pending. Finally my whole application is 
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the 
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the  
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on 
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by 
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
>  \{{ at sun.misc.Unsafe.park(Native Method)}}
>  \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
>  \{{ at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
>  \{{ at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
>  \{{ at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
>  \{{ at 
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
>  \{{ at 
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool.access$200(AbstractConnPool.java:69)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java:245)}}
>  \{{ - locked org.apache.http.pool.AbstractConnPool$2@1e25f993}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java:193)}}
>  \{{ at 
> 

[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928371#comment-16928371
 ] 

ASF subversion and git services commented on HTTPCORE-596:
--

Commit 478e1cf6723475f546a0a68e6aee149da3b345c2 in httpcomponents-core's branch 
refs/heads/master from Oleg Kalnichevski
[ https://gitbox.apache.org/repos/asf?p=httpcomponents-core.git;h=478e1cf ]

HTTPCORE-596: connection pools to use an optional callback to close out 
disposed connections


> Closing idle connections may end up in a blocked connection pool
> 
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime 
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing 
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
>  \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
>  \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
>  \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
>  \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
>  \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
>  \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
>  \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
>  \{{ - locked java.lang.Object@7b9c4e14}}
>  \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
>  \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
>  \{{ at 
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
>  \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
>  \{{ at 
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
>  \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
>  \{{ at 
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
>  \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
>  \{{ at 
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
>  \{{ at 
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
>  \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
>  \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
>  \{{ at 
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
>  }}
>  
> While the thread is waiting here for the close the whole connection pool is 
> blocked due to some lock in the enumAvailble method. Any try to lease a new 
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to 
> that specific route the close is pending. Finally my whole application is 
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the 
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the  
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on 
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by 
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
>  \{{ at sun.misc.Unsafe.park(Native Method)}}
>  \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
>  \{{ at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
>  \{{ at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
>  \{{ at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
>  \{{ at 
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
>  \{{ at 
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool.access$200(AbstractConnPool.java:69)}}
>  

[jira] [Commented] (HTTPCLIENT-2015) Connect timeout value is effective doubled

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928358#comment-16928358
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2015:
---

[~francium25] Timeouts define the maximum period of between two consecutive 
network packets or i/o operations. They are not absolute values. For instance, 
for multi-home DNS addresses HttpClient will attempt to connect to each defined 
IP endpoints by default before throwing a timeout exception.

Oleg

> Connect timeout value is effective doubled
> --
>
> Key: HTTPCLIENT-2015
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2015
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.10
> Environment: Debian-based Linux
> openjdk version "1.8.0_222"
> OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1-b10)
> OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
>Reporter: Chanseok Oh
>Priority: Minor
>
> The connection times out, say, after 6 seconds when calling 
> `setConnectionTimeout(3000)`. The timeout value is effectively doubled. I've 
> provided sample code, which demonstrates that the JDK Socket times out after 
> 3 seconds, but the Apache HTTP Client times out after 6 seconds.
> This may seem like a minor bug to others, but our product functionality 
> depends on timeouts in part and allows the user to adjust the timeouts, and 
> this timeout-doubling-effect is causing some trouble to us.
> {code:java}
>   public static void main(String[] args) {
> long started = System.nanoTime();
> try (Socket socket = new Socket()) {
>   socket.connect(new InetSocketAddress("example.com", 81), 3000);
> } catch (IOException ex) {
>   ex.printStackTrace(System.out);
> } finally {
>   System.out.println("JDK elapsed (s): " + (System.nanoTime() - started) 
> / 10L);
> }
> try (CloseableHttpClient client = HttpClients.createDefault()) {
>   HttpGet httpGet = new HttpGet("https://example.com:81;);
>   
> httpGet.setConfig(RequestConfig.custom().setConnectTimeout(3000).build());
>   long started2 = System.nanoTime();
>   try (CloseableHttpResponse res = client.execute(httpGet)) {
>   } finally {
> System.out.println("Apache elapsed (s): " + (System.nanoTime() - 
> started2) / 10L);
>   }
> } catch (IOException ex) {
>   ex.printStackTrace(System.out);
> }
>   }
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCORE-599) Capacity feedback does not throttle AsyncDataConsumer.consume() for Http1

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928354#comment-16928354
 ] 

ASF subversion and git services commented on HTTPCORE-599:
--

Commit 44cab548cb4e15d56235aa12eaf0898d028351d0 in httpcomponents-core's branch 
refs/heads/HTTPCORE-596 from Oleg Kalnichevski
[ https://gitbox.apache.org/repos/asf?p=httpcomponents-core.git;h=44cab54 ]

HTTPCORE-599: I/O sessions to stop reading from the underlying network channel 
of READ interest is disabled


> Capacity feedback does not throttle AsyncDataConsumer.consume() for Http1
> -
>
> Key: HTTPCORE-599
> URL: https://issues.apache.org/jira/browse/HTTPCORE-599
> Project: HttpComponents HttpCore
>  Issue Type: Bug
>  Components: HttpCore NIO
>Affects Versions: 5.0-beta8
> Environment: macOS Mojave using the IntelliJ CE 2019.2.1 runtime
>Reporter: Roy Hashimoto
>Priority: Blocker
> Fix For: 5.0-beta9
>
> Attachments: SlowConsumerTest.java
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The capacity window mechanism in AbstractHttp1StreamDuplexer does not appear 
> to be throttling calls to AsyncDataConsumer.consume(). This affects both 
> client and server asynchronous entity consumers.
> The attached test program implements a simple 5.0 asynchronous HTTP/1.1 
> server. Aside from boilerplate code, the significant parts are:
>  # Set the HTTP1 initial window size to 8192.
>  # Add an AsyncExchangeHandler that creates an empty response and logs the 
> AsyncDataConsumer method calls to the console.
>  # Updating the capacity channel is intentionally omitted, effectively 
> providing an infinitely slow consumer.
> To test, post data of more than 8 KB to the server (e.g. using curl). I 
> expect to see in the log consume() calls totaling at least 8192 and nothing 
> after that until the socket times out. Instead, what I see is consume() calls 
> delivering all the data and then streamEnd().
> Here is sample output for uploading almost 32 KB. I have used my IDE to add a 
> logging breakpoint to show the capacity window on this line:
> [https://github.com/apache/httpcomponents-core/blob/a60528ea58877d55dab266bd2813e065aac6ff2c/httpcore5/src/main/java/org/apache/hc/core5/http/impl/nio/AbstractHttp1StreamDuplexer.java#L313]
> so those are also included in the output below.
> {{Mon Sep 09, 2019 10:27:18.893 AM SlowConsumerTest main FINE: Listening on 
> /0:0:0:0:0:0:0:0:8080}}
>  {{Mon Sep 09, 2019 10:27:31.294 AM SlowConsumerTest$MyExchangeHandler 
> handleRequest FINEST: handleRequest called}}
>  {{Mon Sep 09, 2019 10:27:31.332 AM SlowConsumerTest$MyExchangeHandler 
> consume FINEST: consume 8192 total 8192}}
>  {{AbstractHttp1StreamDuplexer:313 capacity=0}}
>  {{Mon Sep 09, 2019 10:27:31.341 AM SlowConsumerTest$MyExchangeHandler 
> updateCapacity FINEST: updateCapacity called}}
>  {{Mon Sep 09, 2019 10:27:31.342 AM SlowConsumerTest$MyExchangeHandler 
> consume FINEST: consume 8192 total 16384}}
>  {{AbstractHttp1StreamDuplexer:313 capacity=-8192}}
>  {{Mon Sep 09, 2019 10:27:31.346 AM SlowConsumerTest$MyExchangeHandler 
> updateCapacity FINEST: updateCapacity called}}
>  {{Mon Sep 09, 2019 10:27:31.348 AM SlowConsumerTest$MyExchangeHandler 
> consume FINEST: consume 8192 total 24576}}
>  {{AbstractHttp1StreamDuplexer:313 capacity=-16384}}
>  {{Mon Sep 09, 2019 10:27:31.352 AM SlowConsumerTest$MyExchangeHandler 
> updateCapacity FINEST: updateCapacity called}}
>  {{Mon Sep 09, 2019 10:27:31.354 AM SlowConsumerTest$MyExchangeHandler 
> consume FINEST: consume 8150 total 32726}}
>  {{AbstractHttp1StreamDuplexer:313 capacity=-24534}}
>  {{Mon Sep 09, 2019 10:27:31.357 AM SlowConsumerTest$MyExchangeHandler 
> streamEnd FINEST: streamEnd called}}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCORE-596) Closing idle connections may end up in a blocked connection pool

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928355#comment-16928355
 ] 

ASF subversion and git services commented on HTTPCORE-596:
--

Commit 478e1cf6723475f546a0a68e6aee149da3b345c2 in httpcomponents-core's branch 
refs/heads/HTTPCORE-596 from Oleg Kalnichevski
[ https://gitbox.apache.org/repos/asf?p=httpcomponents-core.git;h=478e1cf ]

HTTPCORE-596: connection pools to use an optional callback to close out 
disposed connections


> Closing idle connections may end up in a blocked connection pool
> 
>
> Key: HTTPCORE-596
> URL: https://issues.apache.org/jira/browse/HTTPCORE-596
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 4.4.11
>Reporter: Thomas Sartissohn
>Priority: Major
>
> We have some http endpoint not returning the CLOSE_NOTIFY in time. Sometime 
> it takes about up to 15 minutes till the connection is closed.
> During that time we find the PoolingHttpClientConnectionManager in closing 
> the idle connections. Here the stack:
> {{"IdleConnectionMonitor:MyHTTPClient" Id=765 in RUNNABLE}}
>  \{{ at java.net.SocketInputStream.socketRead0(Native Method)}}
>  \{{ at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)}}
>  \{{ at java.net.SocketInputStream.read(SocketInputStream.java:171)}}
>  \{{ at java.net.SocketInputStream.read(SocketInputStream.java:141)}}
>  \{{ at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)}}
>  \{{ at sun.security.ssl.InputRecord.read(InputRecord.java:503)}}
>  \{{ at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)}}
>  \{{ - locked java.lang.Object@7b9c4e14}}
>  \{{ at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1761)}}
>  \{{ at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1571)}}
>  \{{ at 
> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1705)}}
>  \{{ at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1594)}}
>  \{{ at 
> a.b.security.zone.api.factories.ZoneSSLSocket.close(ZoneSSLSocket.java:572)}}
>  \{{ - locked a.b.security.zone.api.factories.ZoneSSLSocket@307c017c}}
>  \{{ at 
> a.b.http.client.sdk.HTTPDumpingSSLSocket.close(HTTPDumpingSSLSocket.java:77)}}
>  \{{ - locked a.b.http.client.sdk.HTTPDumpingSSLSocket@658dab05}}
>  \{{ at 
> a.b.http.base.connection.MyDefaultClientConnection.close(MyDefaultClientConnection.java:711)}}
>  \{{ at 
> org.apache.http.impl.conn.CPoolEntry.closeConnection(CPoolEntry.java:70)}}
>  \{{ at org.apache.http.impl.conn.CPoolEntry.close(CPoolEntry.java:96)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool$3.process(AbstractConnPool.java:622)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool.enumAvailable(AbstractConnPool.java:561)}}
>  \{{ at org.apache.http.impl.conn.CPool.enumAvailable(CPool.java:76)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool.closeIdle(AbstractConnPool.java:617)}}
>  \{{ at 
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.closeIdleConnections(PoolingHttpClientConnectionManager.java:424)
>  }}
>  
> While the thread is waiting here for the close the whole connection pool is 
> blocked due to some lock in the enumAvailble method. Any try to lease a new 
> connection needs to wait till the lock is released here.
> The problem is that the whole pool is blocked and not only connections to 
> that specific route the close is pending. Finally my whole application is 
> waiting for this single partner to reply on the close.
> Is there any chance to optimze the pool to be less strict when iterating the 
> connection pool?
> Here an example for a a call waiting for a connection due to the lock in the  
> enumAvailable method:
> {{"pool-144-thread-13" Id=30179 in WAITING on 
> lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@7dbb4bbf owned by 
> IdleConnectionMonitor:MyHTTPClient Id=27798}}
>  \{{ at sun.misc.Unsafe.park(Native Method)}}
>  \{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
>  \{{ at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)}}
>  \{{ at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)}}
>  \{{ at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)}}
>  \{{ at 
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)}}
>  \{{ at 
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)}}
>  \{{ at 
> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:305)}}
>  \{{ at 
> 

[jira] [Resolved] (HTTPCORE-599) Capacity feedback does not throttle AsyncDataConsumer.consume() for Http1

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCORE-599.

Resolution: Fixed

Merged to master in as is. I am however considering a major refactoring of 
TLS/SSL buffering code as a separate work item.

Oleg 

> Capacity feedback does not throttle AsyncDataConsumer.consume() for Http1
> -
>
> Key: HTTPCORE-599
> URL: https://issues.apache.org/jira/browse/HTTPCORE-599
> Project: HttpComponents HttpCore
>  Issue Type: Bug
>  Components: HttpCore NIO
>Affects Versions: 5.0-beta8
> Environment: macOS Mojave using the IntelliJ CE 2019.2.1 runtime
>Reporter: Roy Hashimoto
>Priority: Blocker
> Fix For: 5.0-beta9
>
> Attachments: SlowConsumerTest.java
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The capacity window mechanism in AbstractHttp1StreamDuplexer does not appear 
> to be throttling calls to AsyncDataConsumer.consume(). This affects both 
> client and server asynchronous entity consumers.
> The attached test program implements a simple 5.0 asynchronous HTTP/1.1 
> server. Aside from boilerplate code, the significant parts are:
>  # Set the HTTP1 initial window size to 8192.
>  # Add an AsyncExchangeHandler that creates an empty response and logs the 
> AsyncDataConsumer method calls to the console.
>  # Updating the capacity channel is intentionally omitted, effectively 
> providing an infinitely slow consumer.
> To test, post data of more than 8 KB to the server (e.g. using curl). I 
> expect to see in the log consume() calls totaling at least 8192 and nothing 
> after that until the socket times out. Instead, what I see is consume() calls 
> delivering all the data and then streamEnd().
> Here is sample output for uploading almost 32 KB. I have used my IDE to add a 
> logging breakpoint to show the capacity window on this line:
> [https://github.com/apache/httpcomponents-core/blob/a60528ea58877d55dab266bd2813e065aac6ff2c/httpcore5/src/main/java/org/apache/hc/core5/http/impl/nio/AbstractHttp1StreamDuplexer.java#L313]
> so those are also included in the output below.
> {{Mon Sep 09, 2019 10:27:18.893 AM SlowConsumerTest main FINE: Listening on 
> /0:0:0:0:0:0:0:0:8080}}
>  {{Mon Sep 09, 2019 10:27:31.294 AM SlowConsumerTest$MyExchangeHandler 
> handleRequest FINEST: handleRequest called}}
>  {{Mon Sep 09, 2019 10:27:31.332 AM SlowConsumerTest$MyExchangeHandler 
> consume FINEST: consume 8192 total 8192}}
>  {{AbstractHttp1StreamDuplexer:313 capacity=0}}
>  {{Mon Sep 09, 2019 10:27:31.341 AM SlowConsumerTest$MyExchangeHandler 
> updateCapacity FINEST: updateCapacity called}}
>  {{Mon Sep 09, 2019 10:27:31.342 AM SlowConsumerTest$MyExchangeHandler 
> consume FINEST: consume 8192 total 16384}}
>  {{AbstractHttp1StreamDuplexer:313 capacity=-8192}}
>  {{Mon Sep 09, 2019 10:27:31.346 AM SlowConsumerTest$MyExchangeHandler 
> updateCapacity FINEST: updateCapacity called}}
>  {{Mon Sep 09, 2019 10:27:31.348 AM SlowConsumerTest$MyExchangeHandler 
> consume FINEST: consume 8192 total 24576}}
>  {{AbstractHttp1StreamDuplexer:313 capacity=-16384}}
>  {{Mon Sep 09, 2019 10:27:31.352 AM SlowConsumerTest$MyExchangeHandler 
> updateCapacity FINEST: updateCapacity called}}
>  {{Mon Sep 09, 2019 10:27:31.354 AM SlowConsumerTest$MyExchangeHandler 
> consume FINEST: consume 8150 total 32726}}
>  {{AbstractHttp1StreamDuplexer:313 capacity=-24534}}
>  {{Mon Sep 09, 2019 10:27:31.357 AM SlowConsumerTest$MyExchangeHandler 
> streamEnd FINEST: streamEnd called}}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCORE-599) Capacity feedback does not throttle AsyncDataConsumer.consume() for Http1

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928345#comment-16928345
 ] 

ASF subversion and git services commented on HTTPCORE-599:
--

Commit 44cab548cb4e15d56235aa12eaf0898d028351d0 in httpcomponents-core's branch 
refs/heads/master from Oleg Kalnichevski
[ https://gitbox.apache.org/repos/asf?p=httpcomponents-core.git;h=44cab54 ]

HTTPCORE-599: I/O sessions to stop reading from the underlying network channel 
of READ interest is disabled


> Capacity feedback does not throttle AsyncDataConsumer.consume() for Http1
> -
>
> Key: HTTPCORE-599
> URL: https://issues.apache.org/jira/browse/HTTPCORE-599
> Project: HttpComponents HttpCore
>  Issue Type: Bug
>  Components: HttpCore NIO
>Affects Versions: 5.0-beta8
> Environment: macOS Mojave using the IntelliJ CE 2019.2.1 runtime
>Reporter: Roy Hashimoto
>Priority: Blocker
> Fix For: 5.0-beta9
>
> Attachments: SlowConsumerTest.java
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The capacity window mechanism in AbstractHttp1StreamDuplexer does not appear 
> to be throttling calls to AsyncDataConsumer.consume(). This affects both 
> client and server asynchronous entity consumers.
> The attached test program implements a simple 5.0 asynchronous HTTP/1.1 
> server. Aside from boilerplate code, the significant parts are:
>  # Set the HTTP1 initial window size to 8192.
>  # Add an AsyncExchangeHandler that creates an empty response and logs the 
> AsyncDataConsumer method calls to the console.
>  # Updating the capacity channel is intentionally omitted, effectively 
> providing an infinitely slow consumer.
> To test, post data of more than 8 KB to the server (e.g. using curl). I 
> expect to see in the log consume() calls totaling at least 8192 and nothing 
> after that until the socket times out. Instead, what I see is consume() calls 
> delivering all the data and then streamEnd().
> Here is sample output for uploading almost 32 KB. I have used my IDE to add a 
> logging breakpoint to show the capacity window on this line:
> [https://github.com/apache/httpcomponents-core/blob/a60528ea58877d55dab266bd2813e065aac6ff2c/httpcore5/src/main/java/org/apache/hc/core5/http/impl/nio/AbstractHttp1StreamDuplexer.java#L313]
> so those are also included in the output below.
> {{Mon Sep 09, 2019 10:27:18.893 AM SlowConsumerTest main FINE: Listening on 
> /0:0:0:0:0:0:0:0:8080}}
>  {{Mon Sep 09, 2019 10:27:31.294 AM SlowConsumerTest$MyExchangeHandler 
> handleRequest FINEST: handleRequest called}}
>  {{Mon Sep 09, 2019 10:27:31.332 AM SlowConsumerTest$MyExchangeHandler 
> consume FINEST: consume 8192 total 8192}}
>  {{AbstractHttp1StreamDuplexer:313 capacity=0}}
>  {{Mon Sep 09, 2019 10:27:31.341 AM SlowConsumerTest$MyExchangeHandler 
> updateCapacity FINEST: updateCapacity called}}
>  {{Mon Sep 09, 2019 10:27:31.342 AM SlowConsumerTest$MyExchangeHandler 
> consume FINEST: consume 8192 total 16384}}
>  {{AbstractHttp1StreamDuplexer:313 capacity=-8192}}
>  {{Mon Sep 09, 2019 10:27:31.346 AM SlowConsumerTest$MyExchangeHandler 
> updateCapacity FINEST: updateCapacity called}}
>  {{Mon Sep 09, 2019 10:27:31.348 AM SlowConsumerTest$MyExchangeHandler 
> consume FINEST: consume 8192 total 24576}}
>  {{AbstractHttp1StreamDuplexer:313 capacity=-16384}}
>  {{Mon Sep 09, 2019 10:27:31.352 AM SlowConsumerTest$MyExchangeHandler 
> updateCapacity FINEST: updateCapacity called}}
>  {{Mon Sep 09, 2019 10:27:31.354 AM SlowConsumerTest$MyExchangeHandler 
> consume FINEST: consume 8150 total 32726}}
>  {{AbstractHttp1StreamDuplexer:313 capacity=-24534}}
>  {{Mon Sep 09, 2019 10:27:31.357 AM SlowConsumerTest$MyExchangeHandler 
> streamEnd FINEST: streamEnd called}}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[GitHub] [httpcomponents-core] ok2c merged pull request #148: HTTPCORE-599

[GitBox]

ok2c merged pull request #148: HTTPCORE-599
URL: https://github.com/apache/httpcomponents-core/pull/148
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org