[jira] [Commented] (HTTPCORE-771) Exception when using HttpClient 5.4 with Oracle Java 8

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17887652#comment-17887652
 ] 

Oleg Kalnichevski commented on HTTPCORE-771:


> If this works with OpenJDK, then I would recommend to open an issue with MOSC 
> and have the actual fix backported to Oracle Java 8 after all. totally 
> unclear, why this hasn't been cherry-picked to Oracle's distro.

[~michael-o] I suppose because they see this as a feature not a defect. They 
expect people to have moved away from Java 8 by now.

Oleg

> Exception when using HttpClient 5.4 with Oracle Java 8
> --
>
> Key: HTTPCORE-771
> URL: https://issues.apache.org/jira/browse/HTTPCORE-771
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.3
> Environment: Oracle Java 8 (Windows and Linux)
>Reporter: Bernhard Fey
>Priority: Minor
> Fix For: 5.3.1, 5.4-alpha1
>
> Attachments: J8HttpsErrorTest.java
>
>
> When updating to version 5.4 our tests with Oracle Java 8 fail with the 
> following exception:
> java.lang.UnsupportedOperationException: method is not supported because of 
> the TLS half-close policy
>     at sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:765)
>     at sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:743)
>     at 
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase.close(BHttpConnectionBase.java:255)
>     at 
> org.apache.hc.core5.http.impl.io.DefaultBHttpClientConnection.close(DefaultBHttpClientConnection.java:71)
>     at 
> org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection.close(DefaultManagedHttpClientConnection.java:176)
>     at 
> org.apache.hc.core5.pool.PoolEntry.discardConnection(PoolEntry.java:180)
>     at 
> org.apache.hc.core5.pool.StrictConnPool$PerRoutePool.shutdown(StrictConnPool.java:839)
>     at org.apache.hc.core5.pool.StrictConnPool.close(StrictConnPool.java:142)
>     at 
> org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.close(PoolingHttpClientConnectionManager.java:277)
>     at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.close(InternalHttpClient.java:198)
>     at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.close(InternalHttpClient.java:188)
>     at J8HttpsErrorTest.main(J8HttpsErrorTest.java:22)
> I have attached a test class, which fails in Oracle Java 8 with the above 
> exception, but finishes correctly with Java 11 or OpenJDK 8.
> It has been tested on Windows 10, but the Oracle Java 8 exception also occurs 
> on Linux in our integration.
> The specific VM the exception was reproduced with is "Oracle Corporation Java 
> HotSpot(TM) 64-Bit Server VM 1.8.0_411-b09".



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCORE-771) Exception when using HttpClient 5.4 with Oracle Java 8

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCORE-771:
---
Fix Version/s: 5.3.1
   5.4-alpha1

> Exception when using HttpClient 5.4 with Oracle Java 8
> --
>
> Key: HTTPCORE-771
> URL: https://issues.apache.org/jira/browse/HTTPCORE-771
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.3
> Environment: Oracle Java 8 (Windows and Linux)
>Reporter: Bernhard Fey
>Priority: Minor
> Fix For: 5.3.1, 5.4-alpha1
>
> Attachments: J8HttpsErrorTest.java
>
>
> When updating to version 5.4 our tests with Oracle Java 8 fail with the 
> following exception:
> java.lang.UnsupportedOperationException: method is not supported because of 
> the TLS half-close policy
>     at sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:765)
>     at sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:743)
>     at 
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase.close(BHttpConnectionBase.java:255)
>     at 
> org.apache.hc.core5.http.impl.io.DefaultBHttpClientConnection.close(DefaultBHttpClientConnection.java:71)
>     at 
> org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection.close(DefaultManagedHttpClientConnection.java:176)
>     at 
> org.apache.hc.core5.pool.PoolEntry.discardConnection(PoolEntry.java:180)
>     at 
> org.apache.hc.core5.pool.StrictConnPool$PerRoutePool.shutdown(StrictConnPool.java:839)
>     at org.apache.hc.core5.pool.StrictConnPool.close(StrictConnPool.java:142)
>     at 
> org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.close(PoolingHttpClientConnectionManager.java:277)
>     at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.close(InternalHttpClient.java:198)
>     at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.close(InternalHttpClient.java:188)
>     at J8HttpsErrorTest.main(J8HttpsErrorTest.java:22)
> I have attached a test class, which fails in Oracle Java 8 with the above 
> exception, but finishes correctly with Java 11 or OpenJDK 8.
> It has been tested on Windows 10, but the Oracle Java 8 exception also occurs 
> on Linux in our integration.
> The specific VM the exception was reproduced with is "Oracle Corporation Java 
> HotSpot(TM) 64-Bit Server VM 1.8.0_411-b09".



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Moved] (HTTPCORE-771) Exception when using HttpClient 5.4 with Oracle Java 8

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski moved HTTPCLIENT-2345 to HTTPCORE-771:


  Component/s: HttpCore
   (was: HttpClient (classic))
  Key: HTTPCORE-771  (was: HTTPCLIENT-2345)
Affects Version/s: 5.3
   (was: 5.4)
 Workflow: classic default workflow  (was: Default workflow, 
editable Closed status)
   Issue Type: Improvement  (was: Bug)
  Project: HttpComponents HttpCore  (was: HttpComponents HttpClient)

> Exception when using HttpClient 5.4 with Oracle Java 8
> --
>
> Key: HTTPCORE-771
> URL: https://issues.apache.org/jira/browse/HTTPCORE-771
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.3
> Environment: Oracle Java 8 (Windows and Linux)
>Reporter: Bernhard Fey
>Priority: Minor
> Attachments: J8HttpsErrorTest.java
>
>
> When updating to version 5.4 our tests with Oracle Java 8 fail with the 
> following exception:
> java.lang.UnsupportedOperationException: method is not supported because of 
> the TLS half-close policy
>     at sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:765)
>     at sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:743)
>     at 
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase.close(BHttpConnectionBase.java:255)
>     at 
> org.apache.hc.core5.http.impl.io.DefaultBHttpClientConnection.close(DefaultBHttpClientConnection.java:71)
>     at 
> org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection.close(DefaultManagedHttpClientConnection.java:176)
>     at 
> org.apache.hc.core5.pool.PoolEntry.discardConnection(PoolEntry.java:180)
>     at 
> org.apache.hc.core5.pool.StrictConnPool$PerRoutePool.shutdown(StrictConnPool.java:839)
>     at org.apache.hc.core5.pool.StrictConnPool.close(StrictConnPool.java:142)
>     at 
> org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.close(PoolingHttpClientConnectionManager.java:277)
>     at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.close(InternalHttpClient.java:198)
>     at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.close(InternalHttpClient.java:188)
>     at J8HttpsErrorTest.main(J8HttpsErrorTest.java:22)
> I have attached a test class, which fails in Oracle Java 8 with the above 
> exception, but finishes correctly with Java 11 or OpenJDK 8.
> It has been tested on Windows 10, but the Oracle Java 8 exception also occurs 
> on Linux in our integration.
> The specific VM the exception was reproduced with is "Oracle Corporation Java 
> HotSpot(TM) 64-Bit Server VM 1.8.0_411-b09".



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCLIENT-2344) HTTP/1.1 TLS Upgrade (RFC-2817) should not be default

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2344.
---
Fix Version/s: (was: 5.4.1)
   Resolution: Invalid

[~bplotnick] Please report this issue to the developers of Envoy and ask them 
to correct the behavior on their side.

Oleg

> HTTP/1.1 TLS Upgrade (RFC-2817) should not be default
> -
>
> Key: HTTPCLIENT-2344
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2344
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.4
>Reporter: Ben Plotnick
>Priority: Minor
>
> Version 5.4 added RFC-2817 support, which by default tries to upgrade  since 
> protocolUpgradeEnabled is default enabled.
> Although the strict reading of the spec would indicate that a server should 
> ignore upgrade requests that it cannot service, conservative proxies might 
> reject these requests entirely. This is the case in Envoy today
> I don't see a big advantage to enabling this by default and it is causing 
> real issues now.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Comment Edited] (HTTPCLIENT-2344) HTTP/1.1 TLS Upgrade (RFC-2817) should not be default

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884596#comment-17884596
 ] 

Oleg Kalnichevski edited comment on HTTPCLIENT-2344 at 9/25/24 12:50 PM:
-

I ran a few message exchnages using this sample [1] as a test case. The proxy 
used for tests is Squid 3 running in a docker container. The target server is 
httpbin.org that we all love.   

This is the wire log of a message exchange executed through a proxy tunnel. As 
you can see an `Upgrade` request header does not get added to the outgoing 
request messages (both CONNECT and GET) given the tunnel has already been 
already TLS secured. 

{noformat}
Executing request GET /get via http://localhost:
2024-09-25 14:08:20,739 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ex-01 preparing request execution
2024-09-25 14:08:20,768 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.ProtocolExec] ex-01 
target auth state: UNCHALLENGED
2024-09-25 14:08:20,769 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.ConnectExec] ex-01 
acquiring connection with route 
{tls}->http://localhost:->[https://httpbin.org:443]
2024-09-25 14:08:20,770 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ex-01 acquiring endpoint (3 MINUTES)
2024-09-25 14:08:20,773 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ex-01 endpoint lease request (3 MINUTES) [route: 
{tls}->http://localhost:->[https://httpbin.org:443]][total available: 0; 
route allocated: 0 of 5; total allocated: 0 of 25]
2024-09-25 14:08:20,778 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ex-01 endpoint leased [route: 
{tls}->http://localhost:->[https://httpbin.org:443]][total available: 0; 
route allocated: 1 of 5; total allocated: 1 of 25]
2024-09-25 14:08:20,792 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ex-01 acquired ep-01
2024-09-25 14:08:20,793 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ex-01 acquired endpoint ep-01
2024-09-25 14:08:20,793 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.ConnectExec] ex-01 
opening connection {tls}->http://localhost:->[https://httpbin.org:443]
2024-09-25 14:08:20,794 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ep-01 connecting endpoint (null)
2024-09-25 14:08:20,796 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ep-01 connecting endpoint to http://localhost: (3 MINUTES)
2024-09-25 14:08:20,797 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator] 
localhost resolving remote address
2024-09-25 14:08:20,797 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator] 
localhost resolved to [localhost/127.0.0.1]
2024-09-25 14:08:20,798 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator] 
http://localhost: connecting null->localhost/127.0.0.1: (3 MINUTES)
2024-09-25 14:08:20,799 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator] 
http-outgoing-0 http://localhost: connected 
/127.0.0.1:38572->localhost/127.0.0.1:
2024-09-25 14:08:20,799 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection] 
http-outgoing-0 set socket timeout to 3 MINUTES
2024-09-25 14:08:20,800 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ep-01 connected http-outgoing-0
2024-09-25 14:08:20,800 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ep-01 endpoint connected
2024-09-25 14:08:20,801 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ep-01 start execution ex-01
2024-09-25 14:08:20,803 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ep-01 executing exchange ex-01 over http-outgoing-0
2024-09-25 14:08:20,804 DEBUG [main][org.apache.hc.client5.http.headers] 
http-outgoing-0 >> CONNECT httpbin.org:443 HTTP/1.1
2024-09-25 14:08:20,804 DEBUG [main][org.apache.hc.client5.http.headers] 
http-outgoing-0 >> Host: httpbin.org:443
2024-09-25 14:08:20,804 DEBUG [main][org.apache.hc.client5.http.headers] 
http-outgoing-0 >> User-Agent: Apache-HttpClient/5.4.1-SNAPSHOT (Java/1.8.0_342)
2024-09-25 14:08:21,000 DEBUG [main][org.apache.hc.client5.http.headers] 
http-outgoing-0 << HTTP/1.1 200 Connection established
2024-09-25 14:08:21,000 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.ConnectExec] ex-01 
tunnel to target created.
2024-09-25 14:08:21,000 DEBUG 
[main][org.apache.hc.client5.htt

[jira] [Commented] (HTTPCLIENT-2344) HTTP/1.1 TLS Upgrade (RFC-2817) should not be default

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884596#comment-17884596
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2344:
---

I ran a few message exchnages using this sample [1] as a test case. The proxy 
used for tests is Squid 3 running in a docker container. The target server is 
httpbin.org that we all love.   

This is the wire log of a message exchange executed through a proxy tunnel. As 
you can see an `Upgrade` request header does not get added to outgoing message 
given the tunnel has already been already TLS secured. 

{noformat}
Executing request GET /get via http://localhost:
2024-09-25 14:08:20,739 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ex-01 preparing request execution
2024-09-25 14:08:20,768 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.ProtocolExec] ex-01 
target auth state: UNCHALLENGED
2024-09-25 14:08:20,769 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.ConnectExec] ex-01 
acquiring connection with route 
{tls}->http://localhost:->[https://httpbin.org:443]
2024-09-25 14:08:20,770 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ex-01 acquiring endpoint (3 MINUTES)
2024-09-25 14:08:20,773 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ex-01 endpoint lease request (3 MINUTES) [route: 
{tls}->http://localhost:->[https://httpbin.org:443]][total available: 0; 
route allocated: 0 of 5; total allocated: 0 of 25]
2024-09-25 14:08:20,778 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ex-01 endpoint leased [route: 
{tls}->http://localhost:->[https://httpbin.org:443]][total available: 0; 
route allocated: 1 of 5; total allocated: 1 of 25]
2024-09-25 14:08:20,792 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ex-01 acquired ep-01
2024-09-25 14:08:20,793 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ex-01 acquired endpoint ep-01
2024-09-25 14:08:20,793 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.ConnectExec] ex-01 
opening connection {tls}->http://localhost:->[https://httpbin.org:443]
2024-09-25 14:08:20,794 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ep-01 connecting endpoint (null)
2024-09-25 14:08:20,796 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ep-01 connecting endpoint to http://localhost: (3 MINUTES)
2024-09-25 14:08:20,797 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator] 
localhost resolving remote address
2024-09-25 14:08:20,797 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator] 
localhost resolved to [localhost/127.0.0.1]
2024-09-25 14:08:20,798 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator] 
http://localhost: connecting null->localhost/127.0.0.1: (3 MINUTES)
2024-09-25 14:08:20,799 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator] 
http-outgoing-0 http://localhost: connected 
/127.0.0.1:38572->localhost/127.0.0.1:
2024-09-25 14:08:20,799 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection] 
http-outgoing-0 set socket timeout to 3 MINUTES
2024-09-25 14:08:20,800 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ep-01 connected http-outgoing-0
2024-09-25 14:08:20,800 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ep-01 endpoint connected
2024-09-25 14:08:20,801 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ep-01 start execution ex-01
2024-09-25 14:08:20,803 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ep-01 executing exchange ex-01 over http-outgoing-0
2024-09-25 14:08:20,804 DEBUG [main][org.apache.hc.client5.http.headers] 
http-outgoing-0 >> CONNECT httpbin.org:443 HTTP/1.1
2024-09-25 14:08:20,804 DEBUG [main][org.apache.hc.client5.http.headers] 
http-outgoing-0 >> Host: httpbin.org:443
2024-09-25 14:08:20,804 DEBUG [main][org.apache.hc.client5.http.headers] 
http-outgoing-0 >> User-Agent: Apache-HttpClient/5.4.1-SNAPSHOT (Java/1.8.0_342)
2024-09-25 14:08:21,000 DEBUG [main][org.apache.hc.client5.http.headers] 
http-outgoing-0 << HTTP/1.1 200 Connection established
2024-09-25 14:08:21,000 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.ConnectExec] ex-01 
tunnel to target created.
2024-09-25 14:08:21,000 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ep-01 upgrading endpoint
2024-09-25 14:08:21

[jira] [Commented] (HTTPCLIENT-2344) HTTP/1.1 TLS Upgrade (RFC-2817) should not be default

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884355#comment-17884355
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2344:
---

> Servers are not obliged to follow RFC-2817. It is not part of the HTTP/1.1 
> spec.

[~bplotnick] There is not a word in RFRC 9110 about it making RFC 2817 obsolete.

> This also does not prescribe server behavior and I don't believe a server 
> would be out of spec to reject this request.

You are making things up. There is not a word about about servers rejecting 
such requests, only that it is legal to ignore it.

> Breaking clients by default is backwards incompatible and unacceptable

Really? Breaking? I am very tempted to close this ticker as INVALID right here 
and now. However it will wait for the wire log with the session and will retest 
our code with Squid 3 to make sure we have not missed or overlooked  something.

Oleg

> HTTP/1.1 TLS Upgrade (RFC-2817) should not be default
> -
>
> Key: HTTPCLIENT-2344
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2344
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.4
>Reporter: Ben Plotnick
>Priority: Minor
> Fix For: 5.4.1
>
>
> Version 5.4 added RFC-2817 support, which by default tries to upgrade  since 
> protocolUpgradeEnabled is default enabled.
> Although the strict reading of the spec would indicate that a server should 
> ignore upgrade requests that it cannot service, conservative proxies might 
> reject these requests entirely. This is the case in Envoy today
> I don't see a big advantage to enabling this by default and it is causing 
> real issues now.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2344) HTTP/1.1 TLS Upgrade (RFC-2817) should not be default

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884320#comment-17884320
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2344:
---

> I don't see a big advantage to enabling this by default and it is causing 
> real issues now.

[~bplotnick] Any half-decent HTTP server or proxy should be able to handle 
`Upgrade` and ignore those upgrade protocols they do not support / understand. 
I do not see why this option should be disabled by default because of some 
overzealous proxy?

Please provide a full wire log of the session with that proxy.

Oleg

> HTTP/1.1 TLS Upgrade (RFC-2817) should not be default
> -
>
> Key: HTTPCLIENT-2344
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2344
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.4
>Reporter: Ben Plotnick
>Priority: Minor
> Fix For: 5.4.1
>
>
> Version 5.4 added RFC-2817 support, which by default tries to upgrade  since 
> protocolUpgradeEnabled is default enabled.
> Although the strict reading of the spec would indicate that a server should 
> ignore upgrade requests that it cannot service, conservative proxies might 
> reject these requests entirely. This is the case in Envoy today
> I don't see a big advantage to enabling this by default and it is causing 
> real issues now.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCLIENT-2344) HTTP/1.1 TLS Upgrade (RFC-2817) should not be default

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCLIENT-2344:
--
Fix Version/s: 5.4.1
 Priority: Minor  (was: Major)

> HTTP/1.1 TLS Upgrade (RFC-2817) should not be default
> -
>
> Key: HTTPCLIENT-2344
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2344
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.4
>Reporter: Ben Plotnick
>Priority: Minor
> Fix For: 5.4.1
>
>
> Version 5.4 added RFC-2817 support, which by default tries to upgrade  since 
> protocolUpgradeEnabled is default enabled.
> Although the strict reading of the spec would indicate that a server should 
> ignore upgrade requests that it cannot service, conservative proxies might 
> reject these requests entirely. This is the case in Envoy today
> I don't see a big advantage to enabling this by default and it is causing 
> real issues now.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCLIENT-2343) Regression in setting USER_TOKEN context attribute in PoolingHttpClientConnectionManager

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCLIENT-2343:
--
Fix Version/s: 5.4.1
Affects Version/s: 5.4
   (was: 5.4-beta1)
 Priority: Minor  (was: Major)

> Regression in setting USER_TOKEN context attribute in 
> PoolingHttpClientConnectionManager
> 
>
> Key: HTTPCLIENT-2343
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2343
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.4
>Reporter: Simon F
>Priority: Minor
> Fix For: 5.4.1
>
> Attachments: Reproducer.java
>
>
> Setting the USER_TOKEN context attribute changed between the 5.3.1 and 5.4 
> release.
> I’ve got a testcase where my application calls an https-endpoint five times 
> and checks that the source port used by the clients stays the same (the 
> existing connection is reused).
> Using the httpclient v5.3.1 everything works fine, all requests use the same 
> source port, but after upgrading to v5.4 my testcase fails.
> I’ve tracked down the issue to the USER_TOKEN, which we set explicitly on the 
> Context to reuse ssl connections even when client certificates are used.
> Due to performance reasons this is essential to us.
> We’ve set the Context like this:
> {code:java}
> HttpContext ctx = new HttpClientContext();
> ctx.setAttribute(HttpClientContext.{_}USER_TOKEN{_}, "localhost:8443");
> {code}
> But as seen in the logs, the USER_TOKEN is no longer added to the state:
> {code}
> [ForkJoinPool-1-worker-1] DEBUG 
> o.a.hc.client5.http.impl.io.PoolingHttpClientConnectionManager - 
> ex-01 endpoint lease request (3 MINUTES) [route: 
> \{s}->[https://localhost:65181]][total available: 0; route allocated: 0 of 
> 10; total allocated: 0 of 50]
> {code}
> After changing the way we set the USER_TOKEN, it works again:
> {code:java}
> HttpClientContext ctx = new HttpClientContext();
> ctx.setUserToken(location);
> {code}
> {code}
> [ForkJoinPool-1-worker-1] DEBUG 
> o.a.hc.client5.http.impl.io.PoolingHttpClientConnectionManager - 
> ex-01 endpoint lease request (3 MINUTES) [route: 
> \{s}->[https://localhost:65384]][state: ServiceLocation(hostname=localhost, 
> port=65384, ssl=true)][total available: 0; route allocated: 0 of 10; total 
> allocated: 0 of 50]
> {code}
> The USER_TOKEN was marked as deprecated in v5.4, but it should still work 
> without modification.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-1843) Create module httpclient5-compress to use Apache Commons Compress

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883459#comment-17883459
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-1843:
---

[~jkmlee] `commons-compress` will be kept optional at runtime and will need to 
be explicitly added to the project. Do not worry.

Oleg

> Create module httpclient5-compress to use Apache Commons Compress
> -
>
> Key: HTTPCLIENT-1843
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1843
> Project: HttpComponents HttpClient
>  Issue Type: New Feature
>Reporter: Gary D. Gregory
>Assignee: Gary D. Gregory
>Priority: Major
> Fix For: Future
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> Create module httpclient5-compress to use Apache Commons Compress.
> Allow users to benefit from the support of various algorithms supported 
> through the API of Apache Commons Compress.
> [Apache Commons Compress|https://commons.apache.org/proper/commons-compress/] 
> 1.14 will support Brotli when released.
> HttpClient should be recoded to find codec implementaions based on a Java 
> {{ServiceLoader}}. Then, {{httpclient5-compress}} can add to that registry if 
> it is on the classpath, the user will not have to do anything extra.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Comment Edited] (HTTPCLIENT-2342) Regression: Content-Type header not returned anymore

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883348#comment-17883348
 ] 

Oleg Kalnichevski edited comment on HTTPCLIENT-2342 at 9/20/24 4:48 PM:


[~ctabin] Please report this issue to whoever built the `SAINETV4` server and 
ask them to fix it. In the meantime one can disable automatic protocol upgrade 
by using custom `RequestConfig` with `protocolUpgradeEnabled` set to false.

Oleg


was (Author: olegk):
[~ctabin] Please report this issue to whoever build the `SAINETV4` server and 
ask them to fix it. In the meantime one can disable automatic protocol upgrade 
by using custom `RequestConfig` with `protocolUpgradeEnabled` set6 to false.

Oleg

> Regression: Content-Type header not returned anymore
> 
>
> Key: HTTPCLIENT-2342
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2342
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.4-beta1
>Reporter: Cedric Tabin
>Priority: Major
>
> Hello,
> We just upgraded today to httpclient5 version 5.4. It seems that the header 
> `Content-Type` is not returned anymore by the API, even when listing all the 
> headers.
> {code:java}
> client.execute(httpRequest, r -> {
> //does not print Content-Type header 
> for(Header h : r.getHeaders()) {
> System.out.println("## "+h.getName()+": "+h.getValue());
> }
> System.out.flush();
> return null;
> });
> {code}
> We can ensure that the `Content-Type` header is well set by the server by 
> browsing to the same URL and inspecting the network.
> Our code didn't change and with the version 5.3.1, the Content-Type is 
> correctly received.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2342) Regression: Content-Type header not returned anymore

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883348#comment-17883348
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2342:
---

[~ctabin] Please report this issue to whoever build the `SAINETV4` server and 
ask them to fix it. In the meantime one can disable automatic protocol upgrade 
by using custom `RequestConfig` with `protocolUpgradeEnabled` set6 to false.

Oleg

> Regression: Content-Type header not returned anymore
> 
>
> Key: HTTPCLIENT-2342
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2342
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.4-beta1
>Reporter: Cedric Tabin
>Priority: Major
>
> Hello,
> We just upgraded today to httpclient5 version 5.4. It seems that the header 
> `Content-Type` is not returned anymore by the API, even when listing all the 
> headers.
> {code:java}
> client.execute(httpRequest, r -> {
> //does not print Content-Type header 
> for(Header h : r.getHeaders()) {
> System.out.println("## "+h.getName()+": "+h.getValue());
> }
> System.out.flush();
> return null;
> });
> {code}
> We can ensure that the `Content-Type` header is well set by the server by 
> browsing to the same URL and inspecting the network.
> Our code didn't change and with the version 5.3.1, the Content-Type is 
> correctly received.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2342) Regression: Content-Type header not returned anymore

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883258#comment-17883258
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2342:
---

Just ran the latest HttpClient build. `Content-Type` is fine.  [~ctabin] Please 
note one should _never_ rely on `Content-Type` of a response.  One _must_ use 
`Content-Type` of the response entity.

Oleg

{noformat}
2024-09-20 14:59:31,447 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ex-01 preparing request execution
2024-09-20 14:59:31,479 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.ProtocolExec] ex-01 
target auth state: UNCHALLENGED
2024-09-20 14:59:31,480 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.ProtocolExec] ex-01 
proxy auth state: UNCHALLENGED
2024-09-20 14:59:31,483 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.ConnectExec] ex-01 
acquiring connection with route {}->[http://httpbin.org:80]
2024-09-20 14:59:31,484 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ex-01 acquiring endpoint (3 MINUTES)
2024-09-20 14:59:31,487 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ex-01 endpoint lease request (3 MINUTES) [route: 
{}->[http://httpbin.org:80]][total available: 0; route allocated: 0 of 5; total 
allocated: 0 of 25]
2024-09-20 14:59:31,496 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ex-01 endpoint leased [route: {}->[http://httpbin.org:80]][total 
available: 0; route allocated: 1 of 5; total allocated: 1 of 25]
2024-09-20 14:59:31,513 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ex-01 acquired ep-01
2024-09-20 14:59:31,513 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ex-01 acquired endpoint ep-01
2024-09-20 14:59:31,513 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.ConnectExec] ex-01 
opening connection {}->[http://httpbin.org:80]
2024-09-20 14:59:31,514 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ep-01 connecting endpoint (null)
2024-09-20 14:59:31,516 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ep-01 connecting endpoint to http://httpbin.org:80 (3 MINUTES)
2024-09-20 14:59:31,518 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator] 
httpbin.org resolving remote address
2024-09-20 14:59:31,525 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator] 
httpbin.org resolved to [httpbin.org/34.237.204.224, httpbin.org/34.231.0.251]
2024-09-20 14:59:31,526 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator] 
http://httpbin.org:80 connecting null->httpbin.org/34.237.204.224:80 (3 MINUTES)
2024-09-20 14:59:31,711 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator] 
http-outgoing-0 http://httpbin.org:80 connected 
/192.168.76.83:55060->httpbin.org/34.237.204.224:80
2024-09-20 14:59:31,711 DEBUG 
[main][org.apache.hc.client5.http.impl.io.DefaultManagedHttpClientConnection] 
http-outgoing-0 set socket timeout to 3 MINUTES
2024-09-20 14:59:31,711 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ep-01 connected http-outgoing-0
2024-09-20 14:59:31,711 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ep-01 endpoint connected
2024-09-20 14:59:31,713 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.MainClientExec] ex-01 
executing GET /get HTTP/1.1
2024-09-20 14:59:31,714 DEBUG 
[main][org.apache.hc.client5.http.protocol.RequestUpgrade] Connection is 
upgradable: protocol version = HTTP/1.1
2024-09-20 14:59:31,714 DEBUG 
[main][org.apache.hc.client5.http.protocol.RequestUpgrade] Connection is 
upgradable to TLS: method = GET
2024-09-20 14:59:31,714 DEBUG 
[main][org.apache.hc.client5.http.protocol.RequestAddCookies] ex-01 
Cookie spec selected: strict
2024-09-20 14:59:31,730 DEBUG 
[main][org.apache.hc.client5.http.impl.classic.InternalHttpClient] 
ep-01 start execution ex-01
2024-09-20 14:59:31,732 DEBUG 
[main][org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager] 
ep-01 executing exchange ex-01 over http-outgoing-0
2024-09-20 14:59:31,733 DEBUG [main][org.apache.hc.client5.http.headers] 
http-outgoing-0 >> GET /get HTTP/1.1
2024-09-20 14:59:31,734 DEBUG [main][org.apache.hc.client5.http.headers] 
http-outgoing-0 >> Accept-Encoding: gzip, x-gzip, deflate, br
2024-09-20 14:59:31,734 DEBUG [main][org.apache.hc.client5.http.headers] 
http-outgoing-0 >> Host: httpbin.org:80
2024-09-20 14:59:31,734 DEBUG [main][org.apache.hc.client5.http.headers]

[jira] [Resolved] (HTTPCORE-770) org.apache.hc.core5.net.URIBuilder - Allow to configure plusAsBlank

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCORE-770.

Resolution: Fixed

> org.apache.hc.core5.net.URIBuilder - Allow to configure plusAsBlank
> ---
>
> Key: HTTPCORE-770
> URL: https://issues.apache.org/jira/browse/HTTPCORE-770
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.3
>Reporter: Claus Ibsen
>Priority: Minor
> Fix For: 5.4-alpha1
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When upgrading from Http Client v4 to v5 then the URIBuilder has changed a 
> default value for the plusAsBlank option.
>  
> In v4 the default is true
>  
> org.apache.http.client.utils.URLEncodedUtils.decodeFormFields()
> ~~~
> return urlDecode(content, charset != null ? charset : Consts.UTF_8, true);
> ~~~
> In v5 the default is changed to false
> org.apache.hc.core5.net.URIBuilder.digestURI()
> ~~~
> this.queryParams = parseQuery(uri.getRawQuery(), charset, false);
> ~~~
>  
> I wonder if the URIBuilder could have some kind of flag that can be set to 
> set whether to use true or false.
>  
> import org.apache.hc.core5.http.NameValuePair;
> import org.apache.hc.core5.http.message.BasicNameValuePair;
> import org.apache.hc.core5.net.URIBuilder;
>  
>         URIBuilder uriBuilder;
>         uriBuilder = new 
> URIBuilder("[http://dummy/demo+demo?sample=sample+sample]";);
>         LOG.info("uri="+uriBuilder.toString());
>         List queryParameters = uriBuilder.getQueryParams();
>         queryParameters.add(new BasicNameValuePair("Greeting", "hello"));
>         uriBuilder.setParameters(queryParameters);
>         LOG.info("uri="+uriBuilder.toString());



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2341) DefaultRedirect strategy breaks reserved chars in URI path

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17882062#comment-17882062
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2341:
---

[~xavierb] 

1. HttpClient 4.x conforms to RFC 2396 only. Whatever RFC 3986 says is of no 
relevance here. 
2. We are not going to change the existing behavior of HttpClient 4.x. It is 
not worth it.

Please upgrade to HttpClient 5.x that conforms to RFC 3986, retest and re-open 
this issue if you can reproduce the same issue with HttpClient 5.x

Oleg

> DefaultRedirect strategy breaks reserved chars in URI path
> --
>
> Key: HTTPCLIENT-2341
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2341
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14
> Environment: httpclient4 (4.5.14)
> Linux/Ubuntu 22.04
>Reporter: Xavier BOURGOUIN
>Priority: Major
> Attachments: hc4normalize.tar.gz
>
>
> When an HTTP response has an URI in the Location header with percent-encoded 
> reserved chars (such as %40), these chars are replaced by their normalized 
> equivalent (which is "@" in the case of %40), which seems to contradict RFC 
> 3986 ([https://www.rfc-editor.org/rfc/rfc3986#section-2.2] ), at least in the 
> sense that for such reserved characters, their percent-encoded value doesn't 
> have the same semantic meaning and thus aren't to be interpreted as 
> equivalent.
> One of the impacts is that it breaks any server / API that redirect clients 
> to a S3 blob object (AWS S3 for instance) that would happen to contain a %40 
> in the URI path (ex: location: https:/// container>/foo%40bar.file)
> Disabling URI normalization as show below seems to workaround it:
> {code:java}
> new 
> HttpGet("http://service-that-redirects";).setConfig(RequestConfig.custom().setNormalizeUri(false).build())
>  {code}
> However I'm not sure that's satisfying, if, as we suspect above, it is just 
> always wrong to "normalize" those reserved characters (plus it is enabled by 
> default).
> Note that httpclient5 is fine (the percent-encoded %40 is preserved as it 
> should, and it seems there's no more toggle for the normalization behavior 
> anyways).
> Comparing httpclient 4.x vs 5.x, it seems the URI normalization utility isn't 
> the same, which might explain why httpclient5 has no issue: 
> https://github.com/apache/httpcomponents-client/blob/4.5.x/httpclient/src/main/java/org/apache/http/impl/client/DefaultRedirectStrategy.java#L163
> https://github.com/apache/httpcomponents-client/blob/5.3.x/httpclient5/src/main/java/org/apache/hc/client5/http/impl/DefaultRedirectStrategy.java#L116
> (org.apache.http.client.utils.URIUtils.normalize() for HC4, versus 
> java.net.URI.normalize for HC5)
>  
> This past ticket https://issues.apache.org/jira/browse/HTTPCLIENT-2271 was 
> discussing something very similar, except it was the other way around: some 
> reserved characters were replaced by their percent-encoded equivalent. 
> However in the the lengthy comment thread there, it seems a consensus was 
> finally reach that for such chars, their percent-encoded value aren't 
> equivalent to their original value and thus shouldn't be transformed. So I 
> believe that reasoning should be bijective, and should also apply to the case 
> reported here.
> I worked out a reproducer in the form of a little maven project that I'm 
> attaching to this ticket, inspired from the one of that other ticket, that 
> demo the issue for httpclient 4.5.14 (but probably all 4.x is the same), and 
> compares it with httpclient5 (5.3.1). It should run directly with _mvn 
> exec:java_ and hopefully the output and code content are clear enough to be 
> self-explanatory.
>  
> In essence what it does is :
>  * Start a dummy http server with two services: */foo* that redirect to 
> */foo%40bar* and one that listen on *foo@bar* and reply with HTTP 200.
>  * Test httpclient4 (along with some other clients to demonstrate the 
> differences in behavior) by sending some GET request toward */foo* and 
> observe if and how it follows the redirect toward {*}/foo@bar{*}, which thus 
> allows to observe whether *%40* was replaced by *@*
>  
> {code:java}
> // Dummy server
> public static void main(String[] args) throws IOException, 
> InterruptedException {
> HttpServer server = HttpServer.create(new InetSocketAddress(8000), 0);
> server.createContext("/foo", new RedirectHttpHandler());
> server.createContext("/foo@bar", new SuccessHttpHandler());
> server.setExecutor(null);
> server.start();
> server.stop(0);
>
>// [... test client requets]
> }
> public static class RedirectHttpHandler implements HttpHandler {
>   

[jira] [Resolved] (HTTPCLIENT-2341) DefaultRedirect strategy breaks reserved chars in URI path

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2341.
---
Resolution: Information Provided

> DefaultRedirect strategy breaks reserved chars in URI path
> --
>
> Key: HTTPCLIENT-2341
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2341
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14
> Environment: httpclient4 (4.5.14)
> Linux/Ubuntu 22.04
>Reporter: Xavier BOURGOUIN
>Priority: Major
> Attachments: hc4normalize.tar.gz
>
>
> When an HTTP response has an URI in the Location header with percent-encoded 
> reserved chars (such as %40), these chars are replaced by their normalized 
> equivalent (which is "@" in the case of %40), which seems to contradict RFC 
> 3986 ([https://www.rfc-editor.org/rfc/rfc3986#section-2.2] ), at least in the 
> sense that for such reserved characters, their percent-encoded value doesn't 
> have the same semantic meaning and thus aren't to be interpreted as 
> equivalent.
> One of the impacts is that it breaks any server / API that redirect clients 
> to a S3 blob object (AWS S3 for instance) that would happen to contain a %40 
> in the URI path (ex: location: https:/// container>/foo%40bar.file)
> Disabling URI normalization as show below seems to workaround it:
> {code:java}
> new 
> HttpGet("http://service-that-redirects";).setConfig(RequestConfig.custom().setNormalizeUri(false).build())
>  {code}
> However I'm not sure that's satisfying, if, as we suspect above, it is just 
> always wrong to "normalize" those reserved characters (plus it is enabled by 
> default).
> Note that httpclient5 is fine (the percent-encoded %40 is preserved as it 
> should, and it seems there's no more toggle for the normalization behavior 
> anyways).
> Comparing httpclient 4.x vs 5.x, it seems the URI normalization utility isn't 
> the same, which might explain why httpclient5 has no issue: 
> https://github.com/apache/httpcomponents-client/blob/4.5.x/httpclient/src/main/java/org/apache/http/impl/client/DefaultRedirectStrategy.java#L163
> https://github.com/apache/httpcomponents-client/blob/5.3.x/httpclient5/src/main/java/org/apache/hc/client5/http/impl/DefaultRedirectStrategy.java#L116
> (org.apache.http.client.utils.URIUtils.normalize() for HC4, versus 
> java.net.URI.normalize for HC5)
>  
> This past ticket https://issues.apache.org/jira/browse/HTTPCLIENT-2271 was 
> discussing something very similar, except it was the other way around: some 
> reserved characters were replaced by their percent-encoded equivalent. 
> However in the the lengthy comment thread there, it seems a consensus was 
> finally reach that for such chars, their percent-encoded value aren't 
> equivalent to their original value and thus shouldn't be transformed. So I 
> believe that reasoning should be bijective, and should also apply to the case 
> reported here.
> I worked out a reproducer in the form of a little maven project that I'm 
> attaching to this ticket, inspired from the one of that other ticket, that 
> demo the issue for httpclient 4.5.14 (but probably all 4.x is the same), and 
> compares it with httpclient5 (5.3.1). It should run directly with _mvn 
> exec:java_ and hopefully the output and code content are clear enough to be 
> self-explanatory.
>  
> In essence what it does is :
>  * Start a dummy http server with two services: */foo* that redirect to 
> */foo%40bar* and one that listen on *foo@bar* and reply with HTTP 200.
>  * Test httpclient4 (along with some other clients to demonstrate the 
> differences in behavior) by sending some GET request toward */foo* and 
> observe if and how it follows the redirect toward {*}/foo@bar{*}, which thus 
> allows to observe whether *%40* was replaced by *@*
>  
> {code:java}
> // Dummy server
> public static void main(String[] args) throws IOException, 
> InterruptedException {
> HttpServer server = HttpServer.create(new InetSocketAddress(8000), 0);
> server.createContext("/foo", new RedirectHttpHandler());
> server.createContext("/foo@bar", new SuccessHttpHandler());
> server.setExecutor(null);
> server.start();
> server.stop(0);
>
>// [... test client requets]
> }
> public static class RedirectHttpHandler implements HttpHandler {
> @Override
> public void handle(HttpExchange t) throws IOException {
> t.getResponseHeaders().add("Location", "/foo%40bar");
> t.sendResponseHeaders(302, 0);
> OutputStream os = t.getResponseBody();
> os.close();
> }
> }
> 
> public static class SuccessHttpHandler implements HttpHandler {
> 

[jira] [Resolved] (HTTPCLIENT-2338) Accept-Encoding lost on request retrying

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2338.
---
Resolution: Fixed

> Accept-Encoding lost on request retrying
> 
>
> Key: HTTPCLIENT-2338
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2338
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.3.1
>Reporter: Michał Krysztofik
>Priority: Minor
> Fix For: 5.3.2, 5.4-beta2
>
>
> Make an HTTP request that is retried (e.g. response status 429). The HTTP 
> headers of the retried request does not include an `Accept-Encoding` header.
> {code}
> import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
> import org.apache.hc.client5.http.impl.classic.HttpClients;
> import org.apache.hc.client5.http.protocol.HttpClientContext;
> import org.apache.hc.core5.http.ClassicHttpRequest;
> import org.apache.hc.core5.http.impl.io.DefaultClassicHttpRequestFactory;
> import org.junit.jupiter.api.Test;
> import org.mockserver.integration.ClientAndServer;
> import org.mockserver.model.HttpRequest;
> import org.mockserver.socket.PortFactory;
> import java.io.IOException;
> import java.io.UncheckedIOException;
> import static org.assertj.core.api.Assertions.assertThat;
> import static org.mockserver.model.HttpRequest.request;
> import static org.mockserver.model.HttpResponse.response;
> public class ApacheTest {
>   @Test
>   void repeatedRequestHasNoAcceptEncodingHeader() {
> int serverPort = PortFactory.findFreePort();
> String serverUrl = "http://localhost:"; + serverPort;
> ClientAndServer server = ClientAndServer.startClientAndServer(serverPort);
> server.when(request()).respond(response().withStatusCode(429));
> try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
>   ClassicHttpRequest request = 
> DefaultClassicHttpRequestFactory.INSTANCE.newHttpRequest("GET", serverUrl);
>   httpClient.execute(request, HttpClientContext.create(), r -> r);
> } catch (IOException e) {
>   throw new UncheckedIOException(e.getMessage(), e);
> }
> HttpRequest[] sentRequests = server.retrieveRecordedRequests(null);
> assertThat(sentRequests).satisfiesExactly(
>   first -> assertThat(first.getHeaderList()).extracting(h -> 
> h.getName().getValue())
> .containsExactlyInAnyOrder("content-length", "Connection", 
> "User-Agent", "Host", "Accept-Encoding"),
>   second -> assertThat(second.getHeaderList()).extracting(h -> 
> h.getName().getValue())
> .containsExactlyInAnyOrder("content-length", "Connection", 
> "User-Agent", "Host")
> );
>   }
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Moved] (HTTPCORE-770) org.apache.hc.core5.net.URIBuilder - Allow to configure plusAsBlank

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski moved HTTPCLIENT-2340 to HTTPCORE-770:


  Component/s: HttpCore
   (was: HttpClient (classic))
  Key: HTTPCORE-770  (was: HTTPCLIENT-2340)
Affects Version/s: 5.3
   (was: 5.2.3)
 Workflow: classic default workflow  (was: Default workflow, 
editable Closed status)
  Project: HttpComponents HttpCore  (was: HttpComponents HttpClient)

> org.apache.hc.core5.net.URIBuilder - Allow to configure plusAsBlank
> ---
>
> Key: HTTPCORE-770
> URL: https://issues.apache.org/jira/browse/HTTPCORE-770
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.3
>Reporter: Claus Ibsen
>Priority: Minor
>
> When upgrading from Http Client v4 to v5 then the URIBuilder has changed a 
> default value for the plusAsBlank option.
>  
> In v4 the default is true
>  
> org.apache.http.client.utils.URLEncodedUtils.decodeFormFields()
> ~~~
> return urlDecode(content, charset != null ? charset : Consts.UTF_8, true);
> ~~~
> In v5 the default is changed to false
> org.apache.hc.core5.net.URIBuilder.digestURI()
> ~~~
> this.queryParams = parseQuery(uri.getRawQuery(), charset, false);
> ~~~
>  
> I wonder if the URIBuilder could have some kind of flag that can be set to 
> set whether to use true or false.
>  
> import org.apache.hc.core5.http.NameValuePair;
> import org.apache.hc.core5.http.message.BasicNameValuePair;
> import org.apache.hc.core5.net.URIBuilder;
>  
>         URIBuilder uriBuilder;
>         uriBuilder = new 
> URIBuilder("[http://dummy/demo+demo?sample=sample+sample]";);
>         LOG.info("uri="+uriBuilder.toString());
>         List queryParameters = uriBuilder.getQueryParams();
>         queryParameters.add(new BasicNameValuePair("Greeting", "hello"));
>         uriBuilder.setParameters(queryParameters);
>         LOG.info("uri="+uriBuilder.toString());



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCORE-770) org.apache.hc.core5.net.URIBuilder - Allow to configure plusAsBlank

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCORE-770:
---
Fix Version/s: 5.4-alpha1

> org.apache.hc.core5.net.URIBuilder - Allow to configure plusAsBlank
> ---
>
> Key: HTTPCORE-770
> URL: https://issues.apache.org/jira/browse/HTTPCORE-770
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.3
>Reporter: Claus Ibsen
>Priority: Minor
> Fix For: 5.4-alpha1
>
>
> When upgrading from Http Client v4 to v5 then the URIBuilder has changed a 
> default value for the plusAsBlank option.
>  
> In v4 the default is true
>  
> org.apache.http.client.utils.URLEncodedUtils.decodeFormFields()
> ~~~
> return urlDecode(content, charset != null ? charset : Consts.UTF_8, true);
> ~~~
> In v5 the default is changed to false
> org.apache.hc.core5.net.URIBuilder.digestURI()
> ~~~
> this.queryParams = parseQuery(uri.getRawQuery(), charset, false);
> ~~~
>  
> I wonder if the URIBuilder could have some kind of flag that can be set to 
> set whether to use true or false.
>  
> import org.apache.hc.core5.http.NameValuePair;
> import org.apache.hc.core5.http.message.BasicNameValuePair;
> import org.apache.hc.core5.net.URIBuilder;
>  
>         URIBuilder uriBuilder;
>         uriBuilder = new 
> URIBuilder("[http://dummy/demo+demo?sample=sample+sample]";);
>         LOG.info("uri="+uriBuilder.toString());
>         List queryParameters = uriBuilder.getQueryParams();
>         queryParameters.add(new BasicNameValuePair("Greeting", "hello"));
>         uriBuilder.setParameters(queryParameters);
>         LOG.info("uri="+uriBuilder.toString());



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCORE-769) Force close on connection when 408 is returned

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCORE-769:
---
Fix Version/s: Stuck
   Labels: volunteers-wanted  (was: )
 Priority: Minor  (was: Major)

> Force close on connection when 408 is returned
> --
>
> Key: HTTPCORE-769
> URL: https://issues.apache.org/jira/browse/HTTPCORE-769
> Project: HttpComponents HttpCore
>  Issue Type: Bug
>  Components: HttpCore
>Affects Versions: 5.2.5, 5.3-beta1
>Reporter: Patrick Barry
>Priority: Minor
>  Labels: volunteers-wanted
> Fix For: Stuck
>
>
> Regarding http1.1 requests We have a service we are reaching out to, that 
> is sending back a 408, but no connection: close.  
> According to [https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/408]
>  
> {noformat}
> The HTTP 408 Request Timeout client error response status code indicates that 
> the server would like to shut down this unused connection. A 408 is sent on 
> an idle connection by some servers, even without any previous request by the 
> client.
> A server should send the Connection: close header field in the response, 
> since 408 implies that the server has decided to close the connection rather 
> than continue waiting.
> {noformat}
>  
> We are using the CloseableHttpAsyncClient with 
> PoolingAsyncClientConnectionManager.  When we get a 408 and they do not 
> specify that connection should be closed, how can we force this? Can it be 
> accomplished with an interceptor or is it done by default inside the library? 
>  The response is actually sending back Connection=keep-alive
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Moved] (HTTPCORE-769) Force close on connection when 408 is returned

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski moved HTTPCLIENT-2339 to HTTPCORE-769:


  Component/s: HttpCore
   (was: HttpClient (async))
  Key: HTTPCORE-769  (was: HTTPCLIENT-2339)
Affects Version/s: 5.3-beta1
   5.2.5
   (was: 5.3.1)
 Workflow: classic default workflow  (was: Default workflow, 
editable Closed status)
  Project: HttpComponents HttpCore  (was: HttpComponents HttpClient)

> Force close on connection when 408 is returned
> --
>
> Key: HTTPCORE-769
> URL: https://issues.apache.org/jira/browse/HTTPCORE-769
> Project: HttpComponents HttpCore
>  Issue Type: Bug
>  Components: HttpCore
>Affects Versions: 5.3-beta1, 5.2.5
>Reporter: Patrick Barry
>Priority: Major
>
> Regarding http1.1 requests We have a service we are reaching out to, that 
> is sending back a 408, but no connection: close.  
> According to [https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/408]
>  
> {noformat}
> The HTTP 408 Request Timeout client error response status code indicates that 
> the server would like to shut down this unused connection. A 408 is sent on 
> an idle connection by some servers, even without any previous request by the 
> client.
> A server should send the Connection: close header field in the response, 
> since 408 implies that the server has decided to close the connection rather 
> than continue waiting.
> {noformat}
>  
> We are using the CloseableHttpAsyncClient with 
> PoolingAsyncClientConnectionManager.  When we get a 408 and they do not 
> specify that connection should be closed, how can we force this? Can it be 
> accomplished with an interceptor or is it done by default inside the library? 
>  The response is actually sending back Connection=keep-alive
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2339) Force close on connection when 408 is returned

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881032#comment-17881032
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2339:
---

[~patrickjamesbarry] I did not see a requirement to close the connection based 
on the status code alone in RFC 9110 [1]. I do not even see a recommendation. 
It is entirely up to you to treat this issue as a bug or a change request. At 
any rate it will not warrant an emergency release and you have just missed 
HttpCore 4.3 by a few days.

Oleg

[1] https://www.rfc-editor.org/rfc/rfc9110.html#name-408-request-timeout

> Force close on connection when 408 is returned
> --
>
> Key: HTTPCLIENT-2339
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2339
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>  Components: HttpClient (async)
>Affects Versions: 5.3.1
>Reporter: Patrick Barry
>Priority: Major
>
> Regarding http1.1 requests We have a service we are reaching out to, that 
> is sending back a 408, but no connection: close.  
> According to [https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/408]
>  
> {noformat}
> The HTTP 408 Request Timeout client error response status code indicates that 
> the server would like to shut down this unused connection. A 408 is sent on 
> an idle connection by some servers, even without any previous request by the 
> client.
> A server should send the Connection: close header field in the response, 
> since 408 implies that the server has decided to close the connection rather 
> than continue waiting.
> {noformat}
>  
> We are using the CloseableHttpAsyncClient with 
> PoolingAsyncClientConnectionManager.  When we get a 408 and they do not 
> specify that connection should be closed, how can we force this? Can it be 
> accomplished with an interceptor or is it done by default inside the library? 
>  The response is actually sending back Connection=keep-alive
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2339) Force close on connection when 408 is returned

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881022#comment-17881022
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2339:
---

[~patrickjamesbarry] I suppose all it takes is a custom 
`{color:#00}ConnectionReuseStrategy` or a small tweak to 
`{color}{color:#00}DefaultConnectionReuseStrategy` in HttpCore.{color}

{color:#00}Oleg{color}

{color:#00}`
{color}

`

> Force close on connection when 408 is returned
> --
>
> Key: HTTPCLIENT-2339
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2339
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>  Components: HttpClient (async)
>Affects Versions: 5.3.1
>Reporter: Patrick Barry
>Priority: Major
>
> Regarding http1.1 requests We have a service we are reaching out to, that 
> is sending back a 408, but no connection: close.  
> According to [https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/408]
>  
> {noformat}
> The HTTP 408 Request Timeout client error response status code indicates that 
> the server would like to shut down this unused connection. A 408 is sent on 
> an idle connection by some servers, even without any previous request by the 
> client.
> A server should send the Connection: close header field in the response, 
> since 408 implies that the server has decided to close the connection rather 
> than continue waiting.
> {noformat}
>  
> We are using the CloseableHttpAsyncClient with 
> PoolingAsyncClientConnectionManager.  When we get a 408 and they do not 
> specify that connection should be closed, how can we force this? Can it be 
> accomplished with an interceptor or is it done by default inside the library? 
>  The response is actually sending back Connection=keep-alive
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2338) Accept-Encoding lost on request retrying

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17880576#comment-17880576
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2338:
---

[~michu4k] Please review / test the proposed fix:

[https://github.com/apache/httpcomponents-client/compare/5.3.x...HTTPCLIENT-2338]

Oleg

> Accept-Encoding lost on request retrying
> 
>
> Key: HTTPCLIENT-2338
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2338
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.3.1
>Reporter: Michał Krysztofik
>Priority: Minor
> Fix For: 5.3.2, 5.4-beta2
>
>
> Make an HTTP request that is retried (e.g. response status 429). The HTTP 
> headers of the retried request does not include an `Accept-Encoding` header.
> {code}
> import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
> import org.apache.hc.client5.http.impl.classic.HttpClients;
> import org.apache.hc.client5.http.protocol.HttpClientContext;
> import org.apache.hc.core5.http.ClassicHttpRequest;
> import org.apache.hc.core5.http.impl.io.DefaultClassicHttpRequestFactory;
> import org.junit.jupiter.api.Test;
> import org.mockserver.integration.ClientAndServer;
> import org.mockserver.model.HttpRequest;
> import org.mockserver.socket.PortFactory;
> import java.io.IOException;
> import java.io.UncheckedIOException;
> import static org.assertj.core.api.Assertions.assertThat;
> import static org.mockserver.model.HttpRequest.request;
> import static org.mockserver.model.HttpResponse.response;
> public class ApacheTest {
>   @Test
>   void repeatedRequestHasNoAcceptEncodingHeader() {
> int serverPort = PortFactory.findFreePort();
> String serverUrl = "http://localhost:"; + serverPort;
> ClientAndServer server = ClientAndServer.startClientAndServer(serverPort);
> server.when(request()).respond(response().withStatusCode(429));
> try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
>   ClassicHttpRequest request = 
> DefaultClassicHttpRequestFactory.INSTANCE.newHttpRequest("GET", serverUrl);
>   httpClient.execute(request, HttpClientContext.create(), r -> r);
> } catch (IOException e) {
>   throw new UncheckedIOException(e.getMessage(), e);
> }
> HttpRequest[] sentRequests = server.retrieveRecordedRequests(null);
> assertThat(sentRequests).satisfiesExactly(
>   first -> assertThat(first.getHeaderList()).extracting(h -> 
> h.getName().getValue())
> .containsExactlyInAnyOrder("content-length", "Connection", 
> "User-Agent", "Host", "Accept-Encoding"),
>   second -> assertThat(second.getHeaderList()).extracting(h -> 
> h.getName().getValue())
> .containsExactlyInAnyOrder("content-length", "Connection", 
> "User-Agent", "Host")
> );
>   }
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCLIENT-2337) Potentially unsafe logging of X500Principal in SSLConnectionSocketFactory

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCLIENT-2337:
--
Fix Version/s: Stuck
   (was: 5.4-beta2)
   Labels: volunteers-wanted  (was: )
 Priority: Trivial  (was: Major)

Volunteers are welcome to contribute a patch for this issue.

Oleg

> Potentially unsafe logging of X500Principal in SSLConnectionSocketFactory
> -
>
> Key: HTTPCLIENT-2337
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2337
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>Affects Versions: 4.5.14, 5.3.1, 5.4-beta1
>Reporter: Winfried Gerlach
>Priority: Trivial
>  Labels: volunteers-wanted
> Fix For: Stuck
>
> Attachments: example-cert.pem, image-2024-09-03-08-43-06-757.png
>
>
> We noticed that in both Apache HTTP Client 4.x and 5.x, 
> {{SSLConnectionSocketFactory}} logs the X500Principal on DEBUG level without 
> sanitizing the fields. If, e.g., the CN contains control characters like 
> {{\b}} or {{{}\n{}}}, this could be used by an attacker to tamper with the 
> log of the application (remove stuff, add line breaks etc.).
> !image-2024-09-03-08-43-06-757.png!
> In the screenshot, the CN has a \b after "Control", so the last letter "l" is 
> removed from the log.
> We don't consider this behavior particularly dangerous because it happens on 
> debug level only and the logger can also be turned off completely if needed.
> You may still want to think about sanitizing the RDN values before logging or 
> somehow avoiding to log the X500Principal completely.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2337) Potentially unsafe logging of X500Principal in SSLConnectionSocketFactory

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17879534#comment-17879534
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2337:
---

[~michael-o] [~ckozak] My preference would be 
(1) close the ticket as WONTFIX and do nothing (we do not sanitize message 
headers in DEBUG priority messages either and no one died as far as I know)
(2) poo man's solution

Oleg

> Potentially unsafe logging of X500Principal in SSLConnectionSocketFactory
> -
>
> Key: HTTPCLIENT-2337
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2337
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>Affects Versions: 4.5.14, 5.3.1, 5.4-beta1
>Reporter: Winfried Gerlach
>Priority: Major
> Fix For: 5.4-beta2
>
> Attachments: example-cert.pem, image-2024-09-03-08-43-06-757.png
>
>
> We noticed that in both Apache HTTP Client 4.x and 5.x, 
> {{SSLConnectionSocketFactory}} logs the X500Principal on DEBUG level without 
> sanitizing the fields. If, e.g., the CN contains control characters like 
> {{\b}} or {{{}\n{}}}, this could be used by an attacker to tamper with the 
> log of the application (remove stuff, add line breaks etc.).
> !image-2024-09-03-08-43-06-757.png!
> In the screenshot, the CN has a \b after "Control", so the last letter "l" is 
> removed from the log.
> We don't consider this behavior particularly dangerous because it happens on 
> debug level only and the logger can also be turned off completely if needed.
> You may still want to think about sanitizing the RDN values before logging or 
> somehow avoiding to log the X500Principal completely.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2337) Potentially unsafe logging of X500Principal in SSLConnectionSocketFactory

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17878838#comment-17878838
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2337:
---

[~winfriedgerlach] Feel free to propose a fix by raising a PR at GitHub.

Oleg

> Potentially unsafe logging of X500Principal in SSLConnectionSocketFactory
> -
>
> Key: HTTPCLIENT-2337
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2337
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>Affects Versions: 4.5.14, 5.3.1, 5.4-beta1
>Reporter: Winfried Gerlach
>Priority: Major
> Fix For: 5.4-beta2
>
> Attachments: image-2024-09-03-08-43-06-757.png
>
>
> We noticed that in both Apache HTTP Client 4.x and 5.x, 
> {{SSLConnectionSocketFactory}} logs the X500Principal on DEBUG level without 
> sanitizing the fields. If, e.g., the CN contains control characters like 
> {{\b}} or {{\n}}, this could be used by an attacker to tamper with the log of 
> the application (remove stuff, add line breaks etc.).
> !image-2024-09-03-08-43-06-757.png!
> In the screenshot, the CN has a \b after "Control", so the last letter "l" is 
> removed from the log.
> We don't consider this behavior particularly dangerous because it happens on 
> debug level only and the logger can also be turned off completely if needed.
> You may still want to think about sanitizing the RDN values before logging or 
> somehow avoid ing to log the X500Principal completely.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCLIENT-2337) Potentially unsafe logging of X500Principal in SSLConnectionSocketFactory

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCLIENT-2337:
--
Fix Version/s: 5.4-beta2

> Potentially unsafe logging of X500Principal in SSLConnectionSocketFactory
> -
>
> Key: HTTPCLIENT-2337
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2337
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>Affects Versions: 4.5.14, 5.3.1, 5.4-beta1
>Reporter: Winfried Gerlach
>Priority: Major
> Fix For: 5.4-beta2
>
> Attachments: image-2024-09-03-08-43-06-757.png
>
>
> We noticed that in both Apache HTTP Client 4.x and 5.x, 
> {{SSLConnectionSocketFactory}} logs the X500Principal on DEBUG level without 
> sanitizing the fields. If, e.g., the CN contains control characters like 
> {{\b}} or {{\n}}, this could be used by an attacker to tamper with the log of 
> the application (remove stuff, add line breaks etc.).
> !image-2024-09-03-08-43-06-757.png!
> In the screenshot, the CN has a \b after "Control", so the last letter "l" is 
> removed from the log.
> We don't consider this behavior particularly dangerous because it happens on 
> debug level only and the logger can also be turned off completely if needed.
> You may still want to think about sanitizing the RDN values before logging or 
> somehow avoid ing to log the X500Principal completely.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCLIENT-2337) Potentially unsafe logging of X500Principal in SSLConnectionSocketFactory

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCLIENT-2337:
--
Affects Version/s: (was: 5.0)
   (was: 4.3.5.1-android)
   (was: 4.4.1)
   (was: 4.5)
   (was: 4.5.1)
   (was: 4.5.2)
   (was: 5.0 Alpha2)
   (was: 4.5.3)
   (was: 4.5.4)
   (was: 5.0 Alpha3)
   (was: 5.0 Beta1)
   (was: 4.5.5)
   (was: 5.0 Beta2)
   (was: 4.5.6)
   (was: 4.5.7)
   (was: 5.0 Beta3)
   (was: 5.0 Beta4)
   (was: 4.5.8)
   (was: 4.5.9)
   (was: 5.0 Beta5)
   (was: 4.5.10)
   (was: 5.0 Beta6)
   (was: 4.5.11)
   (was: 5.0 Beta7)
   (was: 4.5.12)
   (was: 5.1-beta1)
   (was: 5.0.1)
   (was: 4.5.13)
   (was: 5.0.2)
   (was: 5.0.3)
   (was: 5.0.4)
   (was: 5.1)
   (was: 5.2-alpha1)
   (was: 5.1.1)
   (was: 5.2-beta1)
   (was: 5.1.2)
   (was: 5.1.3)
   (was: 5.1.4)
   (was: 5.2)
   (was: 5.2.1)
   (was: 5.3-alpha1)
   (was: 4.5.15)
   (was: 5.2.2)
   (was: 5.3)
   (was: 5.4-alpha1)
   (was: 5.2.3)
   (was: 5.4-alpha2)
   (was: 5.3.2)
   (was: 5.4-beta2)

> Potentially unsafe logging of X500Principal in SSLConnectionSocketFactory
> -
>
> Key: HTTPCLIENT-2337
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2337
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>Affects Versions: 4.5.14, 5.3.1, 5.4-beta1
>Reporter: Winfried Gerlach
>Priority: Major
> Attachments: image-2024-09-03-08-43-06-757.png
>
>
> We noticed that in both Apache HTTP Client 4.x and 5.x, 
> {{SSLConnectionSocketFactory}} logs the X500Principal on DEBUG level without 
> sanitizing the fields. If, e.g., the CN contains control characters like 
> {{\b}} or {{\n}}, this could be used by an attacker to tamper with the log of 
> the application (remove stuff, add line breaks etc.).
> !image-2024-09-03-08-43-06-757.png!
> In the screenshot, the CN has a \b after "Control", so the last letter "l" is 
> removed from the log.
> We don't consider this behavior particularly dangerous because it happens on 
> debug level only and the logger can also be turned off completely if needed.
> You may still want to think about sanitizing the RDN values before logging or 
> somehow avoid ing to log the X500Principal completely.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-1625) Completely overhaul GSS-API-based authentication backend

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-1625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17878106#comment-17878106
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-1625:
---

> I have tried to create a WIP PR for easier review, but GH didn't let me.

[~stoty] At some point of time one will have to rebase that branch off master 
and create a feature branch from it. That feature branch will eventually be 
merged back to master sometime past 5.4 GA.

Oleg

> Completely overhaul GSS-API-based authentication backend
> 
>
> Key: HTTPCLIENT-1625
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1625
> Project: HttpComponents HttpClient
>  Issue Type: Task
>  Components: Documentation, HttpClient (classic)
>Affects Versions: 4.5
>Reporter: Michael Osipov
>Priority: Major
>  Labels: stuck, volunteers-wanted
>
> The current implementation does not reflect the way GSS-API-based 
> authentication should be done. It has several design flaws.
> This is an umbrella task for:
> 1. Deprecate all old classes
> 2. Investigate how it has to be plugged into HttpClient
> 3. Reimplement from scratch
> 4. Thoroughly test all new stuff
> 5. Rewrite documentation
> Design notes are canonically available under: 
> https://wiki.apache.org/HttpComponents/IssueTracking/HTTPCLIENT-1625



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPASYNC-172) Requests in AbstractNIOConnPool.pending stays forever.

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPASYNC-172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877361#comment-17877361
 ] 

Oleg Kalnichevski commented on HTTPASYNC-172:
-

> upgrade to HttpClient 5.3. can resolved？

[~xjhnlnl] I cannot say as the original reporter did not provide a reproducer. 
See  HTTPASYNC-109. 

If you however can reproduce the problem with HttpClient 5.3 I will certainly 
look into it.

Oleg

> Requests in AbstractNIOConnPool.pending stays forever.
> --
>
> Key: HTTPASYNC-172
> URL: https://issues.apache.org/jira/browse/HTTPASYNC-172
> Project: HttpComponents HttpAsyncClient
>  Issue Type: Bug
>Affects Versions: 4.1.3
>Reporter: xjhnlnl
>Priority: Blocker
>
> My situation is the same as the previous issue，This is the previous link 
> https://issues.apache.org/jira/projects/HTTPASYNC/issues/HTTPASYNC-109?filter=recentlyviewed
> The phenomena are exactly the same。This problem is that my program is 
> suspended because it cannot get the link from the connection pool.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPASYNC-172) Requests in AbstractNIOConnPool.pending stays forever.

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPASYNC-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPASYNC-172.
-
Resolution: Duplicate

[~xjhnlnl] HttpAyncClient 4.x is at end-of-life and is no longer supported. 
Please upgrade to HttpClient 5.3.

Oleg

> Requests in AbstractNIOConnPool.pending stays forever.
> --
>
> Key: HTTPASYNC-172
> URL: https://issues.apache.org/jira/browse/HTTPASYNC-172
> Project: HttpComponents HttpAsyncClient
>  Issue Type: Bug
>Affects Versions: 4.1.3
>Reporter: xjhnlnl
>Priority: Blocker
>
> My situation is the same as the previous issue，This is the previous link 
> https://issues.apache.org/jira/projects/HTTPASYNC/issues/HTTPASYNC-109?filter=recentlyviewed
> The phenomena are exactly the same。This problem is that my program is 
> suspended because it cannot get the link from the connection pool.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-1625) Completely overhaul GSS-API-based authentication backend

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-1625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876640#comment-17876640
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-1625:
---

> You mean that if the mutual authentication fails locally, then we should 
> generate a synthetic 401/407 response and return that ?

[~stoty] No, I do not. I do not exactly know what "authentication fails 
locally" means exactly, but presumably it is a configuration / unexpected 
context type of issue. In this case an exception should suffice. In case when 
the server returns 401, 407 or any other status for that matter, the 
authentication scheme should propagate the response to the caller instead of 
throwing an exception.

Oleg

> Completely overhaul GSS-API-based authentication backend
> 
>
> Key: HTTPCLIENT-1625
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1625
> Project: HttpComponents HttpClient
>  Issue Type: Task
>  Components: Documentation, HttpClient (classic)
>Affects Versions: 4.5
>Reporter: Michael Osipov
>Priority: Major
>  Labels: stuck, volunteers-wanted
>
> The current implementation does not reflect the way GSS-API-based 
> authentication should be done. It has several design flaws.
> This is an umbrella task for:
> 1. Deprecate all old classes
> 2. Investigate how it has to be plugged into HttpClient
> 3. Reimplement from scratch
> 4. Thoroughly test all new stuff
> 5. Rewrite documentation
> Design notes are canonically available under: 
> https://wiki.apache.org/HttpComponents/IssueTracking/HTTPCLIENT-1625



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-1625) Completely overhaul GSS-API-based authentication backend

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-1625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876408#comment-17876408
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-1625:
---

[~stoty] [~michael-o]  For the sake of consistency with the existing auth 
schemes it would be great that the new scheme

1. throws an exception in case of a protocol violation / misconfiguration / 
unexpected context

2. propagates 401status  in case of authentication failure (bad credentials) 
and if necessary populates additional details in the `HttpContext`

Oleg

> Completely overhaul GSS-API-based authentication backend
> 
>
> Key: HTTPCLIENT-1625
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1625
> Project: HttpComponents HttpClient
>  Issue Type: Task
>  Components: Documentation, HttpClient (classic)
>Affects Versions: 4.5
>Reporter: Michael Osipov
>Priority: Major
>  Labels: stuck, volunteers-wanted
>
> The current implementation does not reflect the way GSS-API-based 
> authentication should be done. It has several design flaws.
> This is an umbrella task for:
> 1. Deprecate all old classes
> 2. Investigate how it has to be plugged into HttpClient
> 3. Reimplement from scratch
> 4. Thoroughly test all new stuff
> 5. Rewrite documentation
> Design notes are canonically available under: 
> https://wiki.apache.org/HttpComponents/IssueTracking/HTTPCLIENT-1625



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCLIENT-2335) Closing chunked responses hangs

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2335.
---
Resolution: Fixed

> Closing chunked responses hangs
> ---
>
> Key: HTTPCLIENT-2335
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2335
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.3.1, 5.4-beta1
>Reporter: Joao Silva
>Priority: Critical
> Fix For: 5.4-beta2
>
>
> While migrating a proxy implementation from httpclient 4 to httpclient 5 we 
> ran into some interesting issues regarding chunked responses.
> There is a specific test that deals with the proxy behavior when the client 
> closes the connection halfway through the response is being served.
> Using chunked responses in httpclient 4, if we close the 
> {{{}CloseableHttpResponse{}}}, the connection closes without issue.
> However, in httpclient 5 the {{CloseableHttpResponse.close()}} hangs.
> Debugging the issue, we see that {{ChunkedInputStream.close()}} hangs when 
> called, because it tries to read from the socket, and the server is not 
> actively sending any data:
> "main@1" tid=0x1 nid=NA runnable
>   java.lang.Thread.State: RUNNABLE
>       at sun.nio.ch.Net.poll(Net.java:-1)
>       at sun.nio.ch.NioSocketImpl.park(NioSocketImpl.java:191)
>       at sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:280)
>       at sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
>       at sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
>       at sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
>       at java.net.Socket$SocketInputStream.read(Socket.java:1109)
>       at 
> org.apache.hc.client5.http.impl.io.LoggingInputStream.read(LoggingInputStream.java:81)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:248)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:147)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:314)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.impl.io.IncomingHttpEntity.close(IncomingHttpEntity.java:111)
>       at 
> org.apache.hc.core5.http.io.entity.HttpEntityWrapper.close(HttpEntityWrapper.java:120)
>       at 
> org.apache.hc.client5.http.impl.classic.ResponseEntityProxy.close(ResponseEntityProxy.java:180)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.message.BasicClassicHttpResponse.close(BasicClassicHttpResponse.java:93)
>       at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpResponse.close(CloseableHttpResponse.java:205)
>       at 
> org.mitre.dsmiley.httpproxy.ChunkedTransferTest.testChunkedTransferClosing(ChunkedTransferTest.java:233)
>  
> Comparing to httpclient 4, we see that ChunkedInputStream.close() 
> does not hang because when it tries to read from the socket, it is already 
> closed, hence it fails. The resulting SocketException is caught in 
> ResponseEntityProxy and discarded.
> Tested in versions 5.4-beta1, 5.3.1, 5.2.3, 5.1.4 and 5.0.4, and they all 
> hang in the same way.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCLIENT-2336) Update PublicSuffixMatcher to use "formal algorithm"

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2336.
---
Fix Version/s: 5.4-beta2
   (was: Future)
   Resolution: Fixed

> Update PublicSuffixMatcher to use "formal algorithm"
> 
>
> Key: HTTPCLIENT-2336
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2336
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>  Components: HttpClient (classic)
>Reporter: Mass Dosage
>Priority: Major
> Fix For: 5.4-beta2
>
>
> We’ve been using the {{PublicSuffixMatcher}} component from 
> “httpcomponents-client” to implement comparisons of domains and public 
> suffixes according to the rules defined by the standard at 
> [https://github.com/publicsuffix/list/wiki/Format#formal-algorithm] and using 
> the Public Suffix List produced by Mozilla at 
> [https://publicsuffix.org/list/effective_tld_names.dat].
> We noticed unexpected results where the current behaviour of 
> {{PublicSuffixMatcher}} deviates from the formal algorithm. We ported over 
> the unit tests from 
> [https://github.com/publicsuffix/list/blob/master/tests/test_psl.txt] to 
> determine all the differences as there are a number of tests which fail. This 
> follows on from an earlier discussion on the mailing list at 
> [https://lists.apache.org/thread/ylom7gcopxtrcb4zm6q8c9k7fo0jt5km].
> We have made changes to get all the tests to pass and will raise a PR shortly 
> to reference this ticket.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-1625) Completely overhaul GSS-API-based authentication backend

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-1625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17874514#comment-17874514
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-1625:
---

> As already noted, this would require some changes in the authentication code, 
> and the state machine defined in HttpAuthenticator, which may or may not 
> break API compatibility.

[~stoty] I will happily help you with that. As far the rest is concerned you 
are at [~michael-o] 's mercy.

> If it can be done without breaking the authentication API could it be added 
> in a patch release ?

No, it could not. This feature should ideally go though the full ALPHA / BETA / 
GA cycle.

> If not, how could this be delivered ?

I am afraid this is already too late for 5.4 which should go GA in September.

Oleg

 

> Completely overhaul GSS-API-based authentication backend
> 
>
> Key: HTTPCLIENT-1625
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1625
> Project: HttpComponents HttpClient
>  Issue Type: Task
>  Components: Documentation, HttpClient (classic)
>Affects Versions: 4.5
>Reporter: Michael Osipov
>Priority: Major
>  Labels: stuck, volunteers-wanted
>
> The current implementation does not reflect the way GSS-API-based 
> authentication should be done. It has several design flaws.
> This is an umbrella task for:
> 1. Deprecate all old classes
> 2. Investigate how it has to be plugged into HttpClient
> 3. Reimplement from scratch
> 4. Thoroughly test all new stuff
> 5. Rewrite documentation
> Design notes are canonically available under: 
> https://wiki.apache.org/HttpComponents/IssueTracking/HTTPCLIENT-1625



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2327) MemcachedHttpAsyncCacheStorage propagates CancellationException from cancelled cache requests to caller

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17873140#comment-17873140
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2327:
---

[~jattisha] I do not mind picking this change-set into 5.3.x but I cannot 
promise there will ever be 5.3.2 release.

Oleg

> MemcachedHttpAsyncCacheStorage propagates CancellationException from 
> cancelled cache requests to caller
> ---
>
> Key: HTTPCLIENT-2327
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2327
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpCache
>Reporter: Jovan Attisha
>Priority: Major
> Fix For: 5.4-beta1
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Spymemcached provides options to handle failure modes when an in-flight 
> request to a Memcached instance fails (for example, when deploying to a 
> Memcached server that is currently serving requests). One of the modes, and 
> the most logical for our scenario in particular, is 
> [Cancel|https://github.com/couchbase/spymemcached/blob/master/src/main/java/net/spy/memcached/FailureMode.java#L51-L54].
>  This, as you would expect, cancels any futures that are in-flight when a 
> Memcached node becomes unreachable.
>  
> The issue that we're having is that the 
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
>  (for restore linked here, as this is our issue, but also for other methods), 
> doesn't handle the {{CancellationException}} gracefully, and instead 
> propagates this Exception up to the caller. Since this is a cache for an HTTP 
> client, I would expect the client to handle the cancellation as a "cache 
> miss" (I would almost expect this behavior for {_}all exceptions{_}, though 
> I'm likely not thinking of some edge cases here) and proceed to make the 
> request to the resource originally requested, since the cache is a mere 
> optimization, and a failure in the cache should not impact the actual request 
> to the resource we're targeting.
>  
> We did POC this by extending the 
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
>  and overriding the {{restore}} method to explicitly handle 
> {{{}CancellationException{}}}s and complete the future to treat it as a cache 
> miss, but before we proceed with this, we want to understand if:
> 1. The Apache HTTPClient agrees that this is the desired behavior
> 2. If so, would the Apache HTTP Client accept this behavior in the client 
> itself, rather than us extending to override this behavior



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCORE-761) support params TCP_KEEPIDLE/TCP_KEEPINTERVAL/TCP_KEEPCOUNT in Socket

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCORE-761:
---
Fix Version/s: 5.3-beta2
   (was: 5.3-alpha1)

> support params TCP_KEEPIDLE/TCP_KEEPINTERVAL/TCP_KEEPCOUNT in Socket
> 
>
> Key: HTTPCORE-761
> URL: https://issues.apache.org/jira/browse/HTTPCORE-761
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Reporter: kkewwei
>Priority: Major
> Fix For: 5.3-beta2
>
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>
> It's known that TCP Keep-Alive time  is 7200 seconds(default), default 
> parameters do not meet all needs.
> If we could support setting TCP_KEEPIDLE, TCP_KEEPINTERVAL, TCP_KEEPCOUNT in 
> Socket.
> [SingleCoreIOReactor|https://github.com/apache/httpcomponents-core/blob/0d4aeb557737ac717e62c6375ca0d8a86f5f886b/httpcore5/src/main/java/org/apache/hc/core5/reactor/SingleCoreIOReactor.java#L268]
> {code:java}
> private void.prepareSocket(final Socket socket) throws IOException {
> socket.setTcpNoDelay(this.reactorConfig.isTcpNoDelay());
> socket.setKeepAlive(this.reactorConfig.isSoKeepAlive());
> if (this.reactorConfig.getSndBufSize() > 0) {
> socket.setSendBufferSize(this.reactorConfig.getSndBufSize());
> }
> if (this.reactorConfig.getRcvBufSize() > 0) {
> socket.setReceiveBufferSize(this.reactorConfig.getRcvBufSize());
> }
> if (this.reactorConfig.getTrafficClass() > 0) {
> socket.setTrafficClass(this.reactorConfig.getTrafficClass());
> }
> final int linger = 
> this.reactorConfig.getSoLinger().toSecondsIntBound();
> if (linger >= 0) {
> socket.setSoLinger(true, linger);
> }
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2335) Closing chunked responses hangs

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17872560#comment-17872560
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2335:
---

[~jpmsilva] Please review the proposed changes [1] and also build the latest 
5.4 snapshot [2] from source test it locally. One can now use `#close` with 
`CloseMode` parameter to close out responses either gracefully or immediately

Oleg

[1] https://github.com/apache/httpcomponents-client/compare/HTTPCLIENT-2335
[2] https://github.com/apache/httpcomponents-client

> Closing chunked responses hangs
> ---
>
> Key: HTTPCLIENT-2335
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2335
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.3.1, 5.4-beta1
>Reporter: Joao Silva
>Priority: Critical
> Fix For: 5.4-beta2
>
>
> While migrating a proxy implementation from httpclient 4 to httpclient 5 we 
> ran into some interesting issues regarding chunked responses.
> There is a specific test that deals with the proxy behavior when the client 
> closes the connection halfway through the response is being served.
> Using chunked responses in httpclient 4, if we close the 
> {{{}CloseableHttpResponse{}}}, the connection closes without issue.
> However, in httpclient 5 the {{CloseableHttpResponse.close()}} hangs.
> Debugging the issue, we see that {{ChunkedInputStream.close()}} hangs when 
> called, because it tries to read from the socket, and the server is not 
> actively sending any data:
> "main@1" tid=0x1 nid=NA runnable
>   java.lang.Thread.State: RUNNABLE
>       at sun.nio.ch.Net.poll(Net.java:-1)
>       at sun.nio.ch.NioSocketImpl.park(NioSocketImpl.java:191)
>       at sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:280)
>       at sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
>       at sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
>       at sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
>       at java.net.Socket$SocketInputStream.read(Socket.java:1109)
>       at 
> org.apache.hc.client5.http.impl.io.LoggingInputStream.read(LoggingInputStream.java:81)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:248)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:147)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:314)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.impl.io.IncomingHttpEntity.close(IncomingHttpEntity.java:111)
>       at 
> org.apache.hc.core5.http.io.entity.HttpEntityWrapper.close(HttpEntityWrapper.java:120)
>       at 
> org.apache.hc.client5.http.impl.classic.ResponseEntityProxy.close(ResponseEntityProxy.java:180)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.message.BasicClassicHttpResponse.close(BasicClassicHttpResponse.java:93)
>       at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpResponse.close(CloseableHttpResponse.java:205)
>       at 
> org.mitre.dsmiley.httpproxy.ChunkedTransferTest.testChunkedTransferClosing(ChunkedTransferTest.java:233)
>  
> Comparing to httpclient 4, we see that ChunkedInputStream.close() 
> does not hang because when it tries to read from the socket, it is already 
> closed, hence it fails. The resulting SocketException is caught in 
> ResponseEntityProxy and discarded.
> Tested in versions 5.4-beta1, 5.3.1, 5.2.3, 5.1.4 and 5.0.4, and they all 
> hang in the same way.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Moved] (HTTPCLIENT-2335) Closing chunked responses hangs

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski moved HTTPCORE-768 to HTTPCLIENT-2335:


  Component/s: HttpClient (classic)
   (was: HttpCore)
  Key: HTTPCLIENT-2335  (was: HTTPCORE-768)
Affects Version/s: 5.4-beta1
   5.3.1
   (was: 5.0.4)
   (was: 5.1.4)
   (was: 5.2.3)
   (was: 5.3-beta1)
 Workflow: Default workflow, editable Closed status  (was: classic 
default workflow)
  Project: HttpComponents HttpClient  (was: HttpComponents HttpCore)

> Closing chunked responses hangs
> ---
>
> Key: HTTPCLIENT-2335
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2335
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.4-beta1, 5.3.1
>Reporter: Joao Silva
>Priority: Minor
>
> While migrating a proxy implementation from httpclient 4 to httpclient 5 we 
> ran into some interesting issues regarding chunked responses.
> There is a specific test that deals with the proxy behavior when the client 
> closes the connection halfway through the response is being served.
> Using chunked responses in httpclient 4, if we close the 
> {{{}CloseableHttpResponse{}}}, the connection closes without issue.
> However, in httpclient 5 the {{CloseableHttpResponse.close()}} hangs.
> Debugging the issue, we see that {{ChunkedInputStream.close()}} hangs when 
> called, because it tries to read from the socket, and the server is not 
> actively sending any data:
> "main@1" tid=0x1 nid=NA runnable
>   java.lang.Thread.State: RUNNABLE
>       at sun.nio.ch.Net.poll(Net.java:-1)
>       at sun.nio.ch.NioSocketImpl.park(NioSocketImpl.java:191)
>       at sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:280)
>       at sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
>       at sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
>       at sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
>       at java.net.Socket$SocketInputStream.read(Socket.java:1109)
>       at 
> org.apache.hc.client5.http.impl.io.LoggingInputStream.read(LoggingInputStream.java:81)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:248)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:147)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:314)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.impl.io.IncomingHttpEntity.close(IncomingHttpEntity.java:111)
>       at 
> org.apache.hc.core5.http.io.entity.HttpEntityWrapper.close(HttpEntityWrapper.java:120)
>       at 
> org.apache.hc.client5.http.impl.classic.ResponseEntityProxy.close(ResponseEntityProxy.java:180)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.message.BasicClassicHttpResponse.close(BasicClassicHttpResponse.java:93)
>       at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpResponse.close(CloseableHttpResponse.java:205)
>       at 
> org.mitre.dsmiley.httpproxy.ChunkedTransferTest.testChunkedTransferClosing(ChunkedTransferTest.java:233)
>  
> Comparing to httpclient 4, we see that ChunkedInputStream.close() 
> does not hang because when it tries to read from the socket, it is already 
> closed, hence it fails. The resulting SocketException is caught in 
> ResponseEntityProxy and discarded.
> Tested in versions 5.4-beta1, 5.3.1, 5.2.3, 5.1.4 and 5.0.4, and they all 
> hang in the same way.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCLIENT-2335) Closing chunked responses hangs

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCLIENT-2335:
--
Fix Version/s: 5.4-beta2
 Priority: Critical  (was: Minor)

> Closing chunked responses hangs
> ---
>
> Key: HTTPCLIENT-2335
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2335
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.3.1, 5.4-beta1
>Reporter: Joao Silva
>Priority: Critical
> Fix For: 5.4-beta2
>
>
> While migrating a proxy implementation from httpclient 4 to httpclient 5 we 
> ran into some interesting issues regarding chunked responses.
> There is a specific test that deals with the proxy behavior when the client 
> closes the connection halfway through the response is being served.
> Using chunked responses in httpclient 4, if we close the 
> {{{}CloseableHttpResponse{}}}, the connection closes without issue.
> However, in httpclient 5 the {{CloseableHttpResponse.close()}} hangs.
> Debugging the issue, we see that {{ChunkedInputStream.close()}} hangs when 
> called, because it tries to read from the socket, and the server is not 
> actively sending any data:
> "main@1" tid=0x1 nid=NA runnable
>   java.lang.Thread.State: RUNNABLE
>       at sun.nio.ch.Net.poll(Net.java:-1)
>       at sun.nio.ch.NioSocketImpl.park(NioSocketImpl.java:191)
>       at sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:280)
>       at sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
>       at sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
>       at sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
>       at java.net.Socket$SocketInputStream.read(Socket.java:1109)
>       at 
> org.apache.hc.client5.http.impl.io.LoggingInputStream.read(LoggingInputStream.java:81)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:248)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:147)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:314)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.impl.io.IncomingHttpEntity.close(IncomingHttpEntity.java:111)
>       at 
> org.apache.hc.core5.http.io.entity.HttpEntityWrapper.close(HttpEntityWrapper.java:120)
>       at 
> org.apache.hc.client5.http.impl.classic.ResponseEntityProxy.close(ResponseEntityProxy.java:180)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.message.BasicClassicHttpResponse.close(BasicClassicHttpResponse.java:93)
>       at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpResponse.close(CloseableHttpResponse.java:205)
>       at 
> org.mitre.dsmiley.httpproxy.ChunkedTransferTest.testChunkedTransferClosing(ChunkedTransferTest.java:233)
>  
> Comparing to httpclient 4, we see that ChunkedInputStream.close() 
> does not hang because when it tries to read from the socket, it is already 
> closed, hence it fails. The resulting SocketException is caught in 
> ResponseEntityProxy and discarded.
> Tested in versions 5.4-beta1, 5.3.1, 5.2.3, 5.1.4 and 5.0.4, and they all 
> hang in the same way.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCORE-768) Closing chunked responses hangs

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17872395#comment-17872395
 ] 

Oleg Kalnichevski commented on HTTPCORE-768:


> In HttpClient 5 there appears to be no way to close the HTTP response in such 
> a way (as in, not meant to be reused).

[~jpmsilva] You are correct. I do not remember at what point the default 
behavior has changed and I do not remember making a conscious decision to 
change the default behavior. Your only option at this point is to abort the 
request. This will cause the underlying to get closed and discarded.

I will try to figure out how to remedy the situation but the fix would only be 
available in the 5.4 release series. I will get back to you.

Oleg

> Closing chunked responses hangs
> ---
>
> Key: HTTPCORE-768
> URL: https://issues.apache.org/jira/browse/HTTPCORE-768
> Project: HttpComponents HttpCore
>  Issue Type: Bug
>  Components: HttpCore
>Affects Versions: 5.0.4, 5.1.4, 5.2.3, 5.3-beta1
>Reporter: Joao Silva
>Priority: Minor
>
> While migrating a proxy implementation from httpclient 4 to httpclient 5 we 
> ran into some interesting issues regarding chunked responses.
> There is a specific test that deals with the proxy behavior when the client 
> closes the connection halfway through the response is being served.
> Using chunked responses in httpclient 4, if we close the 
> {{{}CloseableHttpResponse{}}}, the connection closes without issue.
> However, in httpclient 5 the {{CloseableHttpResponse.close()}} hangs.
> Debugging the issue, we see that {{ChunkedInputStream.close()}} hangs when 
> called, because it tries to read from the socket, and the server is not 
> actively sending any data:
> "main@1" tid=0x1 nid=NA runnable
>   java.lang.Thread.State: RUNNABLE
>       at sun.nio.ch.Net.poll(Net.java:-1)
>       at sun.nio.ch.NioSocketImpl.park(NioSocketImpl.java:191)
>       at sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:280)
>       at sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
>       at sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
>       at sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
>       at java.net.Socket$SocketInputStream.read(Socket.java:1109)
>       at 
> org.apache.hc.client5.http.impl.io.LoggingInputStream.read(LoggingInputStream.java:81)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:248)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:147)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:314)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.impl.io.IncomingHttpEntity.close(IncomingHttpEntity.java:111)
>       at 
> org.apache.hc.core5.http.io.entity.HttpEntityWrapper.close(HttpEntityWrapper.java:120)
>       at 
> org.apache.hc.client5.http.impl.classic.ResponseEntityProxy.close(ResponseEntityProxy.java:180)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.message.BasicClassicHttpResponse.close(BasicClassicHttpResponse.java:93)
>       at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpResponse.close(CloseableHttpResponse.java:205)
>       at 
> org.mitre.dsmiley.httpproxy.ChunkedTransferTest.testChunkedTransferClosing(ChunkedTransferTest.java:233)
>  
> Comparing to httpclient 4, we see that ChunkedInputStream.close() 
> does not hang because when it tries to read from the socket, it is already 
> closed, hence it fails. The resulting SocketException is caught in 
> ResponseEntityProxy and discarded.
> Tested in versions 5.4-beta1, 5.3.1, 5.2.3, 5.1.4 and 5.0.4, and they all 
> hang in the same way.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCORE-768) Closing chunked responses hangs

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17872268#comment-17872268
 ] 

Oleg Kalnichevski commented on HTTPCORE-768:


> I'm trying to close a connection to the server, and I would expect the 
> connection to be closed:

[~jpmsilva] Your expectation is wrong.

> But perhaps I'm missing something about this. Do you know if this is the 
> expected behavior for the API?

You are confusing closure of an entity content stream and closure of the 
underlying connection. HttpClient 4 and 5 behave _exactly_ the same. Entity 
content streams are always read until its delimiter when closed in order to 
ensure there is no trailing garbage and the connection is in an consistent 
state and could be potentially re-used for subsequent message exchanges. 

The problem you are seeing is most likely due to the server not sending the 
closing chunk, which is basically a severe protocol violation, albeit quite 
common.

If you do not intent to re-use the underling connection, you need to close it 
instead of (or before) closing the entity content stream.

Hope this helps

Oleg

> Closing chunked responses hangs
> ---
>
> Key: HTTPCORE-768
> URL: https://issues.apache.org/jira/browse/HTTPCORE-768
> Project: HttpComponents HttpCore
>  Issue Type: Bug
>  Components: HttpCore
>Affects Versions: 5.0.4, 5.1.4, 5.2.3, 5.3-beta1
>Reporter: Joao Silva
>Priority: Minor
>
> While migrating a proxy implementation from httpclient 4 to httpclient 5 we 
> ran into some interesting issues regarding chunked responses.
> There is a specific test that deals with the proxy behavior when the client 
> closes the connection halfway through the response is being served.
> Using chunked responses in httpclient 4, if we close the 
> {{{}CloseableHttpResponse{}}}, the connection closes without issue.
> However, in httpclient 5 the {{CloseableHttpResponse.close()}} hangs.
> Debugging the issue, we see that {{ChunkedInputStream.close()}} hangs when 
> called, because it tries to read from the socket, and the server is not 
> actively sending any data:
> "main@1" tid=0x1 nid=NA runnable
>   java.lang.Thread.State: RUNNABLE
>       at sun.nio.ch.Net.poll(Net.java:-1)
>       at sun.nio.ch.NioSocketImpl.park(NioSocketImpl.java:191)
>       at sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:280)
>       at sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
>       at sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
>       at sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
>       at java.net.Socket$SocketInputStream.read(Socket.java:1109)
>       at 
> org.apache.hc.client5.http.impl.io.LoggingInputStream.read(LoggingInputStream.java:81)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:248)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:147)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:314)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.impl.io.IncomingHttpEntity.close(IncomingHttpEntity.java:111)
>       at 
> org.apache.hc.core5.http.io.entity.HttpEntityWrapper.close(HttpEntityWrapper.java:120)
>       at 
> org.apache.hc.client5.http.impl.classic.ResponseEntityProxy.close(ResponseEntityProxy.java:180)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.message.BasicClassicHttpResponse.close(BasicClassicHttpResponse.java:93)
>       at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpResponse.close(CloseableHttpResponse.java:205)
>       at 
> org.mitre.dsmiley.httpproxy.ChunkedTransferTest.testChunkedTransferClosing(ChunkedTransferTest.java:233)
>  
> Comparing to httpclient 4, we see that ChunkedInputStream.close() 
> does not hang because when it tries to read from the socket, it is already 
> closed, hence it fails. The resulting SocketException is caught in 
> ResponseEntityProxy and discarded.
> Tested in versions 5.4-beta1, 5.3.1, 5.2.3, 5.1.4 and 5.0.4, and they all 
> hang in the same way.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

--

[jira] [Commented] (HTTPCORE-768) Closing chunked responses hangs

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17871943#comment-17871943
 ] 

Oleg Kalnichevski commented on HTTPCORE-768:


[~jpmsilva] The client clearly expects more data to be sent by the server. What 
makes you think this is a bug? 

Oleg

> Closing chunked responses hangs
> ---
>
> Key: HTTPCORE-768
> URL: https://issues.apache.org/jira/browse/HTTPCORE-768
> Project: HttpComponents HttpCore
>  Issue Type: Bug
>  Components: HttpCore
>Affects Versions: 5.0.4, 5.1.4, 5.2.3, 5.3-beta1
>Reporter: Joao Silva
>Priority: Minor
>
> While migrating a proxy implementation from httpclient 4 to httpclient 5 we 
> ran into some interesting issues regarding chunked responses.
> There is a specific test that deals with the proxy behavior when the client 
> closes the connection halfway through the response is being served.
> Using chunked responses in httpclient 4, if we close the 
> {{{}CloseableHttpResponse{}}}, the connection closes without issue.
> However, in httpclient 5 the {{CloseableHttpResponse.close()}} hangs.
> Debugging the issue, we see that {{ChunkedInputStream.close()}} hangs when 
> called, because it tries to read from the socket, and the server is not 
> actively sending any data:
> "main@1" tid=0x1 nid=NA runnable
>   java.lang.Thread.State: RUNNABLE
>       at sun.nio.ch.Net.poll(Net.java:-1)
>       at sun.nio.ch.NioSocketImpl.park(NioSocketImpl.java:191)
>       at sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:280)
>       at sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
>       at sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
>       at sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
>       at java.net.Socket$SocketInputStream.read(Socket.java:1109)
>       at 
> org.apache.hc.client5.http.impl.io.LoggingInputStream.read(LoggingInputStream.java:81)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149)
>       at 
> org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:248)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:147)
>       at 
> org.apache.hc.core5.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:314)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.impl.io.IncomingHttpEntity.close(IncomingHttpEntity.java:111)
>       at 
> org.apache.hc.core5.http.io.entity.HttpEntityWrapper.close(HttpEntityWrapper.java:120)
>       at 
> org.apache.hc.client5.http.impl.classic.ResponseEntityProxy.close(ResponseEntityProxy.java:180)
>       at org.apache.hc.core5.io.Closer.close(Closer.java:48)
>       at 
> org.apache.hc.core5.http.message.BasicClassicHttpResponse.close(BasicClassicHttpResponse.java:93)
>       at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpResponse.close(CloseableHttpResponse.java:205)
>       at 
> org.mitre.dsmiley.httpproxy.ChunkedTransferTest.testChunkedTransferClosing(ChunkedTransferTest.java:233)
>  
> Comparing to httpclient 4, we see that ChunkedInputStream.close() 
> does not hang because when it tries to read from the socket, it is already 
> closed, hence it fails. The resulting SocketException is caught in 
> ResponseEntityProxy and discarded.
> Tested in versions 5.4-beta1, 5.3.1, 5.2.3, 5.1.4 and 5.0.4, and they all 
> hang in the same way.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCLIENT-2333) A request retry upon redirect is incorrectly flagged as a circular redirect

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2333.
---
Resolution: Fixed

> A request retry upon redirect is incorrectly flagged as a circular redirect 
> 
>
> Key: HTTPCLIENT-2333
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2333
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (async), HttpClient (classic)
>Affects Versions: 5.3.1
>Reporter: Teresa Tang
>Priority: Minor
> Fix For: 5.3.2, 5.4-beta2
>
> Attachments: NullHandler.java, TestRedirects.java
>
>
> When a request is redirected and then needs to be retried, the retry is 
> treated as a circular redirect. If circular redirects are not allowed, an 
> error is thrown:
>  
> {code:java}
> org.apache.hc.client5.http.ClientProtocolException: Circular redirect to 
> 'http://localhost:35731/random/50'    at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173)
>     at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
>     at 
> org.apache.hc.client5.testing.sync.TestRedirects.testRetryUponRedirect(TestRedirects.java:241)
>     at java.base/java.lang.reflect.Method.invoke(Method.java:580)
>     at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
>     at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
>     at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> Caused by: org.apache.hc.client5.http.CircularRedirectException: Circular 
> redirect to 'http://localhost:35731/random/50'
>     at 
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:144)
>     at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
>     at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
>     ... 6 more{code}
>  
> Please see testRetryUponRedirect in TestRedirects.java for an example. 
> NullHandler.java will return a bad response only for the first redirected 
> request (files attached)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCLIENT-2334) AsyncDataProducer.produce gets into a tight loop

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2334.
---
Fix Version/s: 5.3.2
   5.4-beta2
   Resolution: Information Provided

> AsyncDataProducer.produce gets into a tight loop
> 
>
> Key: HTTPCLIENT-2334
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2334
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>  Components: HttpClient (async)
>Affects Versions: 5.2.2
>Reporter: Alex
>Priority: Major
> Fix For: 5.3.2, 5.4-beta2
>
>
> I am using async data transfer for http request (POST) and using 
> AsyncDataProducer interface produce method. This method seems to be called 
> multiple times in a tight NIO loop when it does not immediately produce data 
> when called. In my case it needs to store the provided DataStreamChannel and 
> return, so that DataStreamChannel can be used later when data for the request 
> is available. However the method seems to be called again immediately in a 
> tight loop and is causing high CPU consumption. Effectively I had to make it 
> blocking which is not the intent of the API.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2333) A request retry upon redirect is incorrectly flagged as a circular redirect

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17870107#comment-17870107
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2333:
---

[~ttang] Snapshot published. Please see [1]

Generally it is easier to build snapshots from source.

Oleg

[1] 
https://repository.apache.org/content/repositories/snapshots/org/apache/httpcomponents/client5/httpclient5/5.3.2-SNAPSHOT/

> A request retry upon redirect is incorrectly flagged as a circular redirect 
> 
>
> Key: HTTPCLIENT-2333
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2333
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (async), HttpClient (classic)
>Affects Versions: 5.3.1
>Reporter: Teresa Tang
>Priority: Minor
> Fix For: 5.3.2, 5.4-beta2
>
> Attachments: NullHandler.java, TestRedirects.java
>
>
> When a request is redirected and then needs to be retried, the retry is 
> treated as a circular redirect. If circular redirects are not allowed, an 
> error is thrown:
>  
> {code:java}
> org.apache.hc.client5.http.ClientProtocolException: Circular redirect to 
> 'http://localhost:35731/random/50'    at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173)
>     at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
>     at 
> org.apache.hc.client5.testing.sync.TestRedirects.testRetryUponRedirect(TestRedirects.java:241)
>     at java.base/java.lang.reflect.Method.invoke(Method.java:580)
>     at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
>     at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
>     at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> Caused by: org.apache.hc.client5.http.CircularRedirectException: Circular 
> redirect to 'http://localhost:35731/random/50'
>     at 
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:144)
>     at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
>     at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
>     ... 6 more{code}
>  
> Please see testRetryUponRedirect in TestRedirects.java for an example. 
> NullHandler.java will return a bad response only for the first redirected 
> request (files attached)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2334) AsyncDataProducer.produce gets into a tight loop

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869561#comment-17869561
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2334:
---

> If returning zero when no data is available is required the documentation 
> should make it a lot more clear

[~a701440] Agreed. Please see proposed javadoc improvements [1]

> I guess the other question is will it keep calling "available()" in a tight 
> loop then?

No, it will not. Output readiness events will get suspended.

Oleg

[1] 
[https://github.com/apache/httpcomponents-core/compare/5.2.x...HTTPCLIENT-2334] 

 

> AsyncDataProducer.produce gets into a tight loop
> 
>
> Key: HTTPCLIENT-2334
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2334
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>  Components: HttpClient (async)
>Affects Versions: 5.2.2
>Reporter: Alex
>Priority: Major
>
> I am using async data transfer for http request (POST) and using 
> AsyncDataProducer interface produce method. This method seems to be called 
> multiple times in a tight NIO loop when it does not immediately produce data 
> when called. In my case it needs to store the provided DataStreamChannel and 
> return, so that DataStreamChannel can be used later when data for the request 
> is available. However the method seems to be called again immediately in a 
> tight loop and is causing high CPU consumption. Effectively I had to make it 
> blocking which is not the intent of the API.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2334) AsyncDataProducer.produce gets into a tight loop

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869373#comment-17869373
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2334:
---

[~a701440] What output does `#available` of your 
`{color:#00}AsyncDataProducer` implementation produce? If a data producer 
is unable to produce more data momentarily it should return 0 from `#available` 
to suspend output readiness events.
{color}

{color:#00}Oleg{color}

> AsyncDataProducer.produce gets into a tight loop
> 
>
> Key: HTTPCLIENT-2334
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2334
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>  Components: HttpClient (async)
>Affects Versions: 5.2.2
>Reporter: Alex
>Priority: Major
>
> I am using async data transfer for http request (POST) and using 
> AsyncDataProducer interface produce method. This method seems to be called 
> multiple times in a tight NIO loop when it does not immediately produce data 
> when called. In my case it needs to store the provided DataStreamChannel and 
> return, so that DataStreamChannel can be used later when data for the request 
> is available. However the method seems to be called again immediately in a 
> tight loop and is causing high CPU consumption. Effectively I had to make it 
> blocking which is not the intent of the API.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2333) A request retry upon redirect is incorrectly flagged as a circular redirect

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869169#comment-17869169
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2333:
---

[~ttang] Please review / test the proposed fix:

[https://github.com/apache/httpcomponents-client/compare/5.3.x...HTTPCLIENT-2333]

Oleg

> A request retry upon redirect is incorrectly flagged as a circular redirect 
> 
>
> Key: HTTPCLIENT-2333
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2333
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (async), HttpClient (classic)
>Affects Versions: 5.3.1
>Reporter: Teresa Tang
>Priority: Minor
> Fix For: 5.3.2, 5.4-beta2
>
> Attachments: NullHandler.java, TestRedirects.java
>
>
> When a request is redirected and then needs to be retried, the retry is 
> treated as a circular redirect. If circular redirects are not allowed, an 
> error is thrown:
>  
> {code:java}
> org.apache.hc.client5.http.ClientProtocolException: Circular redirect to 
> 'http://localhost:35731/random/50'    at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173)
>     at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
>     at 
> org.apache.hc.client5.testing.sync.TestRedirects.testRetryUponRedirect(TestRedirects.java:241)
>     at java.base/java.lang.reflect.Method.invoke(Method.java:580)
>     at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
>     at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
>     at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> Caused by: org.apache.hc.client5.http.CircularRedirectException: Circular 
> redirect to 'http://localhost:35731/random/50'
>     at 
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:144)
>     at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
>     at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
>     ... 6 more{code}
>  
> Please see testRetryUponRedirect in TestRedirects.java for an example. 
> NullHandler.java will return a bad response only for the first redirected 
> request (files attached)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCLIENT-2333) A request retry upon redirect is incorrectly flagged as a circular redirect

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCLIENT-2333:
--
  Component/s: HttpClient (async)
   HttpClient (classic)
Fix Version/s: 5.3.2
   5.4-beta2

> A request retry upon redirect is incorrectly flagged as a circular redirect 
> 
>
> Key: HTTPCLIENT-2333
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2333
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (async), HttpClient (classic)
>Affects Versions: 5.3.1
>Reporter: Teresa Tang
>Priority: Minor
> Fix For: 5.3.2, 5.4-beta2
>
> Attachments: NullHandler.java, TestRedirects.java
>
>
> When a request is redirected and then needs to be retried, the retry is 
> treated as a circular redirect. If circular redirects are not allowed, an 
> error is thrown:
>  
> {code:java}
> org.apache.hc.client5.http.ClientProtocolException: Circular redirect to 
> 'http://localhost:35731/random/50'    at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173)
>     at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
>     at 
> org.apache.hc.client5.testing.sync.TestRedirects.testRetryUponRedirect(TestRedirects.java:241)
>     at java.base/java.lang.reflect.Method.invoke(Method.java:580)
>     at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
>     at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
>     at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> Caused by: org.apache.hc.client5.http.CircularRedirectException: Circular 
> redirect to 'http://localhost:35731/random/50'
>     at 
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:144)
>     at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
>     at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
>     ... 6 more{code}
>  
> Please see testRetryUponRedirect in TestRedirects.java for an example. 
> NullHandler.java will return a bad response only for the first redirected 
> request (files attached)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCLIENT-2332) Example of decompressing async response

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2332.
---
Resolution: Duplicate

> Example of decompressing async response
> ---
>
> Key: HTTPCLIENT-2332
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2332
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>  Components: HttpClient (async)
>Affects Versions: 5.3.1
>Reporter: Patrick Barry
>Priority: Major
>
> I have scoured the internet trying to find guidance or an example of how to 
> properly decompress an entity while using the async client.  What is best 
> practice for wiring in this logic?
>  
> We have a CloseableHttpAsyncClient and we are currently executing a request 
> like:
> {code:java}
> Future> execFuture = 
> asyncClient.execute(request,  
>  new BasicResponseConsumer<>(new LimitingEntityConsumer(maxSize, 
> maxTotalWaitTimeForResponse, context)),
> null, 
> context, 
> new ClientCallback(completableFuture, context)
> ); {code}
> *LimitingEntityConsumer* [extends BasicAsyncEntityConsumer] keeps track of 
> how many bytes have been read and if we exceed limit, it throws exception. It 
> also throws exception if it is taking too long to read/buffer the response.  
> In the event we have a compressed entity what is the best way to handle 
> this?  I have code already to decompress the Entity, but should that logic be 
> embedded in an overridden method in LimitingEntityConsumer...
>  
> {code:java}
> protected abstract T generateContent( * add decompression code here) throws 
> IOException; {code}
> or 
> register a response interceptor on async client as: 
> {code:java}
> asyncClient.addResponseInterceptorFirst(new Decompressor()) {code}
> or 
> register a AsyncExecChainHandler on async client as: 
> {code:java}
> .addExecInterceptorLast("decompress", new Decompressor()) {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2332) Example of decompressing async response

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17866947#comment-17866947
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2332:
---

[~patrickjamesbarry] This issue is essentially a duplicate of HTTPCLIENT-1822. 

What you need to do is
 # Build a custom {{AsyncDataConsumer}} that can transparently decompress data 
and pass it for further processing to another {{AsyncDataConsumer}}
 # Build a custom {{AsyncDataProducer}} that can transparently compress data 
produced by another {{AsyncDataProducer}} and commit in to the underlying 
{{{}DataStreamChannel{}}}.
 # Build a custom {{ExecResponseInterceptor}} that automatically injects 
compressing / decompressing data producer / consumer into the request execution 
pipeline depending on request / response message composition or {{HttpContext}} 
state.

You can use DigestingEntityConsumer [1] and DigestingEntityProducer [2] as a 
reference and a starting point.

 Oleg

[1] 
https://github.com/apache/httpcomponents-core/blob/5.2.x/httpcore5/src/main/java/org/apache/hc/core5/http/nio/entity/DigestingEntityConsumer.java
[2] 
https://github.com/apache/httpcomponents-core/blob/5.2.x/httpcore5/src/main/java/org/apache/hc/core5/http/nio/entity/DigestingEntityProducer.java

 

> Example of decompressing async response
> ---
>
> Key: HTTPCLIENT-2332
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2332
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>  Components: HttpClient (async)
>Affects Versions: 5.3.1
>Reporter: Patrick Barry
>Priority: Major
>
> I have scoured the internet trying to find guidance or an example of how to 
> properly decompress an entity while using the async client.  What is best 
> practice for wiring in this logic?
>  
> We have a CloseableHttpAsyncClient and we are currently executing a request 
> like:
> {code:java}
> Future> execFuture = 
> asyncClient.execute(request,  
>  new BasicResponseConsumer<>(new LimitingEntityConsumer(maxSize, 
> maxTotalWaitTimeForResponse, context)),
> null, 
> context, 
> new ClientCallback(completableFuture, context)
> ); {code}
> *LimitingEntityConsumer* [extends BasicAsyncEntityConsumer] keeps track of 
> how many bytes have been read and if we exceed limit, it throws exception. It 
> also throws exception if it is taking too long to read/buffer the response.  
> In the event we have a compressed entity what is the best way to handle 
> this?  I have code already to decompress the Entity, but should that logic be 
> embedded in an overridden method in LimitingEntityConsumer...
>  
> {code:java}
> protected abstract T generateContent( * add decompression code here) throws 
> IOException; {code}
> or 
> register a response interceptor on async client as: 
> {code:java}
> asyncClient.addResponseInterceptorFirst(new Decompressor()) {code}
> or 
> register a AsyncExecChainHandler on async client as: 
> {code:java}
> .addExecInterceptorLast("decompress", new Decompressor()) {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17856942#comment-17856942
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2328:
---

[~Zoe Wang] Please test the following change-set 
https://github.com/apache/httpcomponents-client/commit/ee0a102104d03a8d1cfe18e571d179c41242182c
 from master. You will likely need to build HttpClient from the source.

Oleg

> Request hangs if TLS 1.3 connection is half-closed 
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Minor
> Fix For: 5.4-alpha3
>
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java, 
> TlsHalfCloseApache4.java, keystore.jks
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> If a server with TLS 1.3 support closes the connection during the request, 
> more specifically, sending close_notify while the client is still writing to 
> socket,  the request will hang indefinitely. It's not an issue with TLS 1.2 
> because it uses duplex-close policy. With TLS 1.3's half-closed connection 
> policy, it seems Apache HTTP client is not able to detect connection closure 
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should 
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see 
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot 
> of users to make changes on their side. 
>  
> Steps to repro:
>  * Download the attached keystore file
>  * Update ksPath in the server code HalfCloseServer.java to where you 
> download the keystore
>  * Run the server, the server will begin listening on {{localhost:8081}}
>  * Create a random file of size 128MB and update client code "testFile" to 
> where the file is.
>  * Run the client, it should hang
>  ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is 
> uncommented, it will not hang
>  ** It also won’t hang if we we force TLS1.2
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCORE-767) Update JSSEProviderIntegrationTest to skip conscript when on ppc64le

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCORE-767.

Fix Version/s: 5.3-beta2
   Resolution: Fixed

> Update JSSEProviderIntegrationTest to skip conscript when on ppc64le
> 
>
> Key: HTTPCORE-767
> URL: https://issues.apache.org/jira/browse/HTTPCORE-767
> Project: HttpComponents HttpCore
>  Issue Type: Test
>Reporter: Jamie Mark Goodyear
>Priority: Trivial
> Fix For: 5.3-beta2
>
>
> Update JSSEProviderIntegrationTest to skip conscript when on ppc64le, as no 
> binaries are available for that architecture.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCLIENT-2331) Single cookie header should be sent for multiple cookies

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCLIENT-2331:
--
  Component/s: HttpClient (async)
Fix Version/s: 5.4-alpha3
 Priority: Minor  (was: Major)

> Single cookie header should be sent for multiple cookies
> 
>
> Key: HTTPCLIENT-2331
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2331
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (async), HttpClient (classic)
>Reporter: Nikhil Chaudhari
>Priority: Minor
> Fix For: 5.4-alpha3
>
> Attachments: DemoServiceOneController.java, DemoServiceOneLogs.txt, 
> DemoServiceTwoController.java
>
>
> Single cookie header should be sent for multiple cookies
> *Use case* - 
> When *Cookie* header is added explicitly using {*}setHeader{*}() method and 
> there exists multiple cookies in cookie store (BasicCookieStore), then for 
> this use case two Cookie headers are sent by HttpClient.
> *Expectation* - Single Cookie header should be sent by Apache HttpClient 
> where all cookies should be separated by semi-colon
> *Actual* - Two Cookie headers are sent by Apache HttpClient, where in cookie 
> set using setHeader is sent as one cookie Header and cookies in 
> BasicCookieStore are sent in another Cookie header wherein value is all 
> cookies separates by semi-colon.
>  
> *Steps to reproduce -*
>  # Setup two microservices
>  # 1st microservice will set custom Cookie header as part of API request to 
> 2nd micro service.
>  # 2nd microservice will respond back with two Set-Cookie headers 
>  # First time when call is made from 1st micro service to 2nd micro service 
> only custom cookie header will be sent and BasicCookieStore on 1st 
> microservice will be updated with cookies that we got from 2nd microservice
>  # Second time when call is made from 1st micro service to 2nd micro service 
> both custom cookie and cookies from BasicCookieStore will be sent as separate 
> Cookie headers.
>  
> {code:java}
> 2024-06-18 09:45:28.074 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> GET 
> /demo-service-two/test HTTP/1.1
> 2024-06-18 09:45:28.074 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> Cookie: 
> MyCustomCookie=QWERTY
> 2024-06-18 09:45:28.074 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> 
> Accept-Encoding: gzip, x-gzip, deflate
> 2024-06-18 09:45:28.075 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> Host: 
> localhost:8091
> 2024-06-18 09:45:28.075 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> Connection: 
> keep-alive
> 2024-06-18 09:45:28.075 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> User-Agent: 
> Apache-HttpClient/5.1.3 (Java/17.0.8)
> 2024-06-18 09:45:28.075 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> Cookie: 
> ServiceTwoSession1=POIUY; ServiceTwoSession2=ZXCVB {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2331) Single cookie header should be sent for multiple cookies

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17855953#comment-17855953
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2331:
---

> Yes, I see that HttpClient doesn't remove the manually added Cookie headers 
> but shouldn't it merge all cookies

[~nvchaudhari1991] No, it should not. HttpClient does not meddle with manually 
added headers, pretty much never. 

What I am going to change in 5.4 is the behavior of HttpClient when there is a 
Cookie header present in the request message. Most likely I will make it skip 
Cookie header generation altogether.  

Oleg

> Single cookie header should be sent for multiple cookies
> 
>
> Key: HTTPCLIENT-2331
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2331
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Reporter: Nikhil Chaudhari
>Priority: Major
> Attachments: DemoServiceOneController.java, DemoServiceOneLogs.txt, 
> DemoServiceTwoController.java
>
>
> Single cookie header should be sent for multiple cookies
> *Use case* - 
> When *Cookie* header is added explicitly using {*}setHeader{*}() method and 
> there exists multiple cookies in cookie store (BasicCookieStore), then for 
> this use case two Cookie headers are sent by HttpClient.
> *Expectation* - Single Cookie header should be sent by Apache HttpClient 
> where all cookies should be separated by semi-colon
> *Actual* - Two Cookie headers are sent by Apache HttpClient, where in cookie 
> set using setHeader is sent as one cookie Header and cookies in 
> BasicCookieStore are sent in another Cookie header wherein value is all 
> cookies separates by semi-colon.
>  
> *Steps to reproduce -*
>  # Setup two microservices
>  # 1st microservice will set custom Cookie header as part of API request to 
> 2nd micro service.
>  # 2nd microservice will respond back with two Set-Cookie headers 
>  # First time when call is made from 1st micro service to 2nd micro service 
> only custom cookie header will be sent and BasicCookieStore on 1st 
> microservice will be updated with cookies that we got from 2nd microservice
>  # Second time when call is made from 1st micro service to 2nd micro service 
> both custom cookie and cookies from BasicCookieStore will be sent as separate 
> Cookie headers.
>  
> {code:java}
> 2024-06-18 09:45:28.074 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> GET 
> /demo-service-two/test HTTP/1.1
> 2024-06-18 09:45:28.074 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> Cookie: 
> MyCustomCookie=QWERTY
> 2024-06-18 09:45:28.074 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> 
> Accept-Encoding: gzip, x-gzip, deflate
> 2024-06-18 09:45:28.075 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> Host: 
> localhost:8091
> 2024-06-18 09:45:28.075 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> Connection: 
> keep-alive
> 2024-06-18 09:45:28.075 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> User-Agent: 
> Apache-HttpClient/5.1.3 (Java/17.0.8)
> 2024-06-18 09:45:28.075 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> Cookie: 
> ServiceTwoSession1=POIUY; ServiceTwoSession2=ZXCVB {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2331) Single cookie header should be sent for multiple cookies

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17855945#comment-17855945
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2331:
---

[~nvchaudhari1991] HttpClient 5.x conforms to RFC 6265 and generates a single 
`Cookie` header. If the request contains other `Cookie` headers manually added 
by the caller HttpClient does not remove them.

Oleg   

> Single cookie header should be sent for multiple cookies
> 
>
> Key: HTTPCLIENT-2331
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2331
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Reporter: Nikhil Chaudhari
>Priority: Major
> Attachments: DemoServiceOneController.java, DemoServiceOneLogs.txt, 
> DemoServiceTwoController.java
>
>
> Single cookie header should be sent for multiple cookies
> *Use case* - 
> When *Cookie* header is added explicitly using {*}setHeader{*}() method and 
> there exists multiple cookies in cookie store (BasicCookieStore), then for 
> this use case two Cookie headers are sent by HttpClient.
> *Expectation* - Single Cookie header should be sent by Apache HttpClient 
> where all cookies should be separated by semi-colon
> *Actual* - Two Cookie headers are sent by Apache HttpClient, where in cookie 
> set using setHeader is sent as one cookie Header and cookies in 
> BasicCookieStore are sent in another Cookie header wherein value is all 
> cookies separates by semi-colon.
>  
> *Steps to reproduce -*
>  # Setup two microservices
>  # 1st microservice will set custom Cookie header as part of API request to 
> 2nd micro service.
>  # 2nd microservice will respond back with two Set-Cookie headers 
>  # First time when call is made from 1st micro service to 2nd micro service 
> only custom cookie header will be sent and BasicCookieStore on 1st 
> microservice will be updated with cookies that we got from 2nd microservice
>  # Second time when call is made from 1st micro service to 2nd micro service 
> both custom cookie and cookies from BasicCookieStore will be sent as separate 
> Cookie headers.
>  
> {code:java}
> 2024-06-18 09:45:28.074 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> GET 
> /demo-service-two/test HTTP/1.1
> 2024-06-18 09:45:28.074 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> Cookie: 
> MyCustomCookie=QWERTY
> 2024-06-18 09:45:28.074 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> 
> Accept-Encoding: gzip, x-gzip, deflate
> 2024-06-18 09:45:28.075 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> Host: 
> localhost:8091
> 2024-06-18 09:45:28.075 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> Connection: 
> keep-alive
> 2024-06-18 09:45:28.075 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> User-Agent: 
> Apache-HttpClient/5.1.3 (Java/17.0.8)
> 2024-06-18 09:45:28.075 DEBUG 3540 --- [nio-8090-exec-7] 
> org.apache.hc.client5.http.headers       : http-outgoing-1 >> Cookie: 
> ServiceTwoSession1=POIUY; ServiceTwoSession2=ZXCVB {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17854161#comment-17854161
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2328:
---

[~Zoe Wang] Has `ManagedHttpClientConnectionFactory` been upgraded to use new 
core APIs?

Oleg

> Request hangs if TLS 1.3 connection is half-closed 
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Minor
> Fix For: 5.4-alpha3
>
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java, 
> TlsHalfCloseApache4.java, keystore.jks
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> If a server with TLS 1.3 support closes the connection during the request, 
> more specifically, sending close_notify while the client is still writing to 
> socket,  the request will hang indefinitely. It's not an issue with TLS 1.2 
> because it uses duplex-close policy. With TLS 1.3's half-closed connection 
> policy, it seems Apache HTTP client is not able to detect connection closure 
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should 
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see 
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot 
> of users to make changes on their side. 
>  
> Steps to repro:
>  * Download the attached keystore file
>  * Update ksPath in the server code HalfCloseServer.java to where you 
> download the keystore
>  * Run the server, the server will begin listening on {{localhost:8081}}
>  * Create a random file of size 128MB and update client code "testFile" to 
> where the file is.
>  * Run the client, it should hang
>  ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is 
> uncommented, it will not hang
>  ** It also won’t hang if we we force TLS1.2
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853757#comment-17853757
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2330:
---

> And likely no way to avoid that.

[~doktorj] When using TLS one needs to stop counting seconds and to make sure 
the connection does not get stuck for a minute or more. 

However In your particular case you could run your application with TLS debug 
on and see what protocol version gets negotiated and exactly what kind of 
packets get exchanged between the two endpoints after the initial timeout. 
Forcing older / newer version of TLS may help, or force-closing the connection 
and foregoing graceful TLS layer shutdown may solve the problem for you.

Oleg

> Socket timeouts taking 2x time requested
> 
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to 
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues 
> recreating this in a small unit test, however. We have traced through the 
> execution of the code carefully, with trace-level logging enabled, and see 
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https 
> and basic auth (Aloha; using a site like httpstat.us does not seem to 
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds 
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|                
> http-nio-8080-exec-2|         org.apache.hc.client5.http.wire| 
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG|                http-nio-8080-exec-2| 
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the 
> socket timeout, this time will be identical to the socket timeout), followed 
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in 
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote}                this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this 
> line quickly to see it hang here for the socket timeout period; if we wait on 
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket 
> timeout is just a timeout between events (so the connection cannot be idle 
> longer that time span without encountering the socket timeout), but this 
> appears to be clear that a socket timeout is being encountered, followed by a 
> delay equivalent to the socket timeout timespan (another socket timeout being 
> thrown?).
> A better way to handle an overall timeout would be to use something other 
> than just a socket timeout — but this is very unexpected to have a socket 
> timeout taking 2x the configured time (as if 2 separate timeouts are 
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well 
> as 5.x, and we've used both java 17, and java 21.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853751#comment-17853751
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2330:
---

[~doktorj] I suppose it is pretty simple. The TLS layer decides it needs to do 
a handshake with the opposite endpoint that involves an extra read operation 
(most likely the close notify) and there comes that extra socket timeout delay. 

Oleg

> Socket timeouts taking 2x time requested
> 
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to 
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues 
> recreating this in a small unit test, however. We have traced through the 
> execution of the code carefully, with trace-level logging enabled, and see 
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https 
> and basic auth (Aloha; using a site like httpstat.us does not seem to 
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds 
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|                
> http-nio-8080-exec-2|         org.apache.hc.client5.http.wire| 
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG|                http-nio-8080-exec-2| 
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the 
> socket timeout, this time will be identical to the socket timeout), followed 
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in 
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote}                this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this 
> line quickly to see it hang here for the socket timeout period; if we wait on 
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket 
> timeout is just a timeout between events (so the connection cannot be idle 
> longer that time span without encountering the socket timeout), but this 
> appears to be clear that a socket timeout is being encountered, followed by a 
> delay equivalent to the socket timeout timespan (another socket timeout being 
> thrown?).
> A better way to handle an overall timeout would be to use something other 
> than just a socket timeout — but this is very unexpected to have a socket 
> timeout taking 2x the configured time (as if 2 separate timeouts are 
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well 
> as 5.x, and we've used both java 17, and java 21.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853744#comment-17853744
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2330:
---

> We are scratching our heads over why it appears the flush is also taking a 
> period of time equivalent to the socket timeout

[~doktorj] Are you using TLS over that connection?

Oleg

> Socket timeouts taking 2x time requested
> 
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to 
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues 
> recreating this in a small unit test, however. We have traced through the 
> execution of the code carefully, with trace-level logging enabled, and see 
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https 
> and basic auth (Aloha; using a site like httpstat.us does not seem to 
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds 
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|                
> http-nio-8080-exec-2|         org.apache.hc.client5.http.wire| 
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG|                http-nio-8080-exec-2| 
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the 
> socket timeout, this time will be identical to the socket timeout), followed 
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in 
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote}                this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this 
> line quickly to see it hang here for the socket timeout period; if we wait on 
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket 
> timeout is just a timeout between events (so the connection cannot be idle 
> longer that time span without encountering the socket timeout), but this 
> appears to be clear that a socket timeout is being encountered, followed by a 
> delay equivalent to the socket timeout timespan (another socket timeout being 
> thrown?).
> A better way to handle an overall timeout would be to use something other 
> than just a socket timeout — but this is very unexpected to have a socket 
> timeout taking 2x the configured time (as if 2 separate timeouts are 
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well 
> as 5.x, and we've used both java 17, and java 21.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2330.
---
Resolution: Invalid

> Socket timeouts taking 2x time requested
> 
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to 
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues 
> recreating this in a small unit test, however. We have traced through the 
> execution of the code carefully, with trace-level logging enabled, and see 
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https 
> and basic auth (Aloha; using a site like httpstat.us does not seem to 
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds 
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|                
> http-nio-8080-exec-2|         org.apache.hc.client5.http.wire| 
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG|                http-nio-8080-exec-2| 
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the 
> socket timeout, this time will be identical to the socket timeout), followed 
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in 
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote}                this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this 
> line quickly to see it hang here for the socket timeout period; if we wait on 
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket 
> timeout is just a timeout between events (so the connection cannot be idle 
> longer that time span without encountering the socket timeout), but this 
> appears to be clear that a socket timeout is being encountered, followed by a 
> delay equivalent to the socket timeout timespan (another socket timeout being 
> thrown?).
> A better way to handle an overall timeout would be to use something other 
> than just a socket timeout — but this is very unexpected to have a socket 
> timeout taking 2x the configured time (as if 2 separate timeouts are 
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well 
> as 5.x, and we've used both java 17, and java 21.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Comment Edited] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853737#comment-17853737
 ] 

Oleg Kalnichevski edited comment on HTTPCLIENT-2330 at 6/10/24 4:57 PM:


>  but this is very unexpected to have a socket timeout taking 2x the 
> configured time (as if 2 separate timeouts are occurring).

[~doktorj] Moreover, Socket timeout is _not_ a deadline. It is perfectly normal 
that the total message exchange time takes longer than the socket timeout 
value, sometimes much longer, especially when TLS is being used. What you 
likely want is HTTPCLIENT-1074, which all versions and all variants of 
HttpClient presently do not support.

Oleg


was (Author: olegk):
>  but this is very unexpected to have a socket timeout taking 2x the 
> configured time (as if 2 separate timeouts are occurring).

[~doktorj] Moreover, Socket timeout is _not_ a deadline. It is perfectly normal 
that the total message time takes longer than the socket timeout value, 
sometimes much longer, especially when TLS is being used. What you likely want 
is HTTPCLIENT-1074, which all versions and all variants of HttpClient presently 
do not support.

Oleg

> Socket timeouts taking 2x time requested
> 
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to 
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues 
> recreating this in a small unit test, however. We have traced through the 
> execution of the code carefully, with trace-level logging enabled, and see 
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https 
> and basic auth (Aloha; using a site like httpstat.us does not seem to 
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds 
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|                
> http-nio-8080-exec-2|         org.apache.hc.client5.http.wire| 
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG|                http-nio-8080-exec-2| 
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the 
> socket timeout, this time will be identical to the socket timeout), followed 
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in 
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote}                this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this 
> line quickly to see it hang here for the socket timeout period; if we wait on 
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket 
> timeout is just a timeout between events (so the connection cannot be idle 
> longer that time span without encountering the socket timeout), but this 
> appears to be clear that a socket timeout is being encountered, followed by a 
> delay equivalent to the socket timeout timespan (another socket timeout being 
> thrown?).
> A better way to handle an overall timeout would be to use something other 
> than just a socket timeout — but this is very unexpected to have a socket 
> timeout taking 2x the configured time (as if 2 separate timeouts are 
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well 
> as 5.x, and we've used both java 17, and java 21.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853737#comment-17853737
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2330:
---

>  but this is very unexpected to have a socket timeout taking 2x the 
> configured time (as if 2 separate timeouts are occurring).

[~doktorj] Moreover, Socket timeout is _not_ a deadline. It is perfectly normal 
that the total message time takes longer than the socket timeout value, 
sometimes much longer, especially when TLS is being used. What you likely want 
is HTTPCLIENT-1074, which all versions and all variants of HttpClient presently 
do not support.

Oleg

> Socket timeouts taking 2x time requested
> 
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to 
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues 
> recreating this in a small unit test, however. We have traced through the 
> execution of the code carefully, with trace-level logging enabled, and see 
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https 
> and basic auth (Aloha; using a site like httpstat.us does not seem to 
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds 
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|                
> http-nio-8080-exec-2|         org.apache.hc.client5.http.wire| 
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG|                http-nio-8080-exec-2| 
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the 
> socket timeout, this time will be identical to the socket timeout), followed 
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in 
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote}                this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this 
> line quickly to see it hang here for the socket timeout period; if we wait on 
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket 
> timeout is just a timeout between events (so the connection cannot be idle 
> longer that time span without encountering the socket timeout), but this 
> appears to be clear that a socket timeout is being encountered, followed by a 
> delay equivalent to the socket timeout timespan (another socket timeout being 
> thrown?).
> A better way to handle an overall timeout would be to use something other 
> than just a socket timeout — but this is very unexpected to have a socket 
> timeout taking 2x the configured time (as if 2 separate timeouts are 
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well 
> as 5.x, and we've used both java 17, and java 21.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853716#comment-17853716
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2330:
---

[~doktorj] I am not sure I understand the problem and what it is exactly you 
consider a bug. Write operations on blocking sockets do not take the socket 
timeout into consideration. We cannot change that.

Oleg

> Socket timeouts taking 2x time requested
> 
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to 
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues 
> recreating this in a small unit test, however. We have traced through the 
> execution of the code carefully, with trace-level logging enabled, and see 
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https 
> and basic auth (Aloha; using a site like httpstat.us does not seem to 
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds 
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|                
> http-nio-8080-exec-2|         org.apache.hc.client5.http.wire| 
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG|                http-nio-8080-exec-2| 
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the 
> socket timeout, this time will be identical to the socket timeout), followed 
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|                
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03 
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in 
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote}                this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this 
> line quickly to see it hang here for the socket timeout period; if we wait on 
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket 
> timeout is just a timeout between events (so the connection cannot be idle 
> longer that time span without encountering the socket timeout), but this 
> appears to be clear that a socket timeout is being encountered, followed by a 
> delay equivalent to the socket timeout timespan (another socket timeout being 
> thrown?).
> A better way to handle an overall timeout would be to use something other 
> than just a socket timeout — but this is very unexpected to have a socket 
> timeout taking 2x the configured time (as if 2 separate timeouts are 
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well 
> as 5.x, and we've used both java 17, and java 21.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCORE-702) Improve connection limit guarantees

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCORE-702.

Resolution: Won't Fix

Closing due to inactivity.

Oleg

> Improve connection limit guarantees
> ---
>
> Key: HTTPCORE-702
> URL: https://issues.apache.org/jira/browse/HTTPCORE-702
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 4.4.14
> Environment: Ubuntu 18.04.6 LTS
> OpenJDK version 11.0.11
>Reporter: Nikola Bekcic
>Priority: Minor
>  Labels: volunteers-wanted
>
> h3. Background
> To manage the life cycle and re-use HTTP connections we are using 
> *PoolingHttpClientConnectionManager* with the maximum limit of connections 
> defined on a per route basis as well as in total.
> Using PoolingHttpClientConnectionManager and built-in Java management 
> utilities, I want to provide our users with the option to dynamically re-size 
> the connection pool (up and/or down) at runtime without redeploying or 
> restarting the application with a new configuration.
> h3. Issue description
> During testing, we found that decreasing the maximum limit of connection per 
> route does not have (an immediate) effect on the pool size i.e maximum number 
> of connections per route is staying the same (potentially for a very long 
> time).
> E.g. let's use only two routes where the default maximum connections per 
> route are 10 and the maximum total is 20. At some point, the maximum number 
> of connections per route will be riched and the number of allocated 
> connections (available + leased) will be 10. Now, let's say that we need to 
> decrease the maximum number of connections per route to 5, for any reason. 
> Doing that by calling either 
> _PoolingHttpClientConnectionManager#setDefaultMaxPerRoute(int)_ or 
> _PoolingHttpClientConnectionManager#setMaxPerRoute(HttpRoute, int)_ will not 
> change the pool size and the maximum number of connections per route will 
> stay the same (potentially for a very long time).
> This is the sample of the pool stats printed after changing +maxPerRoute+ to 
> 5:
> {noformat}
> {somehost1:8983={"available":4,"leased":6,"max":5,"pending":0}, 
> somehost2:8080={"available":1,"leased":9,"max":5,"pending":2}}
> {somehost1:8983={"available":4,"leased":6,"max":5,"pending":0}, 
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":2}}
> {somehost1:8983={"available":5,"leased":5,"max":5,"pending":0}, 
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":4}}
> {somehost1:8983={"available":5,"leased":5,"max":5,"pending":0}, 
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":4}}
> {somehost1:8983={"available":7,"leased":3,"max":5,"pending":0}, 
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":5}}
> {somehost1:8983={"available":9,"leased":1,"max":5,"pending":0}, 
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":5}}
> {somehost1:8983={"available":9,"leased":1,"max":5,"pending":0}, 
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":6}}
> {somehost1:8983={"available":8,"leased":2,"max":5,"pending":0}, 
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":6}}
> {somehost1:8983={"available":9,"leased":1,"max":5,"pending":0}, 
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":6}}
> {somehost1:8983={"available":8,"leased":2,"max":5,"pending":0}, 
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":7}}
> {somehost1:8983={"available":8,"leased":2,"max":5,"pending":0}, 
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":6}}
> {somehost1:8983={"available":8,"leased":2,"max":5,"pending":0}, 
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":6}}
> {somehost1:8983={"available":9,"leased":1,"max":5,"pending":0}, 
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":6}}
> {somehost1:8983={"available":9,"leased":1,"max":5,"pending":0}, 
> somehost2:8080={"available":1,"leased":9,"max":5,"pending":6}}} {noformat}
> As you can see, not only that the number of allocated connections per route 
> is still 10, but also very often the number of leased connection is higher 
> than 5. In my opinion, this is not what is expected and not really in favor 
> of the principle of least surprise.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCORE-484) check timeout could use TimeWheel algorithm

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCORE-484.

Fix Version/s: (was: Future)
   Resolution: Won't Fix

Closing due to inactivity.

Oleg

> check timeout could use TimeWheel algorithm
> ---
>
> Key: HTTPCORE-484
> URL: https://issues.apache.org/jira/browse/HTTPCORE-484
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore NIO
>Affects Versions: 4.4.6
>Reporter: silver9886
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> check time out every this.selectTimeout,and had to iterator all the 
> SelectionKeys.
> it is not efficient nor accurate when their is a lot of SelectionKeys.
> I suggest use the TimeWheel algorithm when check the channel time out just
> as netty do.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2328.
---
Fix Version/s: (was: 5.3.2)
   Resolution: Fixed

> Request hangs if TLS 1.3 connection is half-closed 
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Minor
> Fix For: 5.4-alpha3
>
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java, 
> TlsHalfCloseApache4.java, keystore.jks
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> If a server with TLS 1.3 support closes the connection during the request, 
> more specifically, sending close_notify while the client is still writing to 
> socket,  the request will hang indefinitely. It's not an issue with TLS 1.2 
> because it uses duplex-close policy. With TLS 1.3's half-closed connection 
> policy, it seems Apache HTTP client is not able to detect connection closure 
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should 
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see 
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot 
> of users to make changes on their side. 
>  
> Steps to repro:
>  * Download the attached keystore file
>  * Update ksPath in the server code HalfCloseServer.java to where you 
> download the keystore
>  * Run the server, the server will begin listening on {{localhost:8081}}
>  * Create a random file of size 128MB and update client code "testFile" to 
> where the file is.
>  * Run the client, it should hang
>  ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is 
> uncommented, it will not hang
>  ** It also won’t hang if we we force TLS1.2
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCLIENT-2326) Support any Exception in RetryStrategy

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2326.
---
Fix Version/s: 5.4-alpha3
   Resolution: Fixed

> Support any Exception in RetryStrategy
> --
>
> Key: HTTPCLIENT-2326
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2326
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>Reporter: Oscar
>Priority: Minor
> Fix For: 5.4-alpha3
>
>
> It would be useful to modify {{HttpRequestRetryExec}} to allow 
> {{HttpRequestRetryStrategy}} to support all kind of exceptions, not only 
> those extending {{{}IOException{}}}.
> I would like HttpClient to retry requests when facing a 
> [TunnelRefusedException|https://hc.apache.org/httpcomponents-client-5.1.x/current/httpclient5/apidocs/org/apache/hc/client5/http/impl/TunnelRefusedException.html]
>  (it happens randomly that the proxy refuses the initial connection, but it 
> usually works ok when retrying).
> I have tried extending 
> [DefaultHttpRequestRetryStrategy|https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/main/java/org/apache/hc/client5/http/impl/DefaultHttpRequestRetryStrategy.java]
>  but it can only capture exceptions inheriting from {{IOException.}}
> So, maybe {{HttpRequestRetryStrategy}} interface should support any kind of 
> {{Exception }} or maybe {{TunnelRefusedException}} should be an 
> {{IOException}}
>  
> This is an example of a stacktrace that I would like to retry:
>  
> {code:java}
> org.apache.hc.client5.http.ClientProtocolException: CONNECT refused by proxy: 
> HTTP/1.1 500 Internal Server Error
> at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173)
> at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
> at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
> [...]
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: org.apache.hc.client5.http.impl.TunnelRefusedException: CONNECT 
> refused by proxy: HTTP/1.1 500 Internal Server Error
> at 
> org.apache.hc.client5.http.impl.classic.ConnectExec.createTunnelToTarget(ConnectExec.java:284)
> at 
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:151)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCORE-766) Serve same content regardless of http authority (RequestHandlerRegistry)

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCORE-766.

Fix Version/s: 5.3-alpha3
   Resolution: Fixed

> Serve same content regardless of http authority (RequestHandlerRegistry)
> 
>
> Key: HTTPCORE-766
> URL: https://issues.apache.org/jira/browse/HTTPCORE-766
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.2.4, 5.3-alpha1, 5.3-alpha2
>Reporter: Erik Wramner
>Priority: Minor
>  Labels: easyfix
> Fix For: 5.3-alpha3
>
>   Original Estimate: 4h
>  Time Spent: 40m
>  Remaining Estimate: 3h 20m
>
> I need to serve the same content on all interfaces and I don't know the IP 
> addresses and host names in advance. Think of it as a classic HTTP server 
> without virtual host support: all requests get to the same set of handlers.
> Unfortunately, ServerBootstrap installs RequestHandlerRegistry, which uses 
> one primary LookupRegistry for a canonical host name and localhost and a map 
> with other registries keyed by name. If a client connects with an unknown 
> name, no handler is found and the server responds with 421.
> There are several possible solutions here:
>  # Modify RequestHandlerRegistry to use the primary registry if no other 
> registry is found. As I recall, that was what Apache did by default. Then it 
> is still possible to register new rules for virtual hosts, but the primary is 
> fallback when no name matches. This could also be configurable on/off.
>  # Add new PrimaryOnlyRequestHandlerRegistry that has no support for virtual 
> hosts and always uses the primary.
>  # Don't fix this in the code, but make it easier for me to fix by making 
> getPatternMatcher protected instead of private.
> In the two last cases we run into another problem: ServerBootstrap has 
> private members without getters, final methods that cannot be overridden and 
> creates RequestHandlerRegistry with new in a long create method. This is the 
> problem that was solved in HTTPCORE-570 for another class. It makes it hard 
> to extend the class, it must basically be rewritten.
> I would propose opening up ServerBootstrap with getters or by making the 
> fields protected. I would also like to extract the RequestHandlerRegistry 
> creation to a protected method that can be overridden. Alternatively, we can 
> add a setter and allow the calling code to prepare an instance outside of the 
> builder.
> I would be happy to make a pull request for these changes on Github. If so, 
> what direction would you like to take?
> I can code around this by copying both classes with my changes, but that 
> feels really dirty.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCLIENT-2329) BasicHttpClientConnectionManager reuses closed connection objects

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2329.
---
Fix Version/s: 5.3.2
   5.4-alpha3
   Resolution: Duplicate

> BasicHttpClientConnectionManager reuses closed connection objects
> -
>
> Key: HTTPCLIENT-2329
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2329
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>Affects Versions: 5.3.1
>Reporter: Teresa Tang
>Priority: Major
> Fix For: 5.3.2, 5.4-alpha3
>
> Attachments: TestBasicConnectionManager.java
>
>
> In the discardEndpoint method of InternalExecRuntime.java, the endpoint and 
> connection are closed. The manager releases the connection with a timevalue 
> of 0 ms. Because 0 is not considered positive, this leads to the expiration 
> being set to Long.MAX_VALUE. Upon the next connection request, the manager 
> will continue to use this unexpired connection object, even though it is 
> closed. 
>  
> The intention of the 0 ms timevalue was to have the connection expire 
> immediately and then be discarded so that the manager will create a new 
> connection for the next request.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2329) BasicHttpClientConnectionManager reuses closed connection objects

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17851985#comment-17851985
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2329:
---

[~ttang] Alternatively get the source from GitHub [1], switch to 5.3.x branch 
and run `mvn clean install` to install the latest SNAPSHOT into your local 
Maven repository.

Oleg

[1] https://github.com/apache/httpcomponents-client/tree/5.3.x

> BasicHttpClientConnectionManager reuses closed connection objects
> -
>
> Key: HTTPCLIENT-2329
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2329
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>Affects Versions: 5.3.1
>Reporter: Teresa Tang
>Priority: Major
> Attachments: TestBasicConnectionManager.java
>
>
> In the discardEndpoint method of InternalExecRuntime.java, the endpoint and 
> connection are closed. The manager releases the connection with a timevalue 
> of 0 ms. Because 0 is not considered positive, this leads to the expiration 
> being set to Long.MAX_VALUE. Upon the next connection request, the manager 
> will continue to use this unexpired connection object, even though it is 
> closed. 
>  
> The intention of the 0 ms timevalue was to have the connection expire 
> immediately and then be discarded so that the manager will create a new 
> connection for the next request.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17851984#comment-17851984
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2328:
---

[~Zoe Wang] Ultimately the fix itself is fairly simple [1]. However it requires 
quite a bit of refactoring in the blocking connection and TLS code [2]. For 
one, it was no longer possible to use SSL socket auto-close mode and SSL 
sockets now must be managed independently from their underling network socket. 
This allows for greater control over TLS protocol at the price of greater 
complexity.

These changes fix the problem for me locally. The use of 
`MonitoringResponseOutOfOrderStrategy` is not necessary.

Please review / test locally/

Oleg

[1 
]https://github.com/apache/httpcomponents-core/pull/468/commits/fc093c403ef4cbee9e0c6100fabe5a5e2ff73efc
[2] 
[https://github.com/apache/httpcomponents-core/pull/468/commits/3277ffe85c76be0f0c6cefeec4d1b1412d5e256d]

> Request hangs if TLS 1.3 connection is half-closed 
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Minor
> Fix For: 5.3.2, 5.4-alpha3
>
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java, 
> TlsHalfCloseApache4.java, keystore.jks
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> If a server with TLS 1.3 support closes the connection during the request, 
> more specifically, sending close_notify while the client is still writing to 
> socket,  the request will hang indefinitely. It's not an issue with TLS 1.2 
> because it uses duplex-close policy. With TLS 1.3's half-closed connection 
> policy, it seems Apache HTTP client is not able to detect connection closure 
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should 
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see 
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot 
> of users to make changes on their side. 
>  
> Steps to repro:
>  * Download the attached keystore file
>  * Update ksPath in the server code HalfCloseServer.java to where you 
> download the keystore
>  * Run the server, the server will begin listening on {{localhost:8081}}
>  * Create a random file of size 128MB and update client code "testFile" to 
> where the file is.
>  * Run the client, it should hang
>  ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is 
> uncommented, it will not hang
>  ** It also won’t hang if we we force TLS1.2
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2329) BasicHttpClientConnectionManager reuses closed connection objects

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17851808#comment-17851808
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2329:
---

[~ttang] This looks like a duplicate of HTTPCLIENT-2316. Could you please 
upgrade to the latest 5.3.2-SNAPSHOT and verify?

Oleg

> BasicHttpClientConnectionManager reuses closed connection objects
> -
>
> Key: HTTPCLIENT-2329
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2329
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>Affects Versions: 5.3.1
>Reporter: Teresa Tang
>Priority: Major
> Attachments: TestBasicConnectionManager.java
>
>
> In the discardEndpoint method of InternalExecRuntime.java, the endpoint and 
> connection are closed. The manager releases the connection with a timevalue 
> of 0 ms. Because 0 is not considered positive, this leads to the expiration 
> being set to Long.MAX_VALUE. Upon the next connection request, the manager 
> will continue to use this unexpired connection object, even though it is 
> closed. 
>  
> The intention of the 0 ms timevalue was to have the connection expire 
> immediately and then be discarded so that the manager will create a new 
> connection for the next request.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2329) BasicHttpClientConnectionManager reuses closed connection objects

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17851645#comment-17851645
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2329:
---

[~ttang] Could you please put together a test case demonstrating the defect 
like those in our test suite?

https://github.com/apache/httpcomponents-client/blob/master/httpclient5-testing/src/test/java/org/apache/hc/client5/testing/sync/TestBasicConnectionManager.java

Oleg

> BasicHttpClientConnectionManager reuses closed connection objects
> -
>
> Key: HTTPCLIENT-2329
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2329
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>Affects Versions: 5.3.1
>Reporter: Teresa Tang
>Priority: Major
>
> In the discardEndpoint method of InternalExecRuntime.java, the endpoint and 
> connection are closed. The manager releases the connection with a timevalue 
> of 0 ms. Because 0 is not considered positive, this leads to the expiration 
> being set to Long.MAX_VALUE. Upon the next connection request, the manager 
> will continue to use this unexpired connection object, even though it is 
> closed. 
>  
> The intention of the 0 ms timevalue was to have the connection expire 
> immediately and then be discarded so that the manager will create a new 
> connection for the next request.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2329) BasicHttpClientConnectionManager reuses closed connection objects

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17851635#comment-17851635
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2329:
---

[~ttang] Please do specify the exact version of HttpClient you are using. 
Better yet make sure to upgrade to the latest GA release and re-test your code.

Oleg

> BasicHttpClientConnectionManager reuses closed connection objects
> -
>
> Key: HTTPCLIENT-2329
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2329
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>Reporter: Teresa Tang
>Priority: Major
>
> In the discardEndpoint method of InternalExecRuntime.java, the endpoint and 
> connection are closed. The manager releases the connection with a timevalue 
> of 0 ms. Because 0 is not considered positive, this leads to the expiration 
> being set to Long.MAX_VALUE. Upon the next connection request, the manager 
> will continue to use this unexpired connection object, even though it is 
> closed. 
>  
> The intention of the 0 ms timevalue was to have the connection expire 
> immediately and then be discarded so that the manager will create a new 
> connection for the next request.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17851326#comment-17851326
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2328:
---

[~Zoe Wang] I can reproduce the defect with the following code snippet using 
HttpCore. Please note the fix will still require the user to explicitly 
configure the client to use `MonitoringResponseOutOfOrderStrategy`.

{code:java}
SSLContext sslContext = SSLContextBuilder.create()
.loadTrustMaterial(Paths.get("keystore.jks"), "password".toCharArray())
.build();
HttpRequester requester = RequesterBootstrap.bootstrap()
.setSslContext(sslContext)
.setSslSetupHandler(sslParameters -> {
sslParameters.setProtocols(new String[]{TLS.V_1_3.id});
})
.setConnectionFactory(DefaultBHttpClientConnectionFactory.builder()

.responseOutOfOrderStrategy(MonitoringResponseOutOfOrderStrategy.INSTANCE)
.build())
.create();
HttpHost target = new HttpHost("https", "localhost", 8081);
// It needs to be a large file (128mb)
ClassicHttpRequest request = ClassicRequestBuilder.put("/")
.setHttpHost(target)
.setEntity(new PathEntity(Paths.get("output.dat"), 
ContentType.APPLICATION_OCTET_STREAM))
.build();
HttpCoreContext context = HttpCoreContext.create();
try (ClassicHttpResponse response = requester.execute(target, request, 
Timeout.ofMinutes(1), context)) {
EntityUtils.consume(response.getEntity());
}

{code}

> Request hangs if TLS 1.3 connection is half-closed 
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Minor
> Fix For: 5.3.2, 5.4-alpha3
>
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java, 
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request, 
> more specifically, sending close_notify while the client is still writing to 
> socket,  the request will hang indefinitely. It's not an issue with TLS 1.2 
> because it uses duplex-close policy. With TLS 1.3's half-closed connection 
> policy, it seems Apache HTTP client is not able to detect connection closure 
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should 
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see 
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot 
> of users to make changes on their side. 
>  
> Steps to repro:
>  * Download the attached keystore file
>  * Update ksPath in the server code HalfCloseServer.java to where you 
> download the keystore
>  * Run the server, the server will begin listening on {{localhost:8081}}
>  * Create a random file of size 128MB and update client code "testFile" to 
> where the file is.
>  * Run the client, it should hang
>  ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is 
> uncommented, it will not hang
>  ** It also won’t hang if we we force TLS1.2
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCLIENT-2328:
--
Fix Version/s: 5.3.2
   5.4-alpha3
 Priority: Minor  (was: Major)

> Request hangs if TLS 1.3 connection is half-closed 
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Minor
> Fix For: 5.3.2, 5.4-alpha3
>
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java, 
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request, 
> more specifically, sending close_notify while the client is still writing to 
> socket,  the request will hang indefinitely. It's not an issue with TLS 1.2 
> because it uses duplex-close policy. With TLS 1.3's half-closed connection 
> policy, it seems Apache HTTP client is not able to detect connection closure 
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should 
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see 
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot 
> of users to make changes on their side. 
>  
> Steps to repro:
>  * Download the attached keystore file
>  * Update ksPath in the server code HalfCloseServer.java to where you 
> download the keystore
>  * Run the server, the server will begin listening on {{localhost:8081}}
>  * Create a random file of size 128MB and update client code "testFile" to 
> where the file is.
>  * Run the client, it should hang
>  ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is 
> uncommented, it will not hang
>  ** It also won’t hang if we we force TLS1.2
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCORE-766) Serve same content regardless of http authority (RequestHandlerRegistry)

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17851313#comment-17851313
 ] 

Oleg Kalnichevski commented on HTTPCORE-766:


[~erik.wramner] Feel free to review the actual pull request:

[https://github.com/apache/httpcomponents-core/pull/467]

If I hear no complaints or no feedback within 3 to 4 days I will merge the PR 
as is.

Oleg

> Serve same content regardless of http authority (RequestHandlerRegistry)
> 
>
> Key: HTTPCORE-766
> URL: https://issues.apache.org/jira/browse/HTTPCORE-766
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.2.4, 5.3-alpha1, 5.3-alpha2
>Reporter: Erik Wramner
>Priority: Minor
>  Labels: easyfix
>   Original Estimate: 4h
>  Time Spent: 10m
>  Remaining Estimate: 3h 50m
>
> I need to serve the same content on all interfaces and I don't know the IP 
> addresses and host names in advance. Think of it as a classic HTTP server 
> without virtual host support: all requests get to the same set of handlers.
> Unfortunately, ServerBootstrap installs RequestHandlerRegistry, which uses 
> one primary LookupRegistry for a canonical host name and localhost and a map 
> with other registries keyed by name. If a client connects with an unknown 
> name, no handler is found and the server responds with 421.
> There are several possible solutions here:
>  # Modify RequestHandlerRegistry to use the primary registry if no other 
> registry is found. As I recall, that was what Apache did by default. Then it 
> is still possible to register new rules for virtual hosts, but the primary is 
> fallback when no name matches. This could also be configurable on/off.
>  # Add new PrimaryOnlyRequestHandlerRegistry that has no support for virtual 
> hosts and always uses the primary.
>  # Don't fix this in the code, but make it easier for me to fix by making 
> getPatternMatcher protected instead of private.
> In the two last cases we run into another problem: ServerBootstrap has 
> private members without getters, final methods that cannot be overridden and 
> creates RequestHandlerRegistry with new in a long create method. This is the 
> problem that was solved in HTTPCORE-570 for another class. It makes it hard 
> to extend the class, it must basically be rewritten.
> I would propose opening up ServerBootstrap with getters or by making the 
> fields protected. I would also like to extract the RequestHandlerRegistry 
> creation to a protected method that can be overridden. Alternatively, we can 
> add a setter and allow the calling code to prepare an instance outside of the 
> builder.
> I would be happy to make a pull request for these changes on Github. If so, 
> what direction would you like to take?
> I can code around this by copying both classes with my changes, but that 
> feels really dirty.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Comment Edited] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850613#comment-17850613
 ] 

Oleg Kalnichevski edited comment on HTTPCLIENT-2328 at 5/30/24 7:30 AM:


> I just don't see how it's violating HTTP spec. 

[~Zoe Wang] HTTP is defined as a *request / response* based protocol. HTTP 
servers are expected to respond with a response message to a request message, 
If the server does not want the client to continue sending content of the 
request message body it must still respond with an out of sequence response 
message and not just half-close connections in the middle of message exchange. 
That is how.

> The workaround we tried that seems to be working is do a 
> "sock.getInputStream().read(new byte[0]);" before each write, which will 
> check `conContext.isInboundClosed()` and raise SocketException if inbound is 
> closed.

This is precisely what ResponseOutOfOrderStrategy does (which completely 
decimates the i/o performance of the client, by the way). Feel free to use a 
custom implementation or enhance the default one shipped with HttpClient.

Oleg

 


was (Author: olegk):
> I just don't see how it's violating HTTP spec. 

[~Zoe Wang] HTTP is defined as a *request / response* based protocol. HTTP 
servers are expected to respond with a response message to a request message, 
If the server does not want the client to continue sending content of the 
request message body it must still respond with an out of sequence response 
message and not just half-close connections in the middle of message exchange. 
That is how.

> The workaround we tried that seems to be working is do a 
> "sock.getInputStream().read(new byte[0]);" before each write, which will 
> check `conContext.isInboundClosed()` and raise SocketException if inbound is 
> closed.

This is precisely what ResponseOutOfOrderStrategy does. Feel free to use a 
custom implementation or enhance the default one shipped with HttpClient.

Oleg

 

> Request hangs if TLS 1.3 connection is half-closed 
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Major
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java, 
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request, 
> more specifically, sending close_notify while the client is still writing to 
> socket,  the request will hang indefinitely. It's not an issue with TLS 1.2 
> because it uses duplex-close policy. With TLS 1.3's half-closed connection 
> policy, it seems Apache HTTP client is not able to detect connection closure 
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should 
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see 
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot 
> of users to make changes on their side. 
>  
> Steps to repro:
>  * Download the attached keystore file
>  * Update ksPath in the server code HalfCloseServer.java to where you 
> download the keystore
>  * Run the server, the server will begin listening on {{localhost:8081}}
>  * Create a random file of size 128MB and update client code "testFile" to 
> where the file is.
>  * Run the client, it should hang
>  ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is 
> uncommented, it will not hang
>  ** It also won’t hang if we we force TLS1.2
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850613#comment-17850613
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2328:
---

> I just don't see how it's violating HTTP spec. 

[~Zoe Wang] HTTP is defined as a *request / response* based protocol. HTTP 
servers are expected to respond with a response message to a request message, 
If the server does not want the client to continue sending content of the 
request message body it must still respond with an out of sequence response 
message and not just half-close connections in the middle of message exchange. 
That is how.

> The workaround we tried that seems to be working is do a 
> "sock.getInputStream().read(new byte[0]);" before each write, which will 
> check `conContext.isInboundClosed()` and raise SocketException if inbound is 
> closed.

This is precisely what ResponseOutOfOrderStrategy does. Feel free to use a 
custom implementation or enhance the default one shipped with HttpClient.

Oleg

 

> Request hangs if TLS 1.3 connection is half-closed 
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Major
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java, 
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request, 
> more specifically, sending close_notify while the client is still writing to 
> socket,  the request will hang indefinitely. It's not an issue with TLS 1.2 
> because it uses duplex-close policy. With TLS 1.3's half-closed connection 
> policy, it seems Apache HTTP client is not able to detect connection closure 
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should 
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see 
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot 
> of users to make changes on their side. 
>  
> Steps to repro:
>  * Download the attached keystore file
>  * Update ksPath in the server code HalfCloseServer.java to where you 
> download the keystore
>  * Run the server, the server will begin listening on {{localhost:8081}}
>  * Create a random file of size 128MB and update client code "testFile" to 
> where the file is.
>  * Run the client, it should hang
>  ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is 
> uncommented, it will not hang
>  ** It also won’t hang if we we force TLS1.2
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCORE-766) Serve same content regardless of http authority (RequestHandlerRegistry)

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850408#comment-17850408
 ] 

Oleg Kalnichevski commented on HTTPCORE-766:


[~erik.wramner] I put together a new class {{RequestRouter}} [1] that I intend 
to use as a replacement for {{RequestHandlerRegistry}}. The new class provides 
new functionality that may be of interest to you: 1. it can use a custom 
function to normalize or resolve service authorities. 2. when not authoritative 
to handler a request it can delegate the request routing to a downstream 
resolver. The basic use of the new class can be seen here [2]

You are welcome to review the change-set in my branch and propose API 
improvements, better class / method names or ideas for additional functionality.

Oleg

[1] 
https://github.com/ok2c/httpcomponents-core/blob/HTTPCORE-766/httpcore5/src/main/java/org/apache/hc/core5/http/impl/routing/RequestRouter.java
[2] 
https://github.com/ok2c/httpcomponents-core/blob/HTTPCORE-766/httpcore5/src/test/java/org/apache/hc/core5/http/impl/routing/TestRequestRouter.java

> Serve same content regardless of http authority (RequestHandlerRegistry)
> 
>
> Key: HTTPCORE-766
> URL: https://issues.apache.org/jira/browse/HTTPCORE-766
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.2.4, 5.3-alpha1, 5.3-alpha2
>Reporter: Erik Wramner
>Priority: Minor
>  Labels: easyfix
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> I need to serve the same content on all interfaces and I don't know the IP 
> addresses and host names in advance. Think of it as a classic HTTP server 
> without virtual host support: all requests get to the same set of handlers.
> Unfortunately, ServerBootstrap installs RequestHandlerRegistry, which uses 
> one primary LookupRegistry for a canonical host name and localhost and a map 
> with other registries keyed by name. If a client connects with an unknown 
> name, no handler is found and the server responds with 421.
> There are several possible solutions here:
>  # Modify RequestHandlerRegistry to use the primary registry if no other 
> registry is found. As I recall, that was what Apache did by default. Then it 
> is still possible to register new rules for virtual hosts, but the primary is 
> fallback when no name matches. This could also be configurable on/off.
>  # Add new PrimaryOnlyRequestHandlerRegistry that has no support for virtual 
> hosts and always uses the primary.
>  # Don't fix this in the code, but make it easier for me to fix by making 
> getPatternMatcher protected instead of private.
> In the two last cases we run into another problem: ServerBootstrap has 
> private members without getters, final methods that cannot be overridden and 
> creates RequestHandlerRegistry with new in a long create method. This is the 
> problem that was solved in HTTPCORE-570 for another class. It makes it hard 
> to extend the class, it must basically be rewritten.
> I would propose opening up ServerBootstrap with getters or by making the 
> fields protected. I would also like to extract the RequestHandlerRegistry 
> creation to a protected method that can be overridden. Alternatively, we can 
> add a setter and allow the calling code to prepare an instance outside of the 
> builder.
> I would be happy to make a pull request for these changes on Github. If so, 
> what direction would you like to take?
> I can code around this by copying both classes with my changes, but that 
> feels really dirty.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850280#comment-17850280
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2328:
---

> which is not technically compliant to the TCP 1.3 spec (outbound and inbound 
> close_notify should be independent)

[~Zoe Wang] With all due respect what you are saying make no sense to me. 
HTTP/1.1 is a request / response oriented protocol. It defines no semantic 
meaning for half-closed connections. Moreover, the server half-closing its 
endpoint while receiving a request message from the client would essentially be 
in violation of the HTTP spec. The client can half-close its endpoint when 
writing out requests in the pipeline mode. Even that, however, goes counter to 
the spec that states that HTTP/1.1 are to be kept persistent.

Oleg

> Request hangs if TLS 1.3 connection is half-closed 
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Major
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java, 
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request, 
> more specifically, sending close_notify while the client is still writing to 
> socket,  the request will hang indefinitely. It's not an issue with TLS 1.2 
> because it uses duplex-close policy. With TLS 1.3's half-closed connection 
> policy, it seems Apache HTTP client is not able to detect connection closure 
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should 
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see 
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot 
> of users to make changes on their side. 
>  
> Steps to repro:
>  * Download the attached keystore file
>  * Update ksPath in the server code HalfCloseServer.java to where you 
> download the keystore
>  * Run the server, the server will begin listening on {{localhost:8081}}
>  * Create a random file of size 128MB and update client code "testFile" to 
> where the file is.
>  * Run the client, it should hang
>  ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is 
> uncommented, it will not hang
>  ** It also won’t hang if we we force TLS1.2
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849909#comment-17849909
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2328:
---

[~Zoe Wang] What is it _specifically_ you expect me to do? As I tried to 
explain in my previous response the classic i/o can either efficiently read 
from the socket or efficiently write to the socket but not both. If need the 
client to be able to efficiently react to asynchronous events you should 
consider switching to the async transport provided by HttpClient. 
Alternatively, enable `jdk.tls.acknowledgeCloseNotify`. I think this is the 
reason it exists in the first place.

Oleg

> Request hangs if TLS 1.3 connection is half-closed 
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Major
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java, 
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request, 
> more specifically, sending close_notify while the client is still writing to 
> socket,  the request will hang indefinitely. It's not an issue with TLS 1.2 
> because it uses duplex-close policy. With TLS 1.3's half-closed connection 
> policy, it seems Apache HTTP client is not able to detect connection closure 
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should 
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see 
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot 
> of users to make changes on their side. 
>  
> Steps to repro:
>  * Download the attached keystore file
>  * Update ksPath in the server code HalfCloseServer.java to where you 
> download the keystore
>  * Run the server, the server will begin listening on {{localhost:8081}}
>  * Create a random file of size 128MB and update client code "testFile" to 
> where the file is.
>  * Run the client, it should hang
>  ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is 
> uncommented, it will not hang
>  ** It also won’t hang if we we force TLS1.2
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Resolved] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2328.
---
Resolution: Not A Problem

> Request hangs if TLS 1.3 connection is half-closed 
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Major
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java, 
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request, 
> more specifically, sending close_notify while the client is still writing to 
> socket,  the request will hang indefinitely. It's not an issue with TLS 1.2 
> because it uses duplex-close policy. With TLS 1.3's half-closed connection 
> policy, it seems Apache HTTP client is not able to detect connection closure 
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should 
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see 
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot 
> of users to make changes on their side. 
>  
> Steps to repro:
>  * Download the attached keystore file
>  * Update ksPath in the server code HalfCloseServer.java to where you 
> download the keystore
>  * Run the server, the server will begin listening on {{localhost:8081}}
>  * Create a random file of size 128MB and update client code "testFile" to 
> where the file is.
>  * Run the client, it should hang
>  ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is 
> uncommented, it will not hang
>  ** It also won’t hang if we we force TLS1.2
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1784#comment-1784
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2328:
---

[~Zoe Wang] The Java classic i/o model has a well-known limitation: there is no 
_efficient_ way of reading from and writing to the same socket at the same 
time. While the process is busy writing out data it is unable to react to _any_ 
incoming events on the same socket. What one can do is solve the problem is to 
perform an extra short read after each chunk of data written to the socket but 
this has a substantial performance penalty. 

We provide a ResponseOutOfOrderStrategy so one can configure HttpClient to 
monitor connection for out-of-sequence events, but it is disabled (no op) by 
default. It needs to be explicitly enabled.  

{code:java}
final PoolingHttpClientConnectionManager cm = 
PoolingHttpClientConnectionManagerBuilder.create()
.setConnectionFactory(ManagedHttpClientConnectionFactory.builder()

.responseOutOfOrderStrategy(MonitoringResponseOutOfOrderStrategy.INSTANCE)
.build())
.build();
try (CloseableHttpClient httpclient = HttpClients.custom()
.setConnectionManager(cm)
.build()) {

for (final URIScheme uriScheme : URIScheme.values()) {
final ClassicHttpRequest request = ClassicRequestBuilder.get()
.setHttpHost(new HttpHost(uriScheme.id, "httpbin.org"))
.setPath("/headers")
.build();
System.out.println("Executing request " + request);
httpclient.execute(request, response -> {
System.out.println("");
System.out.println(request + "->" + new StatusLine(response));
EntityUtils.consume(response.getEntity());
return null;
});
}
}
{code}

Oleg
 

> Request hangs if TLS 1.3 connection is half-closed 
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Major
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java, 
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request, 
> more specifically, sending close_notify while the client is still writing to 
> socket,  the request will hang indefinitely. It's not an issue with TLS 1.2 
> because it uses duplex-close policy. With TLS 1.3's half-closed connection 
> policy, it seems Apache HTTP client is not able to detect connection closure 
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should 
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see 
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot 
> of users to make changes on their side. 
>  
> Steps to repro:
>  * Download the attached keystore file
>  * Update ksPath in the server code HalfCloseServer.java to where you 
> download the keystore
>  * Run the server, the server will begin listening on {{localhost:8081}}
>  * Create a random file of size 128MB and update client code "testFile" to 
> where the file is.
>  * Run the client, it should hang
>  ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is 
> uncommented, it will not hang
>  ** It also won’t hang if we we force TLS1.2
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCORE-766) Serve same content regardless of http authority (RequestHandlerRegistry)

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847292#comment-17847292
 ] 

Oleg Kalnichevski commented on HTTPCORE-766:


[~erik.wramner] I may end up deprecating the whole handler registration logic 
in favor of a completely new request router. I am still looking into it. At any 
rate one does not need to register any handlers at all in which case all 
requests will get routed to the fallback mapper.

> The workaround where authority is removed by an interceptor probably works, 
> but it feels a bit dangerous. Some other code may need the authority and go 
> south if it is null.

The server does not need to remove request URI authority. it can replace it 
with a valid one it is authoritative for.

Oleg

> Serve same content regardless of http authority (RequestHandlerRegistry)
> 
>
> Key: HTTPCORE-766
> URL: https://issues.apache.org/jira/browse/HTTPCORE-766
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.2.4, 5.3-alpha1, 5.3-alpha2
>Reporter: Erik Wramner
>Priority: Minor
>  Labels: easyfix
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> I need to serve the same content on all interfaces and I don't know the IP 
> addresses and host names in advance. Think of it as a classic HTTP server 
> without virtual host support: all requests get to the same set of handlers.
> Unfortunately, ServerBootstrap installs RequestHandlerRegistry, which uses 
> one primary LookupRegistry for a canonical host name and localhost and a map 
> with other registries keyed by name. If a client connects with an unknown 
> name, no handler is found and the server responds with 421.
> There are several possible solutions here:
>  # Modify RequestHandlerRegistry to use the primary registry if no other 
> registry is found. As I recall, that was what Apache did by default. Then it 
> is still possible to register new rules for virtual hosts, but the primary is 
> fallback when no name matches. This could also be configurable on/off.
>  # Add new PrimaryOnlyRequestHandlerRegistry that has no support for virtual 
> hosts and always uses the primary.
>  # Don't fix this in the code, but make it easier for me to fix by making 
> getPatternMatcher protected instead of private.
> In the two last cases we run into another problem: ServerBootstrap has 
> private members without getters, final methods that cannot be overridden and 
> creates RequestHandlerRegistry with new in a long create method. This is the 
> problem that was solved in HTTPCORE-570 for another class. It makes it hard 
> to extend the class, it must basically be rewritten.
> I would propose opening up ServerBootstrap with getters or by making the 
> fields protected. I would also like to extract the RequestHandlerRegistry 
> creation to a protected method that can be overridden. Alternatively, we can 
> add a setter and allow the calling code to prepare an instance outside of the 
> builder.
> I would be happy to make a pull request for these changes on Github. If so, 
> what direction would you like to take?
> I can code around this by copying both classes with my changes, but that 
> feels really dirty.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCORE-766) Serve same content regardless of http authority (RequestHandlerRegistry)

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847279#comment-17847279
 ] 

Oleg Kalnichevski commented on HTTPCORE-766:


[~erik.wramner] I think what I am going to do is to make the existing server 
bootstraps take `{color:#00}HttpRequestMapper` and use it as a fallback for 
those requests whose authority cannot be resolved.
{color}

Oleg

`

> Serve same content regardless of http authority (RequestHandlerRegistry)
> 
>
> Key: HTTPCORE-766
> URL: https://issues.apache.org/jira/browse/HTTPCORE-766
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.2.4, 5.3-alpha1, 5.3-alpha2
>Reporter: Erik Wramner
>Priority: Minor
>  Labels: easyfix
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> I need to serve the same content on all interfaces and I don't know the IP 
> addresses and host names in advance. Think of it as a classic HTTP server 
> without virtual host support: all requests get to the same set of handlers.
> Unfortunately, ServerBootstrap installs RequestHandlerRegistry, which uses 
> one primary LookupRegistry for a canonical host name and localhost and a map 
> with other registries keyed by name. If a client connects with an unknown 
> name, no handler is found and the server responds with 421.
> There are several possible solutions here:
>  # Modify RequestHandlerRegistry to use the primary registry if no other 
> registry is found. As I recall, that was what Apache did by default. Then it 
> is still possible to register new rules for virtual hosts, but the primary is 
> fallback when no name matches. This could also be configurable on/off.
>  # Add new PrimaryOnlyRequestHandlerRegistry that has no support for virtual 
> hosts and always uses the primary.
>  # Don't fix this in the code, but make it easier for me to fix by making 
> getPatternMatcher protected instead of private.
> In the two last cases we run into another problem: ServerBootstrap has 
> private members without getters, final methods that cannot be overridden and 
> creates RequestHandlerRegistry with new in a long create method. This is the 
> problem that was solved in HTTPCORE-570 for another class. It makes it hard 
> to extend the class, it must basically be rewritten.
> I would propose opening up ServerBootstrap with getters or by making the 
> fields protected. I would also like to extract the RequestHandlerRegistry 
> creation to a protected method that can be overridden. Alternatively, we can 
> add a setter and allow the calling code to prepare an instance outside of the 
> builder.
> I would be happy to make a pull request for these changes on Github. If so, 
> what direction would you like to take?
> I can code around this by copying both classes with my changes, but that 
> feels really dirty.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCORE-766) Serve same content regardless of http authority (RequestHandlerRegistry)

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCORE-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847251#comment-17847251
 ] 

Oleg Kalnichevski commented on HTTPCORE-766:


[~erik.wramner] One way to solve the problem once and for all is to drop or 
rewrite the authority on all incoming requests
{code:java}
public class ClassicServerBootstrapTest {

private static final Timeout TIMEOUT = Timeout.ofMinutes(1);

@RegisterExtension
final HttpServerResource serverResource;

@RegisterExtension
final HttpRequesterResource clientResource;

public ClassicServerBootstrapTest() {
serverResource = new HttpServerResource(URIScheme.HTTP, bootstrap -> 
bootstrap
.setSocketConfig(SocketConfig.custom()
.setSoTimeout(TIMEOUT)
.build())
.setHttpProcessor(HttpProcessors.customServer("test-server")
.add((HttpRequestInterceptor) (request, entity, 
context) -> {
request.setAuthority(null);
})
.build()
)
.register("*", new EchoHandler()));

clientResource = new HttpRequesterResource(bootstrap -> bootstrap
.setSocketConfig(SocketConfig.custom()
.setSoTimeout(TIMEOUT)
.build()));
}

@Test
public void testStuff() throws Exception {
final HttpServer server = serverResource.start();
final HttpRequester requester = clientResource.start();

server.start();
final HttpHost target = new HttpHost("http", "localhost", 
server.getLocalPort());
final HttpCoreContext context = HttpCoreContext.create();
final ClassicHttpRequest request = 
ClassicRequestBuilder.get("http://somewhere.in.pampa/";)
.build();
try (final ClassicHttpResponse response = requester.execute(target, 
request, TIMEOUT, context)) {
assertThat(response.getCode(), 
CoreMatchers.equalTo(HttpStatus.SC_OK));
}
}

}

{code}

I will see if there is a reasonable way of plugging a custom replacement for 
`RequestHandlerRegistry` but generally I do not thin that opening up 
`ServerBootstrap` is a good idea.

Oleg

> Serve same content regardless of http authority (RequestHandlerRegistry)
> 
>
> Key: HTTPCORE-766
> URL: https://issues.apache.org/jira/browse/HTTPCORE-766
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.2.4, 5.3-alpha1, 5.3-alpha2
>Reporter: Erik Wramner
>Priority: Minor
>  Labels: easyfix
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> I need to serve the same content on all interfaces and I don't know the IP 
> addresses and host names in advance. Think of it as a classic HTTP server 
> without virtual host support: all requests get to the same set of handlers.
> Unfortunately, ServerBootstrap installs RequestHandlerRegistry, which uses 
> one primary LookupRegistry for a canonical host name and localhost and a map 
> with other registries keyed by name. If a client connects with an unknown 
> name, no handler is found and the server responds with 421.
> There are several possible solutions here:
>  # Modify RequestHandlerRegistry to use the primary registry if no other 
> registry is found. As I recall, that was what Apache did by default. Then it 
> is still possible to register new rules for virtual hosts, but the primary is 
> fallback when no name matches. This could also be configurable on/off.
>  # Add new PrimaryOnlyRequestHandlerRegistry that has no support for virtual 
> hosts and always uses the primary.
>  # Don't fix this in the code, but make it easier for me to fix by making 
> getPatternMatcher protected instead of private.
> In the two last cases we run into another problem: ServerBootstrap has 
> private members without getters, final methods that cannot be overridden and 
> creates RequestHandlerRegistry with new in a long create method. This is the 
> problem that was solved in HTTPCORE-570 for another class. It makes it hard 
> to extend the class, it must basically be rewritten.
> I would propose opening up ServerBootstrap with getters or by making the 
> fields protected. I would also like to extract the RequestHandlerRegistry 
> creation to a protected method that can be overridden. Alternatively, we can 
> add a setter and allow the calling code to prepare an instance outside of the 
> builder.
> I would be happy to make a pull request for these changes on Github. If so, 
> what direction would you like to take?
> I can code around this by copying both classes with my changes, but that 
> feels really

[jira] [Resolved] (HTTPCLIENT-2327) MemcachedHttpAsyncCacheStorage propagates CancellationException from cancelled cache requests to caller

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski resolved HTTPCLIENT-2327.
---
Fix Version/s: 5.4-alpha3
   Resolution: Fixed

> MemcachedHttpAsyncCacheStorage propagates CancellationException from 
> cancelled cache requests to caller
> ---
>
> Key: HTTPCLIENT-2327
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2327
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpCache
>Reporter: Jovan Attisha
>Priority: Major
> Fix For: 5.4-alpha3
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Spymemcached provides options to handle failure modes when an in-flight 
> request to a Memcached instance fails (for example, when deploying to a 
> Memcached server that is currently serving requests). One of the modes, and 
> the most logical for our scenario in particular, is 
> [Cancel|https://github.com/couchbase/spymemcached/blob/master/src/main/java/net/spy/memcached/FailureMode.java#L51-L54].
>  This, as you would expect, cancels any futures that are in-flight when a 
> Memcached node becomes unreachable.
>  
> The issue that we're having is that the 
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
>  (for restore linked here, as this is our issue, but also for other methods), 
> doesn't handle the {{CancellationException}} gracefully, and instead 
> propagates this Exception up to the caller. Since this is a cache for an HTTP 
> client, I would expect the client to handle the cancellation as a "cache 
> miss" (I would almost expect this behavior for {_}all exceptions{_}, though 
> I'm likely not thinking of some edge cases here) and proceed to make the 
> request to the resource originally requested, since the cache is a mere 
> optimization, and a failure in the cache should not impact the actual request 
> to the resource we're targeting.
>  
> We did POC this by extending the 
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
>  and overriding the {{restore}} method to explicitly handle 
> {{{}CancellationException{}}}s and complete the future to treat it as a cache 
> miss, but before we proceed with this, we want to understand if:
> 1. The Apache HTTPClient agrees that this is the desired behavior
> 2. If so, would the Apache HTTP Client accept this behavior in the client 
> itself, rather than us extending to override this behavior



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2327) MemcachedHttpAsyncCacheStorage propagates CancellationException from cancelled cache requests to caller

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846671#comment-17846671
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2327:
---

[~jattisha] I agree. Fell free to submit a PR at GitHub with the change to that 
effect.

Oleg

> MemcachedHttpAsyncCacheStorage propagates CancellationException from 
> cancelled cache requests to caller
> ---
>
> Key: HTTPCLIENT-2327
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2327
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpCache
>Reporter: Jovan Attisha
>Priority: Major
>
> Spymemcached provides options to handle failure modes when an in-flight 
> request to a Memcached instance fails (for example, when deploying to a 
> Memcached server that is currently serving requests). One of the modes, and 
> the most logical for our scenario in particular, is 
> [Cancel|https://github.com/couchbase/spymemcached/blob/master/src/main/java/net/spy/memcached/FailureMode.java#L51-L54].
>  This, as you would expect, cancels any futures that are in-flight when a 
> Memcached node becomes unreachable.
>  
> The issue that we're having is that the 
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
>  (for restore linked here, as this is our issue, but also for other methods), 
> doesn't handle the {{CancellationException}} gracefully, and instead 
> propagates this Exception up to the caller. Since this is a cache for an HTTP 
> client, I would expect the client to handle the cancellation as a "cache 
> miss" (I would almost expect this behavior for {_}all exceptions{_}, though 
> I'm likely not thinking of some edge cases here) and proceed to make the 
> request to the resource originally requested, since the cache is a mere 
> optimization, and a failure in the cache should not impact the actual request 
> to the resource we're targeting.
>  
> We did POC this by extending the 
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
>  and overriding the {{restore}} method to explicitly handle 
> {{{}CancellationException{}}}s and complete the future to treat it as a cache 
> miss, but before we proceed with this, we want to understand if:
> 1. The Apache HTTPClient agrees that this is the desired behavior
> 2. If so, would the Apache HTTP Client accept this behavior in the client 
> itself, rather than us extending to override this behavior



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Comment Edited] (HTTPCLIENT-2327) MemcachedHttpAsyncCacheStorage propagates CancellationException from cancelled cache requests to caller

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846643#comment-17846643
 ] 

Oleg Kalnichevski edited comment on HTTPCLIENT-2327 at 5/15/24 2:10 PM:


[~jattisha] From the design perspective the decision whether or not a cache 
failure should be treated as a cache miss is to be taken by the protocol layer, 
and _not_ by the storage backend. Unless I am missing something here the 
protocol layer already treats `ResourceIOException` as a cache miss:

https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/BasicHttpAsyncCache.java#L160

What am I missing?

Oleg


was (Author: olegk):
[~jattisha] From the design perspective the decision whether or not a cache 
failure should be treated as a cache miss is to be taken by the protocol layer, 
and _not_ by the storage backend. Unless I am missing something here the 
protocol layer already treats `ResourceIOException` as a cache miss:

[https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/BasicHttpAsyncCache.java#L130]

What am I missing?

Oleg

> MemcachedHttpAsyncCacheStorage propagates CancellationException from 
> cancelled cache requests to caller
> ---
>
> Key: HTTPCLIENT-2327
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2327
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpCache
>Reporter: Jovan Attisha
>Priority: Major
>
> Spymemcached provides options to handle failure modes when an in-flight 
> request to a Memcached instance fails (for example, when deploying to a 
> Memcached server that is currently serving requests). One of the modes, and 
> the most logical for our scenario in particular, is 
> [Cancel|https://github.com/couchbase/spymemcached/blob/master/src/main/java/net/spy/memcached/FailureMode.java#L51-L54].
>  This, as you would expect, cancels any futures that are in-flight when a 
> Memcached node becomes unreachable.
>  
> The issue that we're having is that the 
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
>  (for restore linked here, as this is our issue, but also for other methods), 
> doesn't handle the {{CancellationException}} gracefully, and instead 
> propagates this Exception up to the caller. Since this is a cache for an HTTP 
> client, I would expect the client to handle the cancellation as a "cache 
> miss" (I would almost expect this behavior for {_}all exceptions{_}, though 
> I'm likely not thinking of some edge cases here) and proceed to make the 
> request to the resource originally requested, since the cache is a mere 
> optimization, and a failure in the cache should not impact the actual request 
> to the resource we're targeting.
>  
> We did POC this by extending the 
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
>  and overriding the {{restore}} method to explicitly handle 
> {{{}CancellationException{}}}s and complete the future to treat it as a cache 
> miss, but before we proceed with this, we want to understand if:
> 1. The Apache HTTPClient agrees that this is the desired behavior
> 2. If so, would the Apache HTTP Client accept this behavior in the client 
> itself, rather than us extending to override this behavior



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2327) MemcachedHttpAsyncCacheStorage propagates CancellationException from cancelled cache requests to caller

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846643#comment-17846643
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2327:
---

[~jattisha] From the design perspective the decision whether or not a cache 
failure should be treated as a cache miss is to be taken by the protocol layer, 
and _not_ by the storage backend. Unless I am missing something here the 
protocol layer already treats `ResourceIOException` as a cache miss:

[https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/BasicHttpAsyncCache.java#L130]

What am I missing?

Oleg

> MemcachedHttpAsyncCacheStorage propagates CancellationException from 
> cancelled cache requests to caller
> ---
>
> Key: HTTPCLIENT-2327
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2327
> Project: HttpComponents HttpClient
>  Issue Type: Bug
>  Components: HttpCache
>Reporter: Jovan Attisha
>Priority: Major
>
> Spymemcached provides options to handle failure modes when an in-flight 
> request to a Memcached instance fails (for example, when deploying to a 
> Memcached server that is currently serving requests). One of the modes, and 
> the most logical for our scenario in particular, is 
> [Cancel|https://github.com/couchbase/spymemcached/blob/master/src/main/java/net/spy/memcached/FailureMode.java#L51-L54].
>  This, as you would expect, cancels any futures that are in-flight when a 
> Memcached node becomes unreachable.
>  
> The issue that we're having is that the 
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
>  (for restore linked here, as this is our issue, but also for other methods), 
> doesn't handle the {{CancellationException}} gracefully, and instead 
> propagates this Exception up to the caller. Since this is a cache for an HTTP 
> client, I would expect the client to handle the cancellation as a "cache 
> miss" (I would almost expect this behavior for {_}all exceptions{_}, though 
> I'm likely not thinking of some edge cases here) and proceed to make the 
> request to the resource originally requested, since the cache is a mere 
> optimization, and a failure in the cache should not impact the actual request 
> to the resource we're targeting.
>  
> We did POC this by extending the 
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
>  and overriding the {{restore}} method to explicitly handle 
> {{{}CancellationException{}}}s and complete the future to treat it as a cache 
> miss, but before we proceed with this, we want to understand if:
> 1. The Apache HTTPClient agrees that this is the desired behavior
> 2. If so, would the Apache HTTP Client accept this behavior in the client 
> itself, rather than us extending to override this behavior



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2326) Support any Exception in RetryStrategy

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838582#comment-17838582
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2326:
---

> unnelRefusedException should be an IOException, or it should be encapsulated 
> in an IOException

 

[~ofrias] No, it should not.

Please test this change-set:

[https://github.com/ok2c/httpcomponents-client/commit/888f9b1b5ae7f3ca02d429a9d1e2db59a62a412e]

Oleg

> Support any Exception in RetryStrategy
> --
>
> Key: HTTPCLIENT-2326
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2326
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>Reporter: Oscar
>Priority: Minor
>
> It would be useful to modify {{HttpRequestRetryExec}} to allow 
> {{HttpRequestRetryStrategy}} to support all kind of exceptions, not only 
> those extending {{{}IOException{}}}.
> I would like HttpClient to retry requests when facing a 
> [TunnelRefusedException|https://hc.apache.org/httpcomponents-client-5.1.x/current/httpclient5/apidocs/org/apache/hc/client5/http/impl/TunnelRefusedException.html]
>  (it happens randomly that the proxy refuses the initial connection, but it 
> usually works ok when retrying).
> I have tried extending 
> [DefaultHttpRequestRetryStrategy|https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/main/java/org/apache/hc/client5/http/impl/DefaultHttpRequestRetryStrategy.java]
>  but it can only capture exceptions inheriting from {{IOException.}}
> So, maybe {{HttpRequestRetryStrategy}} interface should support any kind of 
> {{Exception }} or maybe {{TunnelRefusedException}} should be an 
> {{IOException}}
>  
> This is an example of a stacktrace that I would like to retry:
>  
> {code:java}
> org.apache.hc.client5.http.ClientProtocolException: CONNECT refused by proxy: 
> HTTP/1.1 500 Internal Server Error
> at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173)
> at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
> at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
> [...]
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: org.apache.hc.client5.http.impl.TunnelRefusedException: CONNECT 
> refused by proxy: HTTP/1.1 500 Internal Server Error
> at 
> org.apache.hc.client5.http.impl.classic.ConnectExec.createTunnelToTarget(ConnectExec.java:284)
> at 
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:151)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2326) Support any Exception in RetryStrategy

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838559#comment-17838559
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2326:
---

[~ofrias] My idea is to modify `TunnelRefusedExcepcion` to include the original 
HTTP response message that `HttpRequestRetryStrategy#retryRequest` can act 
upon. A custom `HttpRequestRetryStrategy` implementation in your code  should 
then be able to decide whether to retry the request based on the proxy response 
rather than based on an I/O (or any other) exception.

Oleg

> Support any Exception in RetryStrategy
> --
>
> Key: HTTPCLIENT-2326
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2326
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>Reporter: Oscar
>Priority: Minor
>
> It would be useful to modify {{HttpRequestRetryExec}} to allow 
> {{HttpRequestRetryStrategy}} to support all kind of exceptions, not only 
> those extending {{{}IOException{}}}.
> I would like HttpClient to retry requests when facing a 
> [TunnelRefusedException|https://hc.apache.org/httpcomponents-client-5.1.x/current/httpclient5/apidocs/org/apache/hc/client5/http/impl/TunnelRefusedException.html]
>  (it happens randomly that the proxy refuses the initial connection, but it 
> usually works ok when retrying).
> I have tried extending 
> [DefaultHttpRequestRetryStrategy|https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/main/java/org/apache/hc/client5/http/impl/DefaultHttpRequestRetryStrategy.java]
>  but it can only capture exceptions inheriting from {{IOException.}}
> So, maybe {{HttpRequestRetryStrategy}} interface should support any kind of 
> {{Exception }} or maybe {{TunnelRefusedException}} should be an 
> {{IOException}}
>  
> This is an example of a stacktrace that I would like to retry:
>  
> {code:java}
> org.apache.hc.client5.http.ClientProtocolException: CONNECT refused by proxy: 
> HTTP/1.1 500 Internal Server Error
> at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173)
> at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
> at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
> [...]
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: org.apache.hc.client5.http.impl.TunnelRefusedException: CONNECT 
> refused by proxy: HTTP/1.1 500 Internal Server Error
> at 
> org.apache.hc.client5.http.impl.classic.ConnectExec.createTunnelToTarget(ConnectExec.java:284)
> at 
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:151)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Commented] (HTTPCLIENT-2326) Support any Exception in RetryStrategy

[Oleg Kalnichevski (Jira)]

[ 
https://issues.apache.org/jira/browse/HTTPCLIENT-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838539#comment-17838539
 ] 

Oleg Kalnichevski commented on HTTPCLIENT-2326:
---

[~ofrias] What I would like  to propose is that `TunnelRefusedException ` would 
get routed to the `HttpRequestRetryStrategy#retryRequest` that takes 
`HttpResponse` as a parameter.

Oleg

> Support any Exception in RetryStrategy
> --
>
> Key: HTTPCLIENT-2326
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2326
> Project: HttpComponents HttpClient
>  Issue Type: Improvement
>Reporter: Oscar
>Priority: Minor
>
> It would be useful to modify {{HttpRequestRetryExec}} to allow 
> {{HttpRequestRetryStrategy}} to support all kind of exceptions, not only 
> those extending {{{}IOException{}}}.
> I would like HttpClient to retry requests when facing a 
> [TunnelRefusedException|https://hc.apache.org/httpcomponents-client-5.1.x/current/httpclient5/apidocs/org/apache/hc/client5/http/impl/TunnelRefusedException.html]
>  (it happens randomly that the proxy refuses the initial connection, but it 
> usually works ok when retrying).
> I have tried extending 
> [DefaultHttpRequestRetryStrategy|https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/main/java/org/apache/hc/client5/http/impl/DefaultHttpRequestRetryStrategy.java]
>  but it can only capture exceptions inheriting from {{IOException.}}
> So, maybe {{HttpRequestRetryStrategy}} interface should support any kind of 
> {{Exception }} or maybe {{TunnelRefusedException}} should be an 
> {{IOException}}
>  
> This is an example of a stacktrace that I would like to retry:
>  
> {code:java}
> org.apache.hc.client5.http.ClientProtocolException: CONNECT refused by proxy: 
> HTTP/1.1 500 Internal Server Error
> at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173)
> at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
> at 
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
> [...]
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: org.apache.hc.client5.http.impl.TunnelRefusedException: CONNECT 
> refused by proxy: HTTP/1.1 500 Internal Server Error
> at 
> org.apache.hc.client5.http.impl.classic.ConnectExec.createTunnelToTarget(ConnectExec.java:284)
> at 
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:151)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
> at 
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at 
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org

[jira] [Updated] (HTTPCORE-759) Content-Length missing on null entity request content

[Oleg Kalnichevski (Jira)]

 [ 
https://issues.apache.org/jira/browse/HTTPCORE-759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Kalnichevski updated HTTPCORE-759:
---
Fix Version/s: (was: 5.2.4)

> Content-Length missing on null entity request content
> -
>
> Key: HTTPCORE-759
> URL: https://issues.apache.org/jira/browse/HTTPCORE-759
> Project: HttpComponents HttpCore
>  Issue Type: Improvement
>  Components: HttpCore
>Affects Versions: 5.2.2
>Reporter: Billy Jaime Beltran
>Priority: Minor
>  Labels: easyfix
> Fix For: 5.3-alpha1
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> When making a POST request with an empty body, the following difference in 
> behaviour was observed.
> In HttpComponents 4.4.x, a null body causes a Content-Length zero header to 
> be added.
> In HttpComponents 5.2.x, a null body causes no Content-Length headers being 
> added.
> While upgrading an Apache Camel application, we noticed that this breaks 
> calls to an upstream IIS server which replies with 411 (Length Required).
> That server expects either a Content-Length 0 or a Transfer-Encoding: chunked 
> header (with a chunk-size of zero) on requests that have a body semantic 
> (POST, PUT).
> As the code currently stands written, if we manually set `Content-Length: 0` 
> a ProtocolException is thrown 
> [RequestContent.java:106|https://github.com/apache/httpcomponents-core/blob/e3c770b55602eb9e5a45dfe7c6a07a1adede2c95/httpcore5/src/main/java/org/apache/hc/core5/http/protocol/RequestContent.java#L106]
> and if we call the constructor with overwrite, this header is removed anyway 
> without being replaced. 
> The previous behaviour (4.x) is documented 
> [RequestContent.java:102|https://github.com/apache/httpcomponents-core/blob/a5c117028b7c620974304636d52f06f172f1d08b/httpcore/src/main/java/org/apache/http/protocol/RequestContent.java#L102]
>  
> A suggested fix: re-add the null check and set Content-Length to zero.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org